Cloud Foundry provides a consistent model for deploying and running applications across multiple clouds. This multi-cloud approach preserves developer choice and flexibility, both today and in the future.

Following on the heels of the BOSH hackathon at the recent OpenStack conference, Piston Cloud today announced plans to distribute and support Cloud Foundry on OpenStack, joining existing BOSH support for vSphere and Amazon Web Services. Piston Cloud will offer this new integrated capability in a future release of Piston Enterprise OS and the new project will be submitted to the OpenStack satellite ecosystem for future consideration as an OpenStack incubation project. Details on the announcement are available in the official press release.

The OpenStack BOSH CPI, like both OpenStack and Cloud Foundry itself, is an open source project under the Apache 2 license. User and developer community contributions are welcomed, and patches can be submitted at https://github.com/piston/openstack-bosh-cpi.

This effort is still in the early stages, but with core Nova team members @0×44 and @(jk0) leading the effort, and @vadimspivak and the Cloud Foundry team on call, rapid progress is expected.

The Piston Cloud project joins other community contributions around Cloud Foundry BOSH including the work being done by Nic Williams of Engine Yard. See his recent blog on his experiences with BOSH, as well as his public talk this Wednesday (May 2, 2012).  Join in on the updates and conversations in the Cloud Foundry BOSH user group.

Earlier this month we moved to a new open source contribution process for Cloud Foundry.  As part of the new process, we also want to share more information about what code is coming in the future.  This post is the first in what will be a regular series on the Cloud Foundry roadmap.

During the Cloud Foundry Anniversary event we made a point to call out that 80% of our work is really “below the water line”. We are doing a lot of work on the core infrastructure, and only a small fraction of what we do surfaces itself as a visible feature.

For those of you watching the repos, I want to give you a little context on some of the pieces that are sitting around in the code, or are in the process of being added.

We have been working on a major deconstruction of the core cloud controller (see the event slides 7-9). This involves systematically removing responsibility from the cloud controller and moving these pieces into independently scalable components, which run with different levels of isolation. If terms like cloud controller, router, and dea are new to you, review this presentation.

If you look closely at the current code base you can see the new User Account and Authentication (UAA) component. We’ve been validating UAA on CloudFoundry.com where it is performing authentication for a subset of the accounts. The UAA code has been in the public GitHub repository for several months, and we have rolled pieces of it incrementally into CloudFoundry.com in phased deployments. We have one more turn of the deployment crank and when this is done, UAA will be the source of all authentication, and the old authentication system embedded in cloud controller can be removed. The UAA component is key for us as we start enabling more advanced forms of authentication and integration.

The deconstruction effort is nearly complete and with that behind us, the last major new replacement component is landing in the code base as we speak. What’s left of the old cloud controller is being replaced with an all new system.

Over the next several days you will see the “cloud_controller_ng” repo appear and will see the first batch of commits. Architecturally, the new cloud controller will adopt the more traditional Sinatra/Sequel framework used in several other components. Functionally, the new cloud controller exposes a new set of objects that provide additional scoping and sharing semantics designed to support operational collaboration and advanced, pooled quota controls. Watch for this repo, and then carefully read the new model and note the new “org” and “app-space” objects.

In the future, this sort of development will happen directly in the Gerrit-based open repos. Moving in code at this stage of development is not our normal mode of operation, but during the final steps of the transition, we have a handful of repos whose move is still in-flight.

The next major component to watch for is the next-generation vmc client. The vmc gem exposes the core cloud foundry API as a simple-to-use set of ruby objects. Unfortunately, the way the API is exposed, some of the key functionality (create and update applications) is poorly exposed through the object model. You literally have to cheat and use pieces of the cli class hierarchy in order to use the gem with these functions. The NG version of vmc addresses this with a completely clean and well-factored object model. In addition, it’s made major improvements in its approach to extensibility. Its approach to scripting and integration is well thought out. And finally it eliminates the use of fake tables in its output.

I’ll do my best to keep you updated as there is a lot of activity in the code base. On your own though, the best way to stay up to date and engaged is to join the code review discussions or the project-level discussions on vcap or BOSH.

-markl

Mark Lucovsky, VP of Engineering – Cloud Foundry

This is the third post of a series of guest blogs by application developers. We are featuring a use case by Romin Irani, an individual developer who built APIs for valuable data sets in the City of Mumbai as a public service and hosted them on Cloud Foundry.

Guest blog by Romin Irani, Mind Storm Software

Access to world-class infrastructure coupled with the pay-per-use model has been a dream come true for start-ups and individual developers. For an individual developer and entrepreneur like me, the way I develop and make my applications available for others to use or evaluate has been permanently transformed.

Mumbai Public Data APIs

It has always been difficult for me to get various data sets for the city that I live in, i.e., Mumbai, India. Mumbai has a population of about 20 million people living in 233 square miles (603 sq. km.). Like many urban centers in developing countries, the city is growing faster than its government is able to provide services and infrastructure for its citizens.

If provided some of the city’s public data sets with APIs for those data, developers can come up with a variety of applications to harness that information and build applications to benefit the city residents. So I took the small step of building out a few data sets, such as:

1. Help lines - Essential services like hospital phone numbers with locations and police emergency numbers. Mumbai has no central emergency telephone number like 9-1-1, so essential service numbers are helpful to have in one place.
2. Blood banks - Nearly 60 city blood banks with phone numbers which can be looked up by their location, although much of the stocking and inventory-keeping is still done manually.
3. Taxi and auto rates – Taxi/auto rate card fares as published by the Mumbai Administration for residents and visitors to correctly determine their meter charges and fares.

City of Mumbai Public Data Sets on Cloud Foundry

This type of scenario works very well with MongoDB.  The application is written in Node.js, interacts with the Cloud Foundry MongoDB service and provides JSON data for the various Mumbai data sets. Over time, I hope to keep introducing new data sets. To develop their own web, desktop or mobile clients, developers can simply invoke the REST Endpoints for each data set.

The data set is exposed via a REST API that is hosted on CloudFoundry.com:  http://mumbaidata.cloudfoundry.com

Why is Cloud Foundry is my preferred PaaS?

The impetus to learn more about Cloud Foundry came via CloudSpokes, a community-driven marketplace where developers take up programming challenges and get paid for that work. I have submitted code for more than six challenges now including some winning entries–all on Cloud Foundry.

These include a Node.js stock ticker, a salesforce.com Login Component, a Web page PDF-to-Box bookmarklet, and a salesforce.com data exporter that generated Word files from salesforce.com data–all of which are hosted and running great on Cloud Foundry. Top reasons why Cloud Foundry is my preferred PaaS:

• Cloud Foundry provides a non-intrusive model for my Java applications. When I want to run a Java Web application that uses my home-grown MVC framework or use the Spring framework, it is all supported. I am no longer looking for workarounds.
• It is easy to use, especially with the STS plug-in for Eclipse which makes developing and deploying with Cloud Foundry a breeze. The fact that you can be in a single environment to develop and deploy to the cloud is very nice! That coupled with being able to start and stop the applications, inspect the logs and even deploy to the Micro Cloud Foundry definitely boosts productivity. This is in sharp contrast to moving in and out of multiple environments for each stage of developing, testing and deploying your applications.
• Also, I have access to services like MongoDB, RabbitMQ and many others that I can quickly integrate into my applications, as well as use the programming stack that is appropriate for the job.

What’s next?

One of my goals over the last year has been to develop a hands-on cloud computing course and I have struggled with the choice of a PaaS platform that would allow programmers with different language skills to all come under one roof and go through the course. With its polyglot nature, I hope to build that course out soon with Cloud Foundry as the engine running all the samples. The fact that I can even run locally via Micro Cloud Foundry is a huge plus.

I truly believe that the polyglot programmer has arrived and an open PaaS like Cloud Foundry is a great enabler to making that a reality.

Signup for Cloud Foundry today, and start building your own cool app!

Romin is a published author of several books and articles, and contributes regularly to ProgrammableWeb, where he focuses on API news. He currently runs a startup, Mind Storm Software, which is focused on quality hands-on training on cloud computing platforms and mobile development.

The Cloud Foundry development team presented Cloud Foundry BOSH at the OpenStack Design Summit on Wednesday in San Francisco, followed by a hackathon focused on adding OpenStack support. BOSH is an open source tool chain for release engineering, deployment and lifecycle management of large-scale distributed services. Cloud Foundry is designed to give customers a choice of clouds, and BOSH supports multiple underlying cloud infrastructure, including vSphere and AWS. BOSH has been used to deploy and manage CloudFoundry.com since its launch and is particularly useful for operating production instances of Cloud Foundry.

Our goals in participating in the OpenStack event were to meet key people in the OpenStack community and learn about OpenStack while explaining the core ideas behind BOSH and kicking off the implementation of BOSH support for OpenStack.

Vadim Spivak (@vadimspivak) kicked things off with a well-attended talk at 3pm. The slides are embedded at the bottom of this post. He first discussed the motivations for building BOSH and the goals for the project, as well as outlining its role in deploying and managing CloudFoundry.com. He then quickly delved into the details, outlining some of the artifacts used in BOSH as well as detailing the architecture of a full-scale BOSH deployment. Documentation for BOSH is available on Github.

The Cloud Provider Interface (CPI) abstracts the underlying virtualized infrastructure from the rest of BOSH, providing support for a variety of different infrastructure platforms. Today the project includes two implementations of the CPI, with production class support for vSphere and work-in-progress support for AWS. With the goal of kick-starting BOSH support for OpenStack, we held a hackathon at 6pm in the developer lounge focusing on an OpenStack CPI implementation. We provided food and alcohol to keep everyone fully fueled and sat down to work. With small groups clustered around a number of key members of the Cloud Foundry development team we started digging into the code.

By the end of the evening we had made good progress. In particular Vishvananda (@vish), working with Cloud Foundry’s Oleg Shaldybin (@olegshaldybin), was able to repurpose the existing AWS implementation to kick off a successful VM build on OpenStack. Fantastic progress!

We greatly enjoyed our day with the OpenStack community, we’d like to thank you all for your great welcome and interest in BOSH. A lot of folks helped make this happen. In particular, we’d like to thank Andrew Shafer (@littleidea), Lloyd Dewolf (@lloyddewolf), Josh McKenty (@jmckenty), Scott Sanchez (@scottsanchez) and Lauren Sell for helping to get this organized.

We really appreciate everyone’s interest and look forward to continuing to work with the OpenStack community for a long time to come.

– The Cloud Foundry Team.

JRuby Rails applications can be deployed to CloudFoundry.com today with simple configuration changes. JRuby applications are commonly deployed to servlet containers by creating a .war file that contains the Rails app. We will do the same for Cloud Foundry with some changes to the database configuration, so the application can also access a database service on CloudFoundry.com.

Changes needed for deploying JRuby on Rails Applications to Cloud Foundry

There are two tasks we need to accomplish in order to get a JRuby application running on CloudFoundry.com. First we need to configure the application to connect to the database service on CloudFoundry.com by modifying the database.yml file in the configuration directory. We also need to run the equivalent of ‘rake db:migrate', when we deploy the application so that the database tables are created. We can do this by adding an initializer in the config/initializers directory.

The information we need to configure the database connection is available in the environment variable, VCAP_SERVICES. We could either parse that variable programmatically or use the convenient Cloud Foundry run-time gem (see Using Cloud Foundry Services with Ruby: Part 2 – Run-time Support for Ruby Applications blogpost), which is what we will do here. To use this gem we need to include it in our Gemfile:

...
gem  'cf-runtime'
...

Now that we have added this gem, we can add some code snippets to the database.yml file to access the database service information for the production environment. The following is the production portion from a database.yml file, where we are using a MySQL database:

config/database.yml

production:
  adapter: mysql
  <% require 'cfruntime/properties' %>
  <% db_svc = CFRuntime::CloudApp.service_props('mysql') %>
  database: <%= db_svc[:database] rescue 'bookshelf_production' %>
  username: <%= db_svc[:username] rescue 'root' %>
  password: <%= db_svc[:password] rescue '' %>
  host: <%= db_svc[:host] rescue 'localhost' %>
  port: <%= db_svc[:port] rescue '3306' %>

As you can see, we added a require statement for cfruntime/properties, and then we get a hash of the service properties by calling the service_props method passing in the type of service we are using. If there is only one service of that type bound to the application, then there is no need to specify the actual name of the service. You will need to specify the actual service name if you bind multiple services of the same type to your app. The hash of service properties is stored in a variable called db_svc, and code extracts the corresponding values to be used for database, username, password, host, and port. Each of these statements have a rescue clause that provides values to use if we are not operating in a Cloud Foundry environment, in which case the db_svc would be nil.

Alternatively, the production portion of the database.yml file would look like this for PostgreSQL:

production:
  adapter: postgresql
  encoding: unicode
  <% require 'cfruntime/properties' %>
  <% db_svc = CFRuntime::CloudApp.service_props('postgresql') %>
  database: <%= db_svc[:database] rescue 'bookshelf_production' %>
  username: <%= db_svc[:username] rescue 'bookshelf' %>
  password: <%= db_svc[:password] rescue '' %>
  host: <%= db_svc[:host] rescue 'localhost' %>
  port: <%= db_svc[:port] rescue '5432' %>

Next, we turn our attention to the creation of the tables we need for our app. For this to happen, we need to add the following initializer to the config/initializers directory when we deploy the app. I named this initializer cf_db_migrate.rb:

config/initializers/cf_db_migrate.rb

require 'cfruntime/properties'

# Run the equivalent of rake db:migrate on startup
if CFRuntime::CloudApp.running_in_cloud?
  migrations = Rails.root.join('db','migrate')
  if migrations.directory?
    ActiveRecord::Migrator.migrate(migrations)
  end
end

We use cfruntime/properties again and check to see that we are running in the cloud. Next we check to see if the db/migrate directory exists and if it does we run the database migration using the migration files in the directory ( ActiveRecord::Migrator.migrate(migrations) ).

One additional change we have to make is to the warble configuration. It does not include the db/migrate directory in the generated war file by default, so we need to add this to the configuration by specifying config.includes = FileList["db/migrate/*"]. Here are the relevant contents of the config/warble.rb file:

config/warble.rb

# Warbler web application assembly configuration file
Warbler::Config.new do |config|

  # Application directories to be included in the webapp.
  config.dirs = %w(app config lib log vendor tmp)

  # Additional files/directories to include, above those in config.dirs
  config.includes = FileList["db/migrate/*"]

end

A complete example

This assumes that you have a working JRuby environment with Rails, Warbler and MySQL gems already installed.

We have seen above what changes are needed, so let’s quickly generate a Rails app and make the required changes and deploy the app to CloudFoundry.com. If you don’t already have JRuby installed a good place to start is Getting Started with JRuby.

Create the JRuby Rails App

First we create the new app and create the first domain object with full scaffolding.

jruby -S rails new bookshelf -d mysql
cd bookshelf
jruby -S rails generate scaffold Book title:string category:string published:integer price:decimal{10.2} isbn:string

Next we remove the generated public/index.html and modify the config/routes.rb to use ‘books’ as the root:

rm public/index.html
vi config/routes.rb

Here is the route that I added in config/routes.rb:

Bookshelf::Application.routes.draw do
  resources :books

  # You can have the root of your site routed with "root"
  # just remember to delete public/index.html.
  # root :to => 'welcome#index'
  root :to => 'books#index'

  # See how all your routes lay out with "rake routes"

end

Now we will run this app locally to make sure it works:

jruby -S rake db:create
jruby -S rake db:migrate
jruby -S rails server

The Rails empty list view of the Book entity shows that this is indeed working. Now I can add new books to my bookshelf.

Modify the JRuby Rails App for CloudFoundry deployment

Let’s start by making the following changes that we mentioned above:

• add the gem cf-runtime to the Gemfile
• modify the “production:” section of the config/database.yml file as shown above
• add a file named config/initializers/cf_db_migrate.rb with the content shown above

Next we need to generate the Warbler config file so we run:

jruby -S warble config

Now we can:

• modify the config/warble.rb to add the db/migrate directory as shown above

Those are all the changes needed and we are now ready to package and deploy this app.

Package and deploy the JRuby Rails App to CloudFoundry

We will use Warbler to package the app into a war and the CloudFoundry vmc command line utility to deploy it.

The process we use to package the application into a war file is a simple: bundle, pre-compile assets and run Warbler:

jruby -S bundle install
jruby -S rake assets:precompile
jruby -S warble

This creates a bookshelf.war in the root directory of our Rails app. At the moment there are some issues running the vmc command with JRuby but we are working on a fix for this. In the meantime we can move the war file to another directory, so that I can easier switch to use regular “C” Ruby. I’ll create a ‘deploy‘ directory and configure that to use Ruby 1.9.2-p290 (I’m using rbenv, but you could use RVM as well):

mkdir deploy
mv bookshelf.war deploy/.
cd deploy
rbenv local 1.9.2-p290
# (if you use RVM the command should be 'rvm ruby-1.9.2-p290')

Now we are ready to log in to CloudFoundry and deploy our application. For this part you need to have vmc installed.

vmc target api.cloudfoundry.com
vmc login cloud@mycompany.com
Password: *****
Successfully logged into [http://api.cloudfoundry.com]
vmc push bookshelf
Would you like to deploy from the current directory? [Yn]: Y
Application Deployed URL [bookshelf.cloudfoundry.com]: mybookshelf.cloudfoundry.com
Detected a Java Web Application, is this correct? [Yn]: Y
Memory reservation (128M, 256M, 512M, 1G, 2G) [512M]: 512M
How many instances? [1]: 1
Bind existing services to 'bookshelf'? [yN]: N
Create services to bind to 'bookshelf'? [yN]: Y
1: mongodb
2: mysql
3: postgresql
4: rabbitmq
5: redis
What kind of service?: 2
Specify the name of the service [mysql-a4fd7]: mysql-books
Create another? [yN]: N
Would you like to save this configuration? [yN]: N
Creating Application: OK
Creating Service [mysql-books]: OK
Binding Service [mysql-books]: OK
Uploading Application:
  Checking for available resources: OK
  Processing resources: OK
  Packing application: OK
  Uploading (707K): OK
Push Status: OK
Staging Application 'bookshelf': OK
Starting Application 'bookshelf': OK

The vmc commands are highlighted above. Most defaults have been accepted except for the URL and whether a service should be created. I used the URL ‘mybookshelf.cloudfoundry.com‘ instead of the default to avoid collision with existing bookshelf applications. I answered ‘Y‘ to the question of creating a new service and picked (2) mysql and gave it the name of ‘mysql-books‘.

We should now see the app running:

vmc apps

+-------------+----+---------+---------------------------------+---------------+
| Application | #  | Health  | URLS                            | Services      |
+-------------+----+---------+---------------------------------+---------------+
| bookshelf   | 1  | RUNNING | mybookshelf.cloudfoundry.com    | mysql-books   |
+-------------+----+---------+---------------------------------+---------------+

So we can now enter ‘http://mybookshelf.cloudfoundry.com/‘ and see the Bookshelf application come to life and add some books.

You can review and download the entire source used for this sample at cloudfoundry-samples/jruby-rails-bookshelf or if you just want to see the changes needed to deploy in Cloud Foundry look at this commit.

Conclusion

We have shown that it’s possible to deploy a simple JRuby on Rails application to cloudfoundry and use a MySQL service as the backing data store. All that is required is some modifications to the apps configuration of the database.

In a future post we’ll take a look at similar changes that we need to do for a JRuby Sinatra app that uses DataMapper for persistence.

- Thomas Risberg
The Cloud Foundry Team

Don’t have a Cloud Foundry account yet?  Sign up for free today

When we launched Cloud Foundry last year we started with an inefficient open source process based on a complex dual repo structure. The workflow was cumbersome for us to maintain, and at times frustrating for you to consume.

Today we are launching a new OSS contribution process based on a fully integrated Gerrit/Jenkins/GitHub workflow. In this workflow, Cloud Foundry contributors send their commits to a public Gerrit server. When a commit occurs, the Jenkins CI system will run various tests. If the tests succeed, the commit is marked as “Verified”.

The code review system allows developers to discuss and iterate on changes. Anyone can comment and vote +1 or -1 for a change, while committers can vote +2 (i.e. approve a change). Once the +2 has been issued and the change has been verified the contributor is free to submit the change.

The following diagram illustrates this workflow:

This new code review process is similar to that used by projects such as Android and OpenStack. Cloud Foundry development will occur in the open and the entire community will have visibility into the project’s progress.

Documentation on how to contribute to Cloud Foundry open source is available at http://cloudfoundry.org.

We are looking forward to working with the open source community using this new process. Thank you all for your interest in and support for Cloud Foundry.

Thanks, Mark Lucovsky, VP of Engineering