Cloud Foundry Blog

About anynines

social bridgebuilder | photographer | techie | speaker | podcaster | MQTT Community Lead / Eclipse Paho co-lead | Cloud Foundry Developer Advocate @ Pivotal | my views are my own

OpenStack Swift as a Cloud Foundry service

This is a guest blog post by anynines, a Cloud Foundry hosted cloud provider from Europe.

In the past few years, the perception of cloud hosting has been revolutionized by the growth of IaaS and PaaS environments. However, many of the major hosting companies are US-based. In order to assure that data subject to European Union data privacy regulations stays in the EU, it might be important for some users to choose an IaaS that is completely EU-based (unlike AWS, for example).

We are committed to solving this challenge. Say “hello” to anynines – a solution for European cloud enthusiasts (or any others!) looking for safe, 100% European PaaS hosting. 

Matching IaaS to your PaaS

Firstly, we needed an IaaS that was EU-based and that worked with our PaaS.

We had two strong IaaS layers to choose from: OpenStack, and VMware vSphere. Both of these are popular and work with Cloud Foundry, BOSH and Fog. Anynines actually runs both vSphere and OpenStack. Our core components are powered by VMware, and our Swift storage service is a part of a separate OpenStack setup.

Nearly every standard web application requires some kind of file storage, and we felt that we needed to find an AWS S3 replacement for our PaaS users.

Why OpenStack Swift?

OpenStack Swift offers a REST API to store and retrieve files or any unstructured data at low cost. Highly scalable, a Swift setup can consist of anything from a few, up to thousands of machines, handling dozens of petabytes of storage… without a single point of failure!

Another important aspect is that Swift also offers scalable file access. Store massive data – access it massively. Simple!

Implementing a Cloud Foundry service

Starting to implement a Cloud Foundry OpenStack Swift service was a bit challenging, as Cloud Foundry itself is being improved every day. However, with the help of Pivotal and their open door policy, we received a lot of help during a two week stay in San Francisco. Thanks, folks!

Now that the story is over, it’s time to share our “recipe”: an implementation of OpenStack Swift as a service for Cloud Foundry applications.

Types of Cloud Foundry services

First, we needed to decide if the service would manage credentials only or also be responsible for provisioning service instances. Provisioning service instances means that the service implementation will be responsible for creating service processes, e.g. spinning up Redis instances. These kind of scenarios can become cumbersome, depending on whether you want isolation between service instances by using Warden containers, and high availability in case a service node breaks down.

The management of OpenStack Swift is out of the scope of Cloud Foundry in this case, so our service implementation fell into the first category: managing credentials only. Therefore, the service implementation required a running OpenStack instance somewhere in or near the datacenter. 

How the Swift Service is made

What about the basic service structure? Current Cloud Foundry services rely on VCAP Services Base – a repository with base code for all current service implementations.

These base classes provide a separation between a service gateway and service nodes.

While the service gateway is responsible for the communication with the Cloud Controller, the service nodes represent service plans, and are responsible for the actual service work. The latter could be, for instance provisioning Redis instances, or in our case talking to OpenStack to create new tenants and users.

Implement from scratch, or inherit?

As Cloud Foundry moves forward, communication between the Cloud Controller and the service gateway may change. Your service implementation should always be tested to be compliant with the current Cloud Controller’s interface.

Cloud Foundry developers and operators who want to integrate Swift as a native Cloud Foundry service could start with the VCAP::Services::Base class. Eventually, they may wish to revisit the minimum viable implementation to remove code related to multi-tenant service provisioning with Warden, since Swift manages tenants outside Cloud Foundry service nodes. We thought about building a new, more lightweight class, but for a minimal viable Swift service and an initial service implementation, VCAP::Services::Base delivered a scaffold to start on, and enabled us to focus on the implementation logic of our new service.

What we needed to implement

The major implementation work in developing a service breaks down into building four key methods: [un]provision(), and [un]bind().

  • During service provisioning, a new Cloud Foundry service instance is created. For MySQL this might be a database, and for Redis a new instance.
  • On bind, user credentials are generated, granting access to the resource. For MySQL, a user is allowed to access the corresponding database, for example.

Along with these methods, a credential generating method is required. You might also want to store some service metadata in a service database, such as tenant names for maintenance and/or caching purposes.   

OpenStack Swift service for Cloud Foundry

Now that we have looked at how to create a Cloud Foundry service in general, we can focus on how we built the OpenStack Swift service.

We created a basic implementation in order to make a running OpenStack Swift easy. Cloud Foundry users don’t have to struggle with OpenStack itself, but can just use the service right away. The Swift service assumes an OpenStack environment is up and running already.

The overall architecture and control flow for our Cloud Foundry Swift service looks like this:

Swift Service control flow

OpenStack tenants, users and role assignments

As the name implies, in OpenStack a tenant, also referred to as a project, is a construct isolating resources between different tenants, such as different Cloud Foundry organisations or users.

In our case, when creating a Cloud Foundry service, individual service instances needed to be isolated from one another. 

Therefore:

  • on provision we create a new tenant.
  • on binding an app to a service we create a user.

This allows us to bind multiple apps to a single Swift tenant, enabling us to share files between the apps. By creating multiple users, apps can have permissions to access a certain Swift account granted or revoked.

Fog and storage providers

A Ruby gem named Fog takes care of interactions between the service and OpenStack.

Tenant, user and role operations are done using the Identity class of the OpenStack provider, but operations that access the storage (Swift) have been implemented using the HP provider. This is because we found the HP storage provider to be more stable than the OpenStack provider as HP’s cloud actually uses a slightly modified OpenStack, underneath. However, it turned out that there’s a slight incompatibility between OpenStack Swift and HP’s flavor of it when it comes to generating temporary URLs.

Temporary URLs are required when you want a third party to access a file in a private directory. In this case, a temporary valid URL could be generated that allowed easy GET access to a file that would otherwise require a proper authorization header. Fortunately, the Fog and HP provider maintainers were more than helpful and gave good feedback while integrating a corresponding pull request. The patch allows OpenStack Swift and anynines Swift service users to pass in a so-called account meta temp url key, and thus use Fog to generate temporary URLs without this issue becoming a problem.

Using the Swift Service

For a detailed description on how to use the anynines Swift service within applications, have a look at our Swift howto.

The Cloud Foundry Swift Service is Open Source, and the code can be found on github. You can try it out with your applications today on anynines.com.

The Future 

Next up, we’ll set our sights on plan and quota management. A very simple approach could be to have two plans. A very restricted basic plan with a limited storage quota and an unlimited plan that will be billed pay-as-you go.

There are more questions to consider: Swift integration using BOSH. Standalone Swift without OpenStack Keystone and Nova – could we wrap it up using BOSH, and make it a part of a Cloud Foundry release? This would ease the overall management of the PaaS, as BOSH remains the primary management tool. However, Swift is by nature a very I/O intense system. Would the I/O overhead caused by virtualization be acceptable? Or should it remain on bare metal? We look forward to working with the community to address these discussions!

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Email

Installing Cloud Foundry on Vagrant

This is a guest blog post by Altoros, a software development firm specializing in PaaS enablement and integration services around Hadoop and NoSQL.

Cloud Foundry is by default a highly distributed multi-tenant platform that is usually deployed at scale (>40 nodes) on top of existing virtualized infrastructure (e.g., VMware vSphere or vCloud, Amazon AWS or OpenStack). For the novice Cloud Foundry developer, a full scale PaaS deployment isn’t always an affordable choice. To address this issue, the Cloud Foundry team released Micro Cloud Foundry as a virtual machine image that could be run on a laptop. Micro Cloud Foundry behaved much like a production cloud and provided developers a sandbox to play with. An alternative to using Micro Cloud Foundry on a virtual machine is to install Cloud Foundry natively on a development tool such as Vagrant.

What is Vagrant?

Vagrant is really a developer’s VM toolbox, as it provides an easy way to create and configure lightweight, reproducible, and portable development environments. It sits on top of a virtualization layer such as Virtualbox, VMware Fusion or VMware Workstation. The VM definition and configuration file (Vagrantfile) is part of the versioned source code repository.

Cloud Foundry on Vagrant

The Cloud Foundry deployment on Vagrant is a self-contained partial Cloud Foundry v2 installation which runs inside of a Vagrant-managed VM with Ubuntu 12.04.2 LTS. To get started, you need to download and install Vagrant. After that, just follow the step by step directions for deploying the Cloud Foundry Vagrant installer on Github.

You should be able to setup and configure everything in under 30 minutes, and easily push a simple Ruby Sinatra web application on the local Cloud Foundry PaaS.

Installation

Vagrant allows you to run Chef recipes after booting up the VM (these are located under the chef directory). Every time you bring up a VM (vagrant up), it will check the Chef folder and determine if any recipes should be run. The first time the Vagrant image is started, it will take some time to install all of the required packages.

The number of Chef recipes may be limited, so be sure to use Berkshelf to manage your cookbooks or their dependencies. Berkshelf maintains proven and stable recipes downloaded from OpsCode. Big thanks to Scott Frederick aka @scottyfred who provided the contribution of Berkshelf support!

Starting and Stopping Cloud Foundry Components

There are two main ways to start or stop the Cloud Foundry components. Most operators prefer to use start.sh and stop.sh which are bash scripts that launch upstart commands to run the components.

$ cd /vagrant
$ ./start.sh
cf-ng start/running

# Verify  running processes
$ ps ax
11733 ?        S      0:00 /bin/bash ./bin/file_server
11734 ?        S      0:00 /bin/bash ./bin/dea_ng
11739 ?        Sl     0:00 ruby spec/bin/file_server.rb
11745 ?        S      0:00 /bin/bash ./bin/dir_server
11750 ?        Sl     0:00 ruby /home/vagrant/.rbenv/versions/1.9.3-p392/bin/nats-server
11752 ?        Sl     0:01 ruby bin/dea config/dea.yml
11759 ?        S      0:00 /bin/bash ./bin/warden
11766 ?        Sl     0:00 ruby /home/vagrant/.rbenv/versions/1.9.3-p392/bin/rake dir_server:run
11793 ?        S      0:00 bash /home/vagrant/.rbenv/plugins/rbenv-sudo/bin/rbenv-sudo bundle exec rake warden:start[config/test_vm.yml]
11842 ?        S      0:00 /bin/bash ./bin/health_manager
11852 ?        S      0:00 /bin/bash ./bin/gorouter
11861 ?        S      0:00 /bin/bash ./bin/cloud_controller
11865 ?        Sl     0:00 ruby ./bin/health_manager
11866 ?        S      0:00 sudo env PATH=/home/vagrant/.rbenv/shims:/home/vagrant/.rbenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin bundle exec rake warden:start[config/test_vm.yml]
11877 ?        S      0:00 sudo ./bin/router -c=config/gorouter.yml
11878 ?        Sl     0:00 ruby /home/vagrant/.rbenv/versions/1.9.3/bin/rake warden:start[config/test_vm.yml]
11896 ?        Sl     0:00 ./bin/router -c=config/gorouter.yml
11946 ?        S      0:00 /bin/bash ./bin/uaa
11951 ?        Sl     0:29 /usr/lib/jvm/default-java/bin/java -classpath /usr/share/maven/boot/plexus-classworlds-2.x.jar -Dclassworlds.conf=/usr/share/maven/bin/m2.conf -Dmaven.home=/usr/share/maven org
12012 ?        Sl     0:00 go/bin/runner -conf config/dea.yml
12103 ?        Sl     0:02 ruby bin/cloud_controller -m

An alternate method is to use Foreman to declare the processes required to run your application in a single file: Procfile. Foreman is a useful Ruby gem that allows you to run several processes with only one command (foreman start). Foreman will start and the display the stdout and stderr of each process – this is useful because you can see all the logs in one terminal with different colors for each process. The scripts below were generated using Foreman (via foreman export).

Our Procfile

nats: nats-server
warden: ./bin/warden
uaa: ./bin/uaa
dir_server: ./bin/dir_server
file_server: ./bin/file_server
dea_ng: ./bin/dea_ng
gorouter: ./bin/gorouter
cc_ng: ./bin/cloud_controller
health_manager: ./bin/health_manager

Foreman output (colored)

foreman output

Custom Config Files

Vagrant uses the Cloud Foundry source code, downloaded directly from the corresponding Github repository for each module (CCNG, Router, NATS, DEA, etc). The .gitmodules file contains configuration details that can be customized or adapted for local installation onto Vagrant. The following are a few examples of different customizations for local Cloud Foundry deployments.

To add logging and increase the verbosity of the logging stream:

logging:
  file: /vagrant/logs/cloud_controller.log
  level: debug2

To make a DB persistent after VM shutdown, configure it in the corresponding custom_config_files/cloud_controller_ng/cloud_controller.yml file.

db:
  database: "sqlite:///vagrant/db/cloud_controller.db"

To set vcap.me as default domain, which is pointed to localhost:

Domain in cloud_controller.yml:

system_domain_organization: vcap.me
system_domain: vcap.me
external_domain:
  - api2.vcap.me
app_domains:
  - vcap.me

There are more examples and detailed changes recommended to the configuration files in the documentation on Altoros’s Github Vagrant Installer.

Building the Virtual Machine

We won’t duplicate the instructions to build the Cloud Foundry virtual machine in this blog post. Just follow the step-by-step instructions provided and you’ll have a Cloud Foundry v2 instance in a matter of minutes.

Learn More About Cloud Foundry

Cloud Foundry on Vagrant is a great tool for learning about the inner workings of Cloud Foundry and its various components. As always, don’t forget to refer to the great wealth of documentation available at the Cloud Foundry project page, and in particular the sections on using and running Cloud Foundry. Also, if you’re a developer you should signup and subscribe to the mailing list.

You can also find more information on the web about Cloud Foundry, including these popular sites:

Feel free to leave us feedback on your experience deploying Cloud Foundry on Vagrant and be sure to check back frequently for updates and improvements on the Github project.

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Email

Want to Contribute to Cloud Foundry? Come on in!

Cloud Foundry is an Open Platform-as-a-Service, and an Open Source project. It has attracted phenomenal interest from the community - including partners, companies using the code internally, and those individual developers with a passion for getting involved. You can find the source code on Github. Community contributions are what help to make the platform so extensible. We are always happy when we receive a Github pull request to offer new functionality or fixes! We also appreciate bug reports submitted through Github Issues.

Looking at the Cloud Foundry project as a whole though… where should you start?

As you might imagine, there are a lot of moving parts in a PaaS. At Pivotal, we have teams working on the backend (the main runtime environment for Cloud Foundry, also known as ‘vcap’); the frontend (the tools and APIs for deploying apps, such as the ‘cf’ gem); services (database, message queues, and related resources); and Cloud Foundry BOSH, the installer and deployment tool, which has different plugins targeting infrastructures like AWS, OpenStack and vSphere. In the future, we would love to see even more Cloud Provider Interfaces (CPIs) for BOSH to allow Cloud Foundry to be deployed onto additional clouds.

One of the best ways to get involved with the community and the codebase is to start with the Github repositories and the mailing lists. If you are not sure where particular functionality resides within the project as a whole, and you’ve had a good look at Github already, then we have a number of mailing lists where you can ask for help. vcap-dev is for those working on the PaaS itself; bosh-users is for those who want to understand how to use BOSH to deploy Cloud Foundry; and bosh-dev is for the elite hackers (like Dr Nic Williams!) who want to help us make BOSH even more awesome. If you participate in the mailing lists you will also be able to keep up with what others are working on, and learn who some of the leading developers on the project are. If you want to get an idea of the general direction of development, you’ll find that in our roadmap.

Once you are up-and-running with Cloud Foundry, or if you know to which part of the project  you want to contribute, then consider the size of an initial contribution. If you are new to the project you might want to start with something small (some Open Source projects call these kind of small changes to smooth out any rough edges “paper cuts”). If you have something larger, we are always keen to work with you to understand how we can integrate your code. Before sending a pull request, take a look at our OSS contribution information in the project readme, which includes details on the Contributor License Agreement for individuals and corporations.

We thought it would be good to highlight a recent pull request which was a superb example of a major addition to the project. Recently we received a contribution via a detailed Github pull request, adding Oracle support to some of the major components inside Cloud Foundry: the User Account and Authentication service (UAA) and the Cloud Controller. Originally, these components were built to use Postgres as a backend, and more recently MySQL support was added. Since many enterprises already use Oracle as their main relational data store, it is great to give them this additional storage option.

Taking a look at the specific request for the Cloud Controller, we were really struck by a few things. The first one is that the pull request itself is very detailed, and walks through the various changes required in order to replace the database with Oracle.

This pull request is a lot to digest. If you want you can review the single feature PR that lead to this PR for some more concise context into some of the decisions I made in creating this PR. Here is a summary of the more notable changes I had to make and some context for why I made those changes:

This level of detail is not always required with every patch, of course, but for something this extensive it was very welcome. Another aspect of this patch is that was was made of up several individual changes (things like tweaks to the database schema, differences between Oracle datatypes and those used by other databases, etc), and they were broken down into separate commits which made it easier to consume. It also helped that this set of changes was extensively discussed on the vcap-dev mailing list before this larger pull request was submitted, so the engineering team were aware of some of the other impacts it would have.

Although the resulting pull request was detailed and clear, it has taken a few days to make all of the changes, as the core code was still being worked on. This was actually one of the rarer cases where we had to do more work to get the patches integrated – and that shows how willing we are to work with contributions. Smaller changes are extremely welcome too, and often much quicker to digest. Don’t be put off by this larger example – come and start hacking with us!

We hope this post highlights some of the things that help to make it smoother to contribute to the project. To recap:

  1. review the contribution guidelines and complete the CLA;
  2. if you want to submit a major code change, discuss it on the relevant mailing list beforehand;
  3. think about how the pull request is structured, and submit changes in separate consumable commits (rather than as one monolithic change) where appropriate;
  4. the patch should be well-tested, should include tests if possible, and should pass our Travis CI process;
  5. the changes should be clearly documented in the request, especially when the updates are more extensive than correcting a few typos.

The past few months have seen the Cloud Foundry Open Source community grow hugely, and mailing list activity continues to be at an all-time high – so we’re really looking forward to continuing to work with other developers to make the platform even more awesome! Thanks for being on this journey with us.

Andy Piper @andypiper

Developer Advocate, Cloud Foundry

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Email

Cloud Foundry and Open PaaS at OSCON

The Cloud Foundry team is at the premier Open Source conference, O’Reilly OSCON this week. What a difference a year makes! This time last year, the Cloud Foundry Open Source project and the idea of an open Platform as a Service (and PaaS generally) was a new concept to many developers. This year, there has been a clear buzz around clouds – particularly open cloud platforms –and sessions on these topics have been very busy.

Cloud Foundry had a number of sessions: Josh Long has been as popular as ever, talking about Spring and Cloud Foundry topics; Raja Rao DV delivered a masterclass in node.js on Cloud Foundry; and I hosted a birds-of-a-feather group to discuss the idea of messaging and integration in the cloud using technologies like RabbitMQ. We also had some of the developers talking about and demonstrating BOSH, the deployment tool that was released to the community a couple of months ago. The exciting part here was that Vadim Spivak and Oleg Shaldybin showed BOSH deploying a full Cloud Foundry environment onto PistonCloud’s OpenStack-based cloud!

Deploying Cloud Foundry to PistonCloud

The SpringSource and Cloud Foundry expo hall booth has been busy, too. Everyone picked up a Micro Cloud Foundry USB stick in their conference schwag bags. Lots of people stopped by to show us their apps running on CloudFoundry.com and had the opportunity to earn a great hoodie.

The Cloud Foundry and SpringSource booth

For me as a Developer Advocate, it has also been fantastic to meet up with many of our partners in the Cloud Foundry ecosystem to see where they are taking their own offerings. I had the pleasure of spending time with Adron Hall talking about Tier 3Iron Foundry, and all kinds of other topics around the evolution of the Cloud Foundry community. If you are not familiar with Iron Foundry, it’s an exciting project that extends Cloud Foundry to the .NET environment.

Our partners over at AppFog brought some “No Vendor Lock-in” placards to the show, and handed out free ice cream outside the front of the Oregon Convention Center. Now, the draw of ice cream is always a good one, but for me it was a great opportunity to chat with the team and talk about what they are up to. AppFog are based locally in Portland, so it was nice to get to see them on “home turf.” I’m a big fan of their nice web console and the fact that they offer EC2, OpenStack and other options for deployment. AppFog have also been running a contest enabling developers to wish the OpenStack project a Happy Birthday, which looks like a lot of fun.

The ActiveState team are here too talking about Stackato, their private enterprise PaaS based on Cloud Foundry. A few months ago we posted that FeedHenry can deploy mobile apps to CloudFoundry.com, but some organisations might prefer to deploy those apps behind an enterprise firewall. Diane Mueller showed me a great example of Stackato and FeedHenry working together, delivering secure mobile webapps (see the sneak peek in the screenshot here).

Apart from all of these Cloud Foundry-based ecosystem partners, it has also been great to chat with our friends on the team at MongoDB, and get feedback from developers at the sessions and on the show floor about what they thought of Cloud Foundry and how I can help them move forward to a PaaS environment.

What a great event. I hope to be back here next year!

Watch our Facebook and Google+ pages for peeks at images from OSCON 2012.

Andy Piper, Cloud Foundry Team

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Email

Administer Cloud Foundry with Mobile Apps

One of the neat things about Cloud Foundry is that, because the code is open source, it’s easy to see how the administration tools (such as the command line-based vmc) work. The Cloud Controller component has a REST API, which provides the ability to query and modify the Cloud Foundry environment. That means it is relatively straightforward to build a management user interface tailored to the platform you are using, or to the requirements and needs of a specific set of users.

To illustrate this, I made a really brief video which I tend to use when I’m speaking about the Cloud Foundry platform and ecosystem. What you see here are two iOS apps - CF Mobile Admin (App Store link) and the AppFog app (App Store Link) – which have the ability to connect to Cloud Foundry instances, list deployed apps, modify instances, stop or start applications and query detailed information about the available resources. These kinds of tools are useful if you don’t have vmc handy, and are obviously great for mobile usage. Note that both of these tools are provided by third parties, and not by the Cloud Foundry development team.

Several developers have asked me if there are equivalent apps for Android or Windows Phone. If there are, I haven’t found them yet. One other example of a custom third-party-provided administration UI–in this case, a desktop-based one–is the Microsoft Management Console (MMC) snap-in, Uhuru Cloud Manager, that Uhuru Software provides as a tool alongside its own PaaS offering. There’s clearly an opportunity to build user interfaces and tools to target platforms such as Android and Windows Phone, or indeed your choice of mobile or desktop OS.

If you come up with anything of your own, do let us know by commenting here on the blog, or by talking to us on Twitter: @cloudfoundry or via the hashtag #cloudfoundry.

Andy Piper, Cloud Foundry Team

Sign up for Cloud Foundry today to try out these mobile administration tools

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Email