Creating a Cloudera Certification Cluster with Cloudsoft's Brooklyn

The Cloudera Certified Technology program is the industry’s first and only self-certification program which allows technology partners to validate that their products can be supported on Apache Hadoop.

Note that you will need to be a registered user or partner to access this program but it is a very light touch registration process (so long as you use a corporate email) and there are two flavors – Hardware Certification and Software Certification.

In order to streamline the Software Certification process, Cloudera has developed a Certification Cluster Builder with Cloudsoft’s help that lets you automatically deploy a CDH cluster onto your preferred cloud.

This tool ensures that the deployed cluster meets the requirements for certification, including having the appropriate Hadoop logging enabled. Under the hood it uses Apache Whirr and Brooklyn to make certification cluster deployment in the cloud ‘push-button simple’.

Brooklyn was open sourced in April 2012 under the Apache 2.0 license by Cloudsoft which provides commercial support and enhancements under its Application Management Platform (AMP) line of products.

Assuming that you are registered with Cloudera the first thing you should do is read Cloudera’s Software Certification Guide. This links to the Certification Cluster Builder download page.

Once you’ve downloaded the zip file and unpacked it you can launch the Certification Cluster Builder from the command line –

Having launched the Certification Cluster Builder you select your preferred cloud – in this example we’ve chosen AWS US East – and plug in your credentials. If you want to follow the progress of your deployment then you can open a console window –

If you are interested you can also login to your AWS Management Console and you will see the AWS instances being created.

Behind the scenes Brooklyn takes care of the deployment of the certification cluster ensuring that all the components are created and wired up according to the blueprint that Cloudera have created. You can launch the Brooklyn console from the CCB tool too.

Once the cluster is up you can dismiss the CCB tool and use the Brooklyn console to connect through to the Cloudera Manager and install and run your software. (For more on how this works and any questions please join the Brooklyn community.)

There is a nice touch in that you can use Brooklyn to collect the metrics (what Cloudera calls the logs) when you are done.

Have fun!