Local Development Setup
To use this development environment on OSX, you need to have Docker and Boot2docker installed.
Simplest way to prepare the working environment is to start the Cloudbreak on your local machine is to use the Cloudbreak Deployer.
First you need to create a sandbox directory which will store the necessary configuration files and dependencies of Cloudbreak Deployer. This directory must be created outside of the cloned Cloudbreak git repository:
To start the complete Cloudbreak ecosystem on your machine just execute the following sequence of commands:
cd cbd-local curl https://raw.githubusercontent.com/hortonworks/cloudbreak-deployer/master/install | sh && cbd --version cbd update master cbd init cbd start
If everything went well then the Cloudbreak will be available on http://192.168.59.103:3000. For more details and config parameters please check the documentation of Cloudbreak Deployer.
The deployer has generated a
certs directory under
cbd-local directory which will be needed later on to set up the IDEA properly.
The next step is to edit the
cbd-local/Profile file with any editor and add the CB_SCHEMA_SCRIPTS_LOCATION environment variable which configures the location of SQL scripts that are in the 'core/src/main/resources/schema' directory in the cloned Cloudbreak git repository. Please note that the full path needs to be configured and env variables like $USER cannot be used.
In order to kill Cloudbreak container running inside the boot2docker and redirect the Cloudbreak related traffic to the Cloudbreak running in IDEA use the followig command:
cbd util local-dev
Cloudbreak can be imported into IDEA as gradle project. Once it is done, you need to import the proper code formatter by using the File -> Import Settings... menu and selecting the
idea_settings.jar located in the
config directory in Cloudbreak git repository.
To launch the Cloudbreak application execute the
com.sequenceiq.cloudbreak.CloudbreakApplication class with VM options:
-XX:MaxPermSize=1024m -Dcb.cert.dir=FULL_PATH_OF_THE_CERTS_DIR_GENERATED_BY_CBD -Dcb.client.id=cloudbreak -Dcb.client.secret=cbsecret2015 -Dcb.db.port.5432.tcp.addr=192.168.59.103 -Dcb.db.port.5432.tcp.port=5432 -Dspring.cloud.consul.host=192.168.59.103 -Dcb.identity.server.url=http://192.168.59.103:8089 -Dserver.port=9091
-Dcb.cert.dir=FULL_PATH_OF_THE_CERTS_DIR_GENERATED_BY_CBD value above needs to be replaced with the full path of
certs directory generated by Cloudbreak Deployer e.g.
To run Cloudbreak from command line you have to create a property file, for example application.properties, with the content below, and execute
./gradlew bootRun -Dspring.config.location=file:/path/of/property/application.properties command at project root.
cb.cert.dir=FULL_PATH_OF_THE_CERTS_DIR_GENERATED_BY_CBD cb.client.id=cloudbreak cb.client.secret=cbsecret2015 cb.db.port.5432.tcp.addr=192.168.59.103 cb.db.port.5432.tcp.port=5432 spring.cloud.consul.host=192.168.59.103 cb.identity.server.url=http://192.168.59.103:8089 server.port=9091
If schema change is required in Cloudbreak database (cbdb), then the developer needs to write SQL scripts to migrate the database accordingly. In Cloudbreak the schema migration is managed by MYBATIS Migrations and the cbd tool provides an easy-to-use wrapper for it. The syntax for using the migration commands is
cbd migrate <database name> <command> [parameters] e.g.
cbd migrate migrate status.
Create a SQL template for schema changes:
cbd migrate cbdb new "CLOUD-123 schema change for new feature"
As as result of the above command an SQL file template is generated under the path specified in
CB_SCHEMA_SCRIPTS_LOCATION environment variable, which is defined in Profile. The structure of the generated SQL template looks like the following:
-- // CLOUD-123 schema change for new feature -- Migration SQL that makes the change goes here. -- //@UNDO -- SQL to undo the change goes here.
Once you have implemented your SQLs then you can execute them with:
cbd migrate cbdb up
If you would like to rollback the last SQL file, then just use the down command:
cbd migrate cbdb down
On order to check the status of database
cbd migrate cbdb status #Every script that has not been executed will be marked as ...pending... in the output of status command: ------------------------------------------------------------------------ -- MyBatis Migrations - status ------------------------------------------------------------------------ ID Applied At Description ================================================================================ 20150421140021 2015-07-08 10:04:28 create changelog 20150421150000 2015-07-08 10:04:28 CLOUD-607 create baseline schema 20150507121756 2015-07-08 10:04:28 CLOUD-576 change instancegrouptype hostgroup to core 20151008090632 ...pending... CLOUD-123 schema change for new feature ------------------------------------------------------------------------
Gradle is used for build and dependency management. Gradle wrapper is added to Cloudbreak git repository, therefore building can be done with:
./gradlew clean build
Development process should happen on separate branches. Then a pull-request should be opened as usual.
To build the project
# make deps needed only once make deps make install
To run the unit tests:
If you want to test the binary CircleCI build from your branch named
fix-something, to validate the PR binary
cbd tool will be tested. It is built by CircleCI for each branch.
cbd update fix-something
We recommend to always use the latest release, but you might want to check new features or bugfixes.
All successful builds from the
master branch are uploaded to the public S3 bucket. You can download it:
curl -L s3.amazonaws.com/public-repo-1.hortonworks.com/HDP/cloudbreak/cloudbreak-deployer_snapshot_$(uname)_x86_64.tgz | tar -xz
Instead of overwriting the released version, download it to a local directory and useit by refering as
Please cover your bash functions with unit tests and run test with:
Release Process of the Clodbreak Deployer tool
The master branch is always built on CircleCI. When you wan’t a new release, all you have to do:
make release-next-ver performs the following steps:
- On the
- Updates the
VERSIONfile by increasing the patch version number (for example from 0.5.2 to 0.5.3)
CHANGELOG.mdwith the release date
- Creates a new Unreleased section in top of
- Updates the
- Creates a PullRequest for the release branch:
- create a new branch with a name like
- this branch should be the same as
- create a pull request into
- create a new branch with a name like
Now you should test this release. You can get it by
cbd update release-x.y.z. Comment with LGTM (Looking Good To Me).
Once the PR is merged, CircleCI will:
- create a new release on GitHub releases tab, with the help of gh-release.
- it will create the git tag with