scipion logo

Buildbot

BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure.

From early stages in the development of Scipion, we have included a number of automatic tests into this tool that executes periodically those tests. Those executions can be examined in http://scipion-test.cnb.csic.es:9980/#/

Our buildbot architecture consists of a single buildmaster and one worker connected in a star topology. The buildmaster makes all decisions about what, when, and how to build. It sends commands to be run on the worker, which simply execute the commands and return the results. (certain steps involve more local decision making, where the overhead of sending a lot of commands back and forth would be inappropriate, but in general the buildmaster is responsible for everything).

System Architecture

The worker runs on a separate machine where the master runs. This machine connect to the buildmaster over a TCP connection. The TCP connections are initiated by the worker and accepted by the buildmaster, but commands and results travel both ways within this connection. The buildmaster is always in charge, so all commands travel exclusively from the buildmaster to the worker.

The buildmaster consists of several pieces:

Build Master

All of our repositories are on GIT. These repositories are constantly being modified. All Changes are fed to the Schedulers which decide when builds should be performed. They collect Changes into BuildRequests, which are then queued for delivery to Builders until a worker is available. The Builders control exactly how each build is performed (with a series of BuildSteps, configured in a BuildFactory).

Currently, our code is divided into three main branches which are tested by buildbot:

  • support: represents the stable code of the release 2.0.0 of Scipion (Diocletian)

  • production: represents the stable code of release 3.0.0 (???????)

  • devel: represents the code in development of the release 3.0.0.

For that reason, we have three BuildFactories that are in charge of automatically executing the code of the aforementioned branches:

  • supportBuildGroupFactory: this BuildFactory is automatically released on Sundays.

  • productionBuildGroupFactory: this BuildFactory is automatically released on Saturday.

  • develBuildGroupFactory: this BuildFactory is automatically released on Monday, Wednesday and Friday.

Complete Build Master

If a Notifier status target is active, the completion of a build will cause a notification to be sent to any developers or the Scipion team whose Changes were incorporated into this Build. The Notifier can be configured to only send a notification or mail upon failing builds, or for builds which have just transitioned from passing to failing. Other status targets can provide similar real-time notification via different communication channels, like Slack.

Adding your plugin in Buildbot

In order to include a Scipion plugin within buildbot and have it tested, the following steps must be followed:

  1. Make sure the plugin is on one Version Control System like GIT (to test in devel mode).

  2. Make sure the plugin is on Pypi (to test in production mode).

  3. Contact white Scipion Team in order to include it into Buildbot. For that steps, send us an email with the link to repository.