Inception creationThe example includes a script (called create_inception_pipeline.py) which creates the pipeline that will create the pipelines (you run this only once):
from gomatic import GoCdConfigurator, HostRestClient, ExecTask, GitMaterial configurator = GoCdConfigurator(HostRestClient("localhost:8153")) pipeline = configurator\ .ensure_pipeline_group("inception")\ .ensure_replacement_of_pipeline("inception")\ .set_git_material(GitMaterial("https://github.com/ivanmoore/inception.git", polling=False))\ .set_timer("0 0 * * * ?") # run on the hour, every hour inception_job = pipeline.ensure_stage("inception").ensure_job("inception") inception_job.ensure_task(ExecTask(["python", "inception.py"])) configurator.save_updated_config()
This creates a pipeline in GoCD (here running on localhost) which runs inception.py on a timer.
The video starts with this script being run, and the "inception" pipeline has been created by time 0:19.
InceptionThe inception.py script creates a pipeline for every repo of a particular github user. For a real system, you might want to do something more sophisticated; this example has been kept deliberately simple.
from gomatic import GoCdConfigurator, HostRestClient, ExecTask from github import Github, GithubException github = Github() me = github.get_user("teamoptimization") for repo in me.get_repos(): try: print "configuring", repo.name configurator = GoCdConfigurator(HostRestClient("localhost:8153")) pipeline = configurator\ .ensure_pipeline_group("auto-created")\ .ensure_pipeline(repo.name)\ .set_git_url(repo.clone_url) job = pipeline\ .ensure_initial_stage("bootstrap")\ .ensure_job("configure-pipeline") bootstrap_file_url = "https://raw.githubusercontent.com/ivanmoore/inception/master/bootstrap.py" job.ensure_task(ExecTask(["bash", "-c", "curl -fSs " + bootstrap_file_url + " | python - " + repo.name])) configurator.save_updated_config() except GithubException: print 'ignoring', repo.name
This script creates a pipeline with a "bootstrap" stage for each repo (unless it already exists). As long as nothing else creates pipelines with the same names, the "bootstrap" stage will end up being the first stage. The bootstrap stage runs the bootstrap.py script described later, passing it the name of the repo/pipeline as an argument.
In the video, the "inception" pipeline is triggered manually at time 0:20 (rather than waiting for the timer) and has finished by time 1:03 (and has no affect yet as the relevant user has no repositories).
The bootstrap stageIn this example, the bootstrap.py script creates a stage for every line of a file (called commands.txt); this makes it easy to demonstrate one of the key features of the approach - the ability to keep the pipeline in sync with the repo it is for. One of the subtleties is that the bootstrap has to make sure that it doesn't remove itself, but does need to remove all other stages so that if stages are removed from commands.txt then they will be removed from the pipeline. Note that because of how gomatic works, if there is no difference as a result of removing then re-adding the stages, then no POST request will be sent to GoCD, i.e. it would be entirely unaffected.
import sys from gomatic import GoCdConfigurator, HostRestClient, ExecTask configurator = GoCdConfigurator(HostRestClient("localhost:8153")) pipeline_name = sys.argv pipeline = configurator\ .ensure_pipeline_group("auto-created")\ .find_pipeline(pipeline_name) for stage in pipeline.stages()[1:]: pipeline.ensure_removal_of_stage(stage.name()) commands = open("commands.txt").readlines() for command in commands: command_name, thing_to_execute = command.strip().split('=') pipeline\ .ensure_stage(command_name)\ .ensure_job(command_name)\ .ensure_task(ExecTask(thing_to_execute.split(" "))) configurator.save_updated_config()
A real bootstrap script might be much more sophisticated, for example, creating a build stage automatically for any repo which contains a certain file (e.g. build.xml or maven.pom) and creating deployment stage(s) automatically. The example bootstrap.py script is as short as I could make it for the purposes of demonstrating the approach.
In the video, the user creates a repository (from time 1:04 - 1:39) and then creates a commands.txt file, commits and pushes (up to time 2:18). Rather than waiting for the timer, the "inception" pipeline is manually triggered at time 2:22 and by 2:43 the pipeline is created for "project1". Rather than wait for GoCD to run the new pipeline (which it would after a minute or so) it is manually triggered at time 2:54, and when it runs, it creates the stage defined in commands.txt.
In the video, at time 3:54 the user adds another line to commands.txt and commits and pushes. Rather than wait for GoCD to run the pipeline (which it would after a minute or so) it is manually triggered at time 4:27, and when it runs, it adds the new stage defined in commands.txt.
Copyright © 2015 Ivan Moore