Day 5 - Setting up CI/CD

Published by Michael Banzon on Fri Feb 17, 2017

In this part we are going to set up continuous integration and deployment of the backend we wrote previously. We’ll make sure that changes made in the backend is applied and that they don’t cause problems.

What is that?!

Continious integration is a practice where developers checkin code often to a central repository which trigger automated processes for building the code etc. to validate the recent changes. Continious deployment is the practice where the automatically processed and validated changes in the code are deployed to production.

These terms have a tendency to be a bit vague in describing excactly what is going on - to put it in plain english: We are going to have changes pushed to the git repository build and deployed to the server automatically.

How?

To achieve this goal we are going to use an external service: Codeship

The code for the project is stored in GitHub and Codeship can be configured to trigger whenever something new is pushed to a GitHub repository.

The steps needed for this process include:

  1. Fetch the code + dependencies
  2. Build the binary
  3. Copy the binary to the server
  4. Restart the binary on the server

Part 1 - Fetching the code

This part is easy - Codeship actually does this. We just have to set up a project on Codeship and point it to our GitHub repository. First you have to connect using your GitHub account information - then point the Codeship project to your GitHub project - this will make Codeship setup a webhook that makes GitHub message Codeship when changes are pushed:

Point Codeship to the repository

Last it is time to set up the scripts to run when the webhook is triggered and the latest code have been fetched.

Setup the script to be run when a new version of the code has been fetched

The standard setup for Go won’t cut it - as the code we’ve written for the currency converter uses features that aren’t in the version of Go used by default in the Codeship system (as of this writing). Fortunately this little script in the setup commands part of the custom scripts will install the latest (as of this writing) version of Go - along with the dependencies needed for the project - I can’t remember where I found it (it was probably somewhere on Codeship), but it works:

GO_VERSION=${GO_VERSION:="1.8.0"}
# strip all components from PATH which point toa GO installation and configure the
# download location
CLEANED_PATH=$(echo $PATH | sed -r 's|/(usr/local\|tmp)/go(/([0-9]\.)+[0-9])?/bin:||g')
CACHED_DOWNLOAD="${HOME}/cache/go${GO_VERSION}.linux-amd64.tar.gz"
# configure the new GOROOT and PATH
export GOROOT="/tmp/go/${GO_VERSION}"
export PATH="${GOROOT}/bin:${CLEANED_PATH}"
# no set -e because this file is sourced and with the option set a failing command
# would cause an infrastructur error message on Codeship.
mkdir -p "${GOROOT}"
wget --continue --output-document "${CACHED_DOWNLOAD}" "https://storage.googleapis.com/golang/go${GO_VERSION}.linux-amd64.tar.gz"
tar -xaf "${CACHED_DOWNLOAD}" --strip-components=1 --directory "${GOROOT}"
# check the correct version is used
go version | grep ${GO_VERSION}
go get -t -v ./...

The command to execute the tests is simply:

go test -v ./...

Next we’ll add the script for deployment. First we’ll define that commits/pushes to a branch with the exact name “production”.

Setting up the branch to use for deployment

The option we’ll choose is the custom script and the following can be used:

go build
scp -P 2222 currencyconverter root@$SERVER:/root/tmp_currencyconverter
ssh -p 2222 root@$SERVER<<EOF
    killall currencyconverter
    rm /root/currencyconverter
    mv /root/tmp_currencyconverter /root/currencyconverter
    chmod +x /root/currencyconverter
    GFS_CURRENCY_HOST=0.0.0.0 GFS_CURRENCY_PORT=80 nohup /root/currencyconverter > /dev/null 2>&1 &
EOF

The script above builds the source and scp it to the server. After transferring it, the it connects with ssh to the server and stops the service, moves files around, fixes permissions and restarts the process (similar to how the service is started using crontab in the previous article).

For this to work the public SSH key (from the Project settings > General) needs to be added to the root user ~/.ssh/authorized_keys - this gives Codeship the ability to connect without using/exchanging passwords.

Please note the $SERVER variable used - this has to be set up in Codeship - this is a nice way to ensure you can just change the value if you need to change the server:

Setting up the server IP environment

In conclusion

There you have it! CI/CD it now working!

Every time we push changes to the branch called “production” Codeship will be notified and it will test the code, build the binary, copy it to the server, and restart the service. All working automatically!

What’s next

Next we’ll be setting up monitoring so we can make sure the server is running and being responsive etc.

Continue to day six…