In the world of MicroServices, immutable deployments is a highly recommended strategy. It demands that every release get a fresh environment, all the way down to the lowest level - or in case of AWS, the AMI.

An AMI can bundle the base operating system, application server/runtime, scripts, agents, etc. along with a versioned application artifact. In the spirit of Infrastructure as Code, the AMI itself can be versioned as a simple json file, as we will see in this post.

With respect to Automation, creation of the AMI can be triggered as a downstream Jenkins Job, after successful pre-release tests, and can then act as an upstream Job for the Infrastructure Build-out using CloudFormation, or Terraform. Once the infrastructure is built, it can be made live via blue/green, red/black or any other deployment plan.

In this post, we will build an AMI that can be used to launch 100% identical instances in the infrastructure. We will use Packer to build the AMI, and create Jenkins Jobs to simulate a simplified, real-world CI/CD pipeline.

Packer Gitflow

AMI

An AMI, or Amazon Machine Image is a template that describes an instance - Operating System (Linux, Windows..), Architecture (i386, x86_64..), Virtualization (pvm, hvm), Block Device Mapping, Root Volume Type (EBS, Instance), and any other software packaged along. An AMI can be used to launch instances, usually in an Auto Scaling Group. An AMI is tied to a region, but can be copied over to other region(s). AMIs can be private (to an account), public, or shared across accounts. This privacy model controls which accounts can launch instances using this AMI. AMIs can be created from an instance via AWS CLI using the create-image command.

AMIs and Immutable Deployments

An AMI can be thought of as the lowest level of control for a deployment. Going up the chain, there are application servers, with application artifacts (RPM, WAR, node package) at the highest level. The goal is to achieve immutability by bundling the artifact in the AMI (baking in) so the instances created from this AMI would need no other configuration, and completely identical instances can be spun up in an automated manner. The logs on this instance can be monitored by pushing them out to a central logging service, such as Splunk or Loggly, or a home grown ELK setup. There should absolutely be no need to ssh into any of these instances.

Packer

Packer is a tool to automate building of AMIs (and other targeted Machine Images) from configuration files, or templates. The templates are JSON files containing basic information about the AMI to be created, and can contain inline scripts to provision software on the resulting AMI. Any instances launched from this AMI will have the exact same configuration - from the Operating System to the deployed code. It is ideal to not have any ssh access on these instances either to achieve a much higher, or absolute level of immutability. That may take a few iterations and maturity cycles though.

Setup

Refer to the Hello World SparkJava App, which is created using the SparkJava WAR Maven archetype. This WAR (versioned artifact) is available in github and will be baked into the AMI. We will use Jenkins to create an AMI for a release version of our war file.

In the real world, the WAR will be a real service, and the artifact would have been uploaded to Nexus or Artifactory after successfully passing pre-release test cycle. In this case, the war is available as a release in this github repo.

Download Packer from packer.io. It would be ideal to have it run on your Jenkins node, as we will be using Jenkins to create an AMI.

bash-3.2$ packer                               
Usage: packer [--version] [--help] <command> [<args>]

Available commands are:
   build       build image(s) from template
   fix         fixes templates from old versions of packer
   inspect     see components of a template
   push        push a template and supporting files to a Packer build service
   validate    check that a template is valid
   version     Prints the Packer version

Before creating the Jenkins job, it would help to take a look at a typical, basic Packer template. This file is located here.

{
  "_comment" : "Simple Packer Template using Amazon Linux 2017.03",
  "variables": {
    "aws_access_key": "",
    "aws_secret_key": ""
  },
  "builders": [{
    "type": "amazon-ebs",
    "access_key": "{{user `aws_access_key`}}",
    "secret_key": "{{user `aws_secret_key`}}",
    "region": "us-east-1",
    "source_ami": "ami-c58c1dd3",
    "instance_type": "t2.micro",
    "ssh_username": "ec2-user",
    "ami_name": "HelloWorld Build-{{user `Build`}}"
  }]
}

The source_ami I’ve used here is latest release 2017.03 (at the time of writing) of Amazon Linux in us-east-1. You can view the AMI IDs here. The source_ami is the base AMI - you may want to have a different base AMI for your company or project. In order to not think of this as a chicken and egg problem, the source_ami can be created from an EC2 instance, which has the desired software and block mappings etc. on it.

In this case we will just use the bare bones Amazon Linux AMI to build the image. The type defines EBS-backed-store AMI in this case. Packer will build the AMI by creating an instance off of the source_ami, run any installation scripts (to set up servers like Tomcat and deploying the WAR file), and register the image created after this process as a new AMI. The EC2 instance used to create the AMI will then be terminated by Packer.

All of this is defined under the builders section of the template, which can have multiple builders. We then use the template’s variable system to inject the artifact build version and timestamp in the created AMI name. In the real world, this Build variable will be a Jenkins parameter passed over by an upstream job.

Regarding the aws_access_key and aws_secret_key - these can be passed to the Jenkins job as parameters, or ideally be pre-configured on the Jenkins node under ~/.aws/credentials. More details on Packer Templates here.

Provisioning the AMI

With the base AMI identified, we will need to install Tomcat, and deploy the WAR artifact in the AMI being created. This is achieved via a simple script, contained in a provisioner section in packer.json.

The script would do the following -

  1. Download and install OpenJDK8 and Tomcat8
  2. Download the artifact (again, I am just using a simple git release here)
  3. Deploy it under Tomcat, and make Tomcat start on bootup
{
  "_comment" : "Simple Packer Template using Amazon Linux 2017.03",
  "variables": {
    "aws_access_key": "",
    "aws_secret_key": ""
  },
  "builders": [{
    "type": "amazon-ebs",
    "access_key": "{{user `aws_access_key`}}",
    "secret_key": "{{user `aws_secret_key`}}",
    "region": "us-east-1",
    "source_ami": "ami-c58c1dd3",
    "instance_type": "t2.micro",
    "ssh_username": "ec2-user",
    "ami_name": "HelloWorld Build-{{user `Build`}}"
  }],
  "provisioners": [{
    "type": "shell",
    "inline": [
      "sleep 30",
      "sudo yum update -y",
      "sudo yum install java-1.8.0 java-1.8.0-openjdk-devel tomcat8-webapps -y",
      "sudo yum remove java-1.7.0-openjdk  -y",
      "sudo wget https://github.com/lobster1234/helloworld-api/files/953511/helloworld-api.war.gz -O /usr/share/tomcat8/webapps/helloworld-api.war.gz",
      "sudo gunzip /usr/share/tomcat8/webapps/helloworld-api.war.gz",
      "sudo chkconfig tomcat8 on"
    ]
  }]
}

Now we are ready to run packer via Jenkins, but let us validate the JSON first.

bash-3.2$ packer validate packer.json
Template validated successfully.

Creating the Jenkins Job to build the AMI

This is the job that gets triggered after pre-release tests have passed, and an artifact is ready to be pulled. This is a parameterized build, and the parameter(s) will help it pull the correct artifact for the version being built.

We call the Jenkins Job hello-world-api-packer, created as a FreeStyle Project, pointed it to the git repo, and add the following under Build/Execute Shell.

/Downloads/packer build -var Build=$Build src/main/resources/packer.json

There is 1 string parameter, called Build. Please see the config.xml of this job in the References section below.

Once run, this is the job output -

/Downloads/packer build -var Build=$Build src/main/resources/packer.json
[hello-world-api-packer] $ /bin/sh -xe /Users/mpandit/Downloads/tomcat/temp/hudson108285086652597491.sh
+ /Users/mpandit/Downloads/packer build -var Build=1.0 src/main/resources/packer.json
amazon-ebs output will be in this color.

amazon-ebs: Prevalidating AMI Name...
amazon-ebs: Found Image ID: ami-c58c1dd3
amazon-ebs: Creating temporary keypair: packer_58fed083-13d4-5740-0417-277163f775de
amazon-ebs: Creating temporary security group for this instance...
amazon-ebs: Authorizing access to port 22 the temporary security group...
amazon-ebs: Launching a source AWS instance...
amazon-ebs: Instance ID: i-0a0980f97a10aa4a0
amazon-ebs: Waiting for instance (i-0a0980f97a10aa4a0) to become ready...
amazon-ebs: Adding tags to source instance
amazon-ebs: Adding tag: "Name": "Packer Builder"
amazon-ebs: Waiting for SSH to become available...
amazon-ebs: Connected to SSH!
amazon-ebs: Provisioning with shell script: /var/folders/vf/d0q4kjg964581kjjz4969dbny407x7/T/packer-shell299271769
amazon-ebs: Stopping the source instance...
amazon-ebs: Waiting for the instance to stop...
amazon-ebs: Creating the AMI: HelloWorld Build-1.0
amazon-ebs: AMI: ami-08fc671e
amazon-ebs: Waiting for AMI to become ready...
amazon-ebs: Terminating the source AWS instance...
amazon-ebs: Cleaning up any extra volumes...
amazon-ebs: No volumes to clean up, skipping
amazon-ebs: Deleting temporary security group...
amazon-ebs: Deleting temporary keypair...
'amazon-ebs' finished.

==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:

us-east-1: ami-08fc671e

Finished: SUCCESS

As we can see, the new AMI has been created with an ID ami-08fc671e in us-east-1 region. Since this AMI is private, It is visible under My AMIs on the AWS Console.

AMI

Upon selecting this AMI and launching an instance in a public subnet with a security group allowing TCP traffic to port 8080 from 0.0.0.0/0, we can see the Tomcat landing page.

The resources below contain a fully functional pipeline - you’d need to configure AWS CLI on the Jenkins node, and place packer in /Downloads folder on the node. You may want to change these paths if your folder structure is different.

Hope you found this tutorial useful, and can embark on the immutable deployment journey with Packer and AWS!

Resources

  1. Hello World API Github Project, which also has the packer.json under src/main/resources.

  2. hello-world-api Jenkins Job to build the Hello World API Project and produce a WAR.

  3. hello-world-api-packer Jenkins Job to create the AMI.