How to build your own AMI from Ubuntu Pro using Packer

This how-to will walk you through the steps needed to create a Golden Image (or Amazon Machine Image) on AWS from an Ubuntu Pro AMI.

We are going to see how to get the latest Ubuntu Pro image from AWS and what considerations are needed in the provisioning stage to create a working Ubuntu Pro image, while revisiting the regular elements needed, all inside the Packer JSON template File.

We will be using Ubuntu Pro 20.04 for this tutorial, but this method is applicable to any other version.

What you’ll learn

• How to create golden images (AMIs) using Ubuntu Pro
• How to add your own customisations in the Packer template file without breaking the Ubuntu Pro activation mechanism.

What you’ll need

• An AWS account
• Basic understanding of AWS: EC2, IAM
• Packer installed on your workstation
• Some experience using Packer

The first step is to get all the tools ready. We will need to:

• Install packer
• (Optional) Subscribe to the AWS Marketplace listing (Only needed for the FIPS AMI)
• Create AWS API credentials.

Skip to the second step if you don’t need assistance with the points above.

Installing Packer in your workstation:

Go to https://www.packer.io/downloads and follow the instructions based on the platform you will be using.

If you are already working on Ubuntu, installing Packer is even easier. Just sudo apt install packer will do the job.

Subscribing to Ubuntu Pro in AWS (Only needed for FIPS AMI)

This is only required if you want to use the Ubuntu Pro FIPS image as base. If you are using just Ubuntu Pro, feel free to skip to the next point.

Go to the AWS Marketplace and search for Ubuntu Pro FIPS. Select Ubuntu Pro FIPS 20.04 LTS from the list and click “Continue to subscribe” as shown in the picture below.

If you are not logged in, AWS Marketplace page will ask you to do so.

In the next page you will see the summary page, click on Accept terms and wait for the activation of the subscription. Remember: you don’t get charged if you are not running any instance of Ubuntu Pro.

Creating AWS Credentials

Go to the IAM page for creating credentials and scroll down to the “Programmatic access” section. You can learn more about this on the same page.

Your credentials should look something like this:

Secret: AKIAIOSFODNN7EXAMPLE
Access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Remember to copy and paste them in a secure place. You will be needing them when writing the template.

Let’s create a blank file to be saved as packer.json. The basic JSON template consists of several objects which for our use case will be the variables, builders and provisioners objects. They all reside at the same root level of the structure.

Here we will start adding our variables only, which for simplicity will be our AWS credentials.

{
    "variables": {
        "aws_access_key": "YOUR_ACCESS_KEY",
        "aws_secret_key": "YOUR_SECRET_KEY"
    },
}

In this step, we will provide the configuration for the builder, such as the image type we are going to use for building ours, the region and the base image we will be using with the owner ID to ensure we are getting the right one.

We can add the following section (which is not complete) next to the variables object, right after the comma. Feel free to change region, instance_type or the ami_name.

"builders": [
      {
        "type": "amazon-ebs",
        "access_key": "{{user `aws_access_key`}}",
        "secret_key": "{{user `aws_secret_key`}}",
        "region": "us-east-1",
        "instance_type": "t2.micro",
        "ami_name": "My-Ubuntu-Pro-20.04-{{timestamp}}",

Next, we will add the source image we will be using. We are going to use a very useful functionality in Packer which searches for image types with a given string and fetches the latest AMI found. We will add the sources_ami_filter”object as follows:

        "source_ami_filter": {
            "filters": {
                "virtualization-type": "hvm",
                "name": "ubuntu-pro-server*20.04-amd64*",
                "root-device-type": "ebs"
              },
          "owners": ["099720109477"],
          "most_recent": true
        },

This will search the EC2 API for an image with ubuntu-pro-server*20.04-amd64 as name with Canonical’s owner ID (Canonical is the official publisher these images of Ubuntu Pro on AWS), using wildcards (*) for replacing any text that could change in the future. If you are using the FIPS listings from AWS Marketplace, then replace the owner ID with 679593333241.

And then, the method for accessing the instance for the provisioning step.

"ssh_username": "ubuntu"

We can close the builder object by adding a closing curly bracket } and the builders array with the square closing bracket ].

To this point we have a pretty standard -but working- Packer template file with no modifications done to the base image.

During this stage is when the magic happens since we will specify what we want to install, configure and how. Typical use cases here are hardening the image, configuring Active Directory login, adding specific configurations for management and compliance, installing software, copying plain files, etc.

This process can be done via inline shell commands, bash scripts or even using configuration tools such as Ansible, Chef, Puppet and so on.

No matter what tool you prefer, we suggest you add two important blocks that will help to make sure the Ubuntu Advantage process goes smoothly.

The first command should be added at the beginning of the process, which is cloud-init status --wait. This will tell the script to wait until all the initialization processes are finished, including the Ubuntu Advantage activation process. If you skip this line, you may have errors during the build process, since the Ubuntu Advantage client needs to change configurations and repositories right after booting.

The second block of commands are relevant for removing information that is particular to the instance used to build the image, such as the machine ID and the Ubuntu Advantage generated token. We need generic AMIs with no duplication of unique information.

This should be added at the end of the process.

sudo ua detach --assume-yes
sudo rm -rf /var/log/ubuntu-advantage.log
sudo cloud-init clean --machine-id

This will ensure that every time you spin up a new instance from this AMI, you will have a “fresh start”.

In an “inline shell”, it will look like this:

    "provisioners": [
      {
        "type": "shell",
        "inline": [
          "cloud-init status --wait",
          "sudo apt-get update && sudo apt-get upgrade -y"
        ]
      },
      {
        "type": "shell",
        "inline": [
          "sudo ua detach --assume-yes",
          "sudo rm -rf /var/log/ubuntu-advantage.log",
          "sudo cloud-init clean --machine-id"
        ]
      }
    ]

You can also include them directly in your script or provisioning tool. In the example, all your scripts or provisioning tool goes just in the middle of those two blocks.

This is how my packer.json file looks:

{
    "variables": {
        "aws_access_key": "YOURACCESSKEY",
        "aws_secret_key": "YOURSECRETKEY"
    },
    "builders": [
      {
        "type": "amazon-ebs",
        "access_key": "{{user `aws_access_key`}}",
        "secret_key": "{{user `aws_secret_key`}}",
        "region": "us-east-1",
        "instance_type": "t2.micro",
        "ami_name": "packer-base-ubuntu-{{timestamp}}",
        "source_ami_filter": {
            "filters": {
                "virtualization-type": "hvm",
                "name": "ubuntu-pro-server*20.04-amd64*",
                "root-device-type": "ebs"
              },
          "owners": ["099720109477"],
          "most_recent": true
        },
      "ssh_username": "ubuntu"
      }
    ],
    "provisioners": [
      {
        "type": "shell",
        "inline": [
          "cloud-init status --wait",
          "sudo apt-get update && sudo apt-get upgrade -y"
        ]
      },
      {
        "type": "shell",
        "scripts": ["my_script.sh"]
      },
      {
        "type": "shell",
        "inline": [
          "sudo ua detach --assume-yes",
          "sudo rm -rf /var/log/ubuntu-advantage.log",
          "sudo cloud-init clean --machine-id"
        ]
      }
    ]
}

The final step is to build the AMI by running packer with our json file:

packer build packer.json

Wait for a few minutes, enjoy watching the outputs from your script, and once the process finishes you should get the AMI ID of you own created Golden Image.

This is the last step of the process. We will verify if the AMI created works (yes, it is quite common to end up with an image that produces non bootable instances!).

For the sake of simplicity, we are going to launch the new instance from the AWS EC2 web console. We just need to remember to launch the instance type with the same architecture that our new AMI has (AMD64 for this how-to).

Log in into the AWS EC2 console and select Launch instances. You should see the quickstart menu with the default Amazon AMIs. On the left hand menu, select “my AMIs”.

You should get the following screen:

Select the new AMI and follow the wizard for configuring the instance’s options (instance type, disk, security groups, roles, key-pair and so on).

Now that we have launched it, let’s access the new instance to check if everything is in place

Once inside the instance, let’s check the Ubuntu Advantage status. This will show you if you are already getting all the benefits from Ubuntu Pro and will be a good sign that everything went well.

Type the following in the console:

Sudo ua status --wait 

You should get the following screen:

We only need to check whether we have esm-apps, esm-infra and livepatch enabled, active and check the entitlements we have by using Pro, such as CIS, FIPS and CC-EAL.

Now you have a simple way to create Ubuntu Pro derivative images (a.k.a. Golden Images, Machine images, AMIs), that are less prone to errors.

Remember to terminate the instance if you are not going to use it and delete the AMI to avoid any storage cost.

Further reading

• Finding EC2 images
• Ubuntu Pro on AWS
• Ubuntu Pro FIPS 20.04 LTS Marketplace image (FIPS pre-enabled)
• Getting started with Ubuntu Pro