This post is an overview on running ElasticMQ in Amazon ECS. This can help simulate SQS for development purposes, and running it in ECS would help with resourcing, as well as having an ALB to act as an endpoint-url. I’ve used EC2 and not Fargate, but this can just as easily be launched as a Fargate task. While there is also localstack, for this particular case, I just wanted to run SQS Mock and not all other services localstack comes bundled with.

Familiarity with ECS, specially around creating Task Definitions, Services, and associating them with Application Load Balancers will definitely help.

The Github Repository is here

Setup

First off, we create a custom configuration for ElasticMQ, mostly to ensure that the endpoint URL is different than localhost, and maps to whatever ALB this service runs as.

Here is the custom configuration:

include classpath("application.conf")

node-address {
    protocol = http
    host = "*"
    port = 9324
    context-path = ""
}

rest-sqs {
    enabled = true
    bind-port = 9324
    bind-hostname = "0.0.0.0"
    sqs-limits = strict
}

generate-node-address = false

queues {

}

Next we create the Dockerfile:

FROM openjdk:8u151-alpine
WORKDIR /tmp
RUN wget  https://s3-eu-west-1.amazonaws.com/softwaremill-public/elasticmq-server-0.13.9.jar
COPY custom.conf /tmp/
EXPOSE 9324
CMD  java -Dconfig.file=custom.conf -jar elasticmq-server-0.13.9.jar

Build and Deploy

To build the docker image :

$ git clone git@github.com:lobster1234/elasticmq-docker.git
$ cd elasticmq-docker
$ docker build .
$ docker run -p9324:9324 <image_id>

I’ve pushed this image on dockerhub as lobster1234/elasticmq-docker. This can be used in ECS while creating the container definition.

As a reference, here is the task definition JSON for ECS. As you can see, I’ve used 256MB and 128vCPU for this task. I’ve also used us-east-1, and called the task as elasticmq. Everything else is default ECS including log driver and IAM roles, etc.

{
  "executionRoleArn": "arn:aws:iam::ACCOUNT_ID_HERE:role/ecsTaskExecutionRole",
  "containerDefinitions": [
    {
      "dnsSearchDomains": null,
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/elasticmq",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "entryPoint": null,
      "portMappings": [
        {
          "hostPort": 0,
          "protocol": "tcp",
          "containerPort": 9324
        }
      ],
      "command": null,
      "linuxParameters": null,
      "cpu": 0,
      "environment": [],
      "ulimits": null,
      "dnsServers": null,
      "mountPoints": [],
      "workingDirectory": null,
      "dockerSecurityOptions": null,
      "memory": null,
      "memoryReservation": null,
      "volumesFrom": [],
      "image": "lobster1234/elasticmq-docker",
      "disableNetworking": null,
      "healthCheck": null,
      "essential": true,
      "links": null,
      "hostname": null,
      "extraHosts": null,
      "user": null,
      "readonlyRootFilesystem": null,
      "dockerLabels": null,
      "privileged": null,
      "name": "elasticmq-container"
    }
  ],
  "placementConstraints": [],
  "memory": "256",
  "taskRoleArn": "arn:aws:iam::ACCOUNT_ID_HERE:role/ecsTaskExecutionRole",
  "compatibilities": [
    "EC2"
  ],
  "taskDefinitionArn": "arn:aws:ecs:us-east-1:ACCOUNT_ID_HERE:task-definition/elasticmq:1",
  "family": "elasticmq",
  "requiresAttributes": [
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.task-iam-role"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "ecs.capability.execution-role-awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
    }
  ],
  "requiresCompatibilities": [
    "EC2"
  ],
  "networkMode": "bridge",
  "cpu": "128",
  "revision": 1,
  "status": "ACTIVE",
  "volumes": []
}

Next, you’d create an ECS Service using this task definition.

When you’d create a service using this task, create only 1 instance. You’d create a new listener on the ALB (preferably port 9324) with path as /. Please edit the health check for the target group to check for a HTTP 404 and NOT HTTP 200 under Success Codes.

If not, then you’d end up getting unhealthy hosts and infinite drain and initial cycles.

Verify

I’ll use aws-shell to verify that our ElasticMQ service is working as we expect.

aws> sqs create-queue --queue-name manish_test_queue --endpoint-url http://your_alb_dns_name:9324
{
    "QueueUrl": "http://your_alb_dns_name:9324/queue/manish_test_queue"
}
aws> sqs list-queues --endpoint-url http://your_alb_dns_name:9324
{
    "QueueUrls": [
        "http://your_alb_dns_name:9324/queue/manish_test_queue"
    ]
}

aws> sqs send-message --queue-url http://your_alb_dns_name:9324/queue/manish_test_queue --endpoint-url http://your_alb_dns_name:9324 --message-body "foo"
{
    "MD5OfMessageBody": "acbd18db4cc2f85cedef654fccc4a4d8",
    "MD5OfMessageAttributes": "d41d8cd98f00b204e9800998ecf8427e",
    "MessageId": "671a0afc-56b3-4db0-8b7b-e8a1fedf574f"
}

aws> sqs receive-message --queue-url http://your_alb_dns_name:9324/queue/manish_test_queue --endpoint-url http://your_alb_dns_name:9324
{
    "Messages": [
        {
            "Body": "foo",
            "ReceiptHandle": "671a0afc-56b3-4db0-8b7b-e8a1fedf574f#ebff7f56-626c-409c-bc56-8c2ea509cff9",
            "MD5OfBody": "acbd18db4cc2f85cedef654fccc4a4d8",
            "MessageId": "671a0afc-56b3-4db0-8b7b-e8a1fedf574f"
        }
    ]
}

Hope this helps with running ElasticMQ under ECS. Questions or comments? Please let me know!

Comments