We continue to build our Lambda functions beyond the Hello World that we saw in the previous post. Our end goal is to build an API, and make it available via Amazon API Gateway. We will build the foundation of our Lambda-API-Gateway set up in this tutorial. We will also set up DynamoDB. Feel free to replace DynamoDB with any persistance layer, or even a mock.

The source code listed in this this post can be found here.

Set up Serverless Framework

Please refer to AWS Lambda with Serverless Framework and Java/Maven - Part 1.

Once serverless framework is set up, come back here to build a new project.

The API

For the sake of simplicity, here is the schema - There are payment accounts, every account has transactions, and every transaction has an amount, a transaction ID, and a transaction date.

The focus is here on simplicity of the example, as we focus on the AWS framework, deployment, and management.

Here is the API Schema -

POST /accounts/:account_id/transactions

{
  "transaction_id":"4f9a6c39-f3ba-4b62-9e30-ab408dd43933",
  "amount": 22.44
}

GET /accounts/:account_id/transactions

The date is UTC milliseconds, as we are not bothering with dateformats and conversions here.

[
   {
      "account_id":"1234",
      "transaction_date":1492160332168,
      "transaction_id":"4f9a6c39-f3ba-4b62-9e30-ab408dd43933",
      "amount":3.5
   },
   {
      "account_id":"1234",
      "transaction_date":1492160332122,
      "transaction_id":"a411e6bd-941a-4ec9-9ff6-5fe816759256",
      "amount":100.20
   }
]

Create a serverless project

Start by creating a new Serverless project, or you may re-use the one created during the Part-1 of this series.

bash-3.2$ mkdir transactions-api
bash-3.2$ serverless create -t aws-java-maven -n transactions-api
Serverless: Generating boilerplate...
 _______                             __
|   _   .-----.----.--.--.-----.----|  .-----.-----.-----.
|   |___|  -__|   _|  |  |  -__|   _|  |  -__|__ --|__ --|
|____   |_____|__|  \___/|_____|__| |__|_____|_____|_____|
|   |   |             The Serverless Application Framework
|       |                           serverless.com, v1.10.0
 -------'

Serverless: Successfully generated boilerplate for template: "aws-java-maven"

Update the POM and YAML

Since serverless creates a project called “hello”, lets go modify it to sound a bit more…functional.

First, change the pom.xml

 <groupId>com.serverless</groupId>
  <artifactId>hello</artifactId>
  <packaging>jar</packaging>
  <version>dev</version>
  <name>hello</name>

to

 <groupId>com.serverless</groupId>
  <artifactId>transactions-api</artifactId>
  <packaging>jar</packaging>
  <version>dev</version>
  <name>transactions-api</name>

Also, since the project will use DynamoDB, add the AWS Java DynamoDB SDK dependency in pom.xml.

Add the below just before </dependencies>.

 <dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-java-sdk-dynamodb</artifactId>
  <version>1.11.119</version>
</dependency>

Next, in serverless.yml, we describe the lambda handlers, and the deployment artifact name. Change the following

# you can add packaging information here
package:
  artifact: target/hello-dev.jar

functions:
  hello:
    handler: com.serverless.Handler

to

# you can add packaging information here
package:
  artifact: target/transactions-api-dev.jar

functions:
  get-transactions:
    handler: com.serverless.GetTransactionsHandler
  post-transaction:
    handler: com.serverless.PostTransactionsHandler

Configure DynamoDB Resource and Lambda Role

Read more on DynamoDB here .

Since we’re also creating a DynamoDB Table, lets have serverless create it for us. We will also need to set permissions for our lambda function in order to access this table (execution role).

More details on Lambda Permissions Model here .

Serverless takes care of creating a basic lambda execution role, however, it only has CloudWatch Log permissions. So let us add permissions for DynamoDB as well.

To do so, add the following to serverless.yml above the commented out section that says # you can add statements to the Lambda function's IAM Role here.

Make sure this is under the provider section, so indent accordingly.. Refer to the code in the github repo to verify.

iamRoleStatements:
   - Effect: "Allow"
     Action:
       - "dynamodb:*"
     Resource: "*"

To create the table, at the bottom of the serverless.yml, add this CloudFormation section. This represents the schema. Since the serverless user (created in Part-1) in IAM has CloudFormation permissions across all resources, the framework will be able to create this resource via CloudFormation. As you can see, we are using account_id as the HASH key, and transaction_date as our RANGE key. Other attributes like transaction_id and amount do not need to be declared.

resources:
  Resources:
    transactionsTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: transactions_table
        AttributeDefinitions:
          - AttributeName: account_id
            AttributeType: S
          - AttributeName: transaction_date
            AttributeType: S
        KeySchema:
          - AttributeName: account_id
            KeyType: HASH
          - AttributeName: transaction_date
            KeyType: RANGE
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1

Now we code!

I use IntelliJ, but any IDE that works with a Maven project will do. In IntelliJ, click Open, then select the pom.xml and pick Open as Project.

Since there are declared 2 handlers GetTransactionsHandler.java and PostTransactionsHandler.java, copy the auto-generated Hander.java as those two new files. There are now 3 Java files under src/main/java/com/serverless. We will modify the copied files to build our API.

DynamoDB POJO

Lets create the POJO we will need for our API. We use com.serverless.data package to hold it. Please note that we use DynamoDB annotations in this POJO to make our life easier.

Transaction.java - This POJO represents the DynamoDB Item.

package com.serverless.data;

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable;

import java.util.Date;

@DynamoDBTable(tableName = "transactions_table")
public class Transaction {

    @DynamoDBHashKey(attributeName = "account_id")
    String account_id; //Hash Key

    @DynamoDBRangeKey(attributeName = "transaction_date")
    Date transaction_date; //Range Key

    @DynamoDBAttribute(attributeName = "transaction_id")
    String transaction_id;

    @DynamoDBAttribute(attributeName = "amount")
    Float amount;

    public String getAccount_id() {
        return account_id;
    }

    public void setAccount_id(String account_id) {
        this.account_id = account_id;
    }

    public Date getTransaction_date() {
        return transaction_date;
    }

    public void setTransaction_date(Date transaction_date) {
        this.transaction_date = transaction_date;
    }

    public String getTransaction_id() {
        return transaction_id;
    }

    public void setTransaction_id(String transaction_id) {
        this.transaction_id = transaction_id;
    }

    public Float getAmount() {
        return amount;
    }

    public void setAmount(Float amount) {
        this.amount = amount;
    }
}

DynamoDB Adapter

Since our API uses DynamoDB, let us create a class to handle all DynamoDB work. Lets call it DynamoDBAdapter and put it under com.serverless.db package. We will make it a quick and dirty singleton. This provides 2 simple methods - to get transactions, and to store transactions. It uses DynamoDBMapper Class from AWS Java SDK.

package com.serverless.db;

import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBQueryExpression;
import com.amazonaws.services.dynamodbv2.model.AttributeValue;
import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.CreateTableResult;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;
import com.serverless.data.Transaction;
import org.apache.log4j.Logger;

import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class DynamoDBAdapter {

    private Logger logger = Logger.getLogger(this.getClass());

    private final static DynamoDBAdapter adapter = new DynamoDBAdapter();

    private final AmazonDynamoDB client;

    private DynamoDBAdapter() {
        client = AmazonDynamoDBClientBuilder.standard().withEndpointConfiguration(
                new AwsClientBuilder.EndpointConfiguration("https://dynamodb.us-east-1.amazonaws.com", "us-east-1"))
                .build();
        logger.info("Created DynamoDB client");
    }

    public static DynamoDBAdapter getInstance(){
        return adapter;
    }

    public List<Transaction> getTransactions(String accountId) throws IOException {
        DynamoDBMapper mapper = new DynamoDBMapper(client);
        Map<String, AttributeValue> vals = new HashMap<>();
        vals.put(":val1",new AttributeValue().withS(accountId));
        DynamoDBQueryExpression<Transaction> queryExpression = new DynamoDBQueryExpression<Transaction>()
                .withKeyConditionExpression("account_id = :val1 ")
                .withExpressionAttributeValues(vals);
        return mapper.query(Transaction.class, queryExpression);
    }


    public void putTransaction(Transaction transaction) throws IOException{
        DynamoDBMapper mapper = new DynamoDBMapper(client);
        mapper.save(transaction);
    }

}

Lambda Function Handlers

Now we get to the fun part - writing our lambda handler. We had copied the Handler that Serverless created for us. Let us start there.

Referring to AWS Documentation, we can extract request parameters from the request that the API Gateway sends us “as is”.

API Gateway Proxy For Lambda I/O format.

We need to extract our path parameter, account_id for the POST (and GET) request from pathParameters element in this JSON.

This part is a little tricky, as some elements are strings to be parsed as JSONs, others are serialized HashMaps. Please be sure to read the documentation linked above.

Here is the code for PostTransactionsHandler’s handleRequest function.

@Override
public ApiGatewayResponse handleRequest(Map<String, Object> input, Context context) {
    LOG.info("received: " + input);
    //lets get our path parameter for account_id
    try{
      ObjectMapper mapper = new ObjectMapper();
      Map<String,String> pathParameters =  (Map<String,String>)input.get("pathParameters");
      String accountId = pathParameters.get("account_id");
      Transaction tx = new Transaction();
      tx.setAccount_id(accountId);
      JsonNode body = mapper.readTree((String) input.get("body"));
      float amount = (float) body.get("amount").asDouble();
      String txId = body.get("transaction_id").asText();
      tx.setAmount(amount);
      tx.setTransaction_date(new Date(System.currentTimeMillis()));
      tx.setTransaction_id(txId);
      DynamoDBAdapter.getInstance().putTransaction(tx);
    } catch(Exception e){
      LOG.error(e,e);
      Response responseBody = new Response("Failure putting transaction", input);
      return ApiGatewayResponse.builder()
      .setStatusCode(500)
      .setObjectBody(responseBody)
      .setHeaders(Collections.singletonMap("X-Powered-By", "AWS Lambda & serverless"))
      .build();
    }
    Response responseBody = new Response("Transaction added successfully!", input);
    return ApiGatewayResponse.builder()
    .setStatusCode(200)
    .setObjectBody(responseBody)
    .setHeaders(Collections.singletonMap("X-Powered-By", "AWS Lambda & serverless"))
    .build();
}

In the code above, we’re getting the pathParameters as a Map and extracting account_id from it. Then we create a Transaction object and populate it, and call the DynamoDBAdapter to persist the object. We parse the body using Jackson, which is already available to us via AWS Java SDK.

Similarly, we code our GetTransactionsHandler’s handleRequest function.

@Override
public ApiGatewayResponse handleRequest(Map<String, Object> input, Context context) {
  LOG.info("received: " + input);
  List<Transaction> tx;
  try {
      Map<String, String> pathParameters = (Map<String, String>) input.get("pathParameters");
      String accountId = pathParameters.get("account_id");
      LOG.info("Getting transactions for " + accountId);
      tx = DynamoDBAdapter.getInstance().getTransactions(accountId);
    } catch (Exception e) {
      LOG.error(e, e);
      Response responseBody = new Response("Failure getting transactions", input);
      return ApiGatewayResponse.builder()
      .setStatusCode(500)
      .setObjectBody(responseBody)
      .setHeaders(Collections.singletonMap("X-Powered-By", "AWS Lambda & serverless"))
      .build();
   }
    return ApiGatewayResponse.builder()
    .setStatusCode(200)
    .setObjectBody(tx)
    .setHeaders(Collections.singletonMap("X-Powered-By", "AWS Lambda & serverless"))
    .build();
}

In this case, we’re calling DynamoDBAdapter to get the transactions for an account, and passing it as response body. We are not converting it to JSON or anything, hence you notice the transaction_date as a long.

We are done with coding now. Lets deploy this!

Deploying the two lambda functions

This is where Serverless shines - it takes care of uploading the artifact, creating the AWS resources and IAM roles, CloudWatch Logs, etc.

Build the jar, which will be the deployable artifact.

bash-3.2$ mvn clean install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building transactions-api dev
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.099 s
[INFO] Finished at: 2017-04-14T02:44:01-07:00
[INFO] Final Memory: 39M/224M
[INFO] ------------------------------------------------------------------------

Next, we go ahead deploy.

bash-3.2$ serverless deploy
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
.....
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading service .zip file to S3 (7.57 MB)...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
...........................
Serverless: Stack update finished...
Service Information
service: transactions-api
stage: dev
region: us-east-1
api keys:
  None
endpoints:
  None
functions:
  get-transactions: transactions-api-dev-get-transactions
  post-transaction: transactions-api-dev-post-transaction
bash-3.2$

If you want to dive deeper, check out the contents of .serverless folder. This has 2 CloudFormation templates that did the heavy lifting via Serverless Framework.

bash-3.2$ tree .serverless/
.serverless/
├── cloudformation-template-create-stack.json
└── cloudformation-template-update-stack.json

0 directories, 2 files

The create templte creates the S3 bucket to upload the artifact, and update template takes care of everything else. More details on CloudFormation here .

Unfortunately invoking these 2 functions will not be straightforward via CLI, as they expects the input request as a Map<String, Object>.

bash-3.2$ serverless invoke -f get-transactions
{
    "statusCode": 500,
    "body": "{\"message\":\"Failure getting transactions\",\"input\":{}}",
    "headers": {
        "X-Powered-By": "AWS Lambda & serverless"
    },
    "isBase64Encoded": false
}

Feel free to log into the AWS console, and look at the lambda just set up under Lambda section. Also, take a look at CloudFormation stacks. We can test it by creating a body similar to the AWS Lambda Proxy request from the documentation link above in the Console.

In the next and final part, we will wire up these 2 lambda functions to act as API endpoints.

Thanks for staying with me so far!