Getting GitHub Unified Diff Emails using AWS Lambda and API Gateway

GitHub Unified Diff Email

I just wanted to share some code to get a unified diff email from GitHub webhooks. The above image is the end result due to my lack of design skills nevertheless the source code is available for those who wish to customize it.

The interesting thing about this little interesting project is that Sydney just got AWS API Gateway and Lambda yay. So I wanted to combine the two and have a server-less way of achieving the unified diff. Basically API Gateway will have an endpoint that the GitHub webhook will hit with its push payload. Then API Gateway will execute our lambda function which figures out the diff and then sends off the email. It sends the email via SES but you can use any SMTP server. SES is not yet available in APAC-2 but I set it up in another region and just exposed it as an SMTP Server.

For the more experienced folks who want to skip the AWS setup stuff, here's the source code link:

The rest of this will assume knowledge of some AWS basics like IAM. I'll try to post only the bits that might be tricksy. The AWS Console makes it pretty easy to walk through the steps for everything else.

Update July 22, 2016####

I found that there was quite a few steps involved in setting this up but it really helped me understand API Gateway and Lambda functions much better so this post is still good as a reminder of that.

I created another repository though that uses the Serverless framework which makes deployment a cinch. Check out the instructions and the repository here:

The API Mapping, IAM Role, Lambda function and API Gateway will be created for you.

Setup API Gateway#

Sign into the AWS Console and Create a new API Gateway:
create api gateway

Under resources we want to add a new endpoint but if we select lambda function, it will ask you to point to a lambda function that doesn't yet exist. So let's create that Lambda function as well.

Lambda Function##

Create Lambda Function

And configure the trigger:
configure lambda trigger to api gateway

For testing purposes we can leave Security blank and authenticate whether or not the request is coming from GitHub via the Hmac that they provide us in the header.
The Hmac is a combination of the secret key which we'll see later and the POST body payload.

For the function itself you can choose to either upload a zip file or point to your own S3 bucket. Just make sure to include the node_modules folder when uploading the zip and do not zip up at the folder level.

Before uploading make sure you modify the config file:

        "host": "SMTP SERVER",
        "username": "USERNAME",
        "password": "PASSWORD",
        "from": "",
        "to": ","
        "secret": "github web hook secret"

Remember that secret, we'll need it near the end when we setup the GitHub webhook.

Mapping Template####

Now back at API Gateway we'll see the new endpoint pointing straight to the Lambda function. First configure the POST end point and then click on the Integration function. When API Gateway invokes the function we need to tell it to map the headers and body payload to the function so that we have access to it. Luckily AWS provides a pre-defined template we can use in a dropdown:

configure lambda integration endpoint
GitHub provides the content as application/json which we have to manually type into the Content-Type for now.

Go ahead and publish this to production under the Actions dropdown. We should then get it's URL:

API Gateway production URL

Granting The API Gateway Lambda Execution Rights####

However, it cannot yet invoke the lambda function because we haven't given it permission to yet. We'll need to create an IAM execution role with those rights.

IAM Role execution rights for API gateway and lambda

Attach this policy to the lambda function. Notice the Resource/ARN reflects any environment including the production one as noted by the asterisk. This resource is the one we published above for API Gateway.

GitHub WebHook Settings##

Once everything is setup we'll need to navigate to the Settings tab in our repo and have something similar to this:
Github webhook settings

The Payload URL is our API Gateway Endpoint and the secret is the one we put in our config file earlier. This is so that the lambda function verifies it's coming from this repo.

To test it all out simply commit a change to the repo. Notice at the bottom that GitHub logs recent deliveries. The green checkmark is good and we can take a gander at the payload. For testing we can also have it replay the request too! Nice.

I hope that helps!