[Pycon] [new paper] "Toshiya Doi" - Share Python libraries between AWS Lambda Functions and AWS Lambda Layers and continuously deploy wi

[info a pycon.it]

Title: Share Python libraries between AWS Lambda Functions and AWS Lambda Layers and continuously deploy wi
Duration: 45 (includes Q&A)
Q&A Session: 15
Language: en
Type: Talk

Abstract: The more you use AWS managed services, the more you need to handle event notifications from these services. In addition, you might want to send them to somewhere you can check. We use various AWS managed services which can send events via, for example, Amazon CloudWatch Events to us and they almost always involve a requirement: handling an event with an AWS Lambda Function and send it to Slack. This will be a problem when we manage multiple AWS Lambda Functions, projects and environments (staging and production) because we need to handle an event and send it to Slack everywhere in a slightly different way depending on the projects requirements. The problem gets more troublesome when we add another AWS managed service.

In this talk, I will introduce to you how you can share Python libraries between AWS Lambda Functions with AWS Lambda Layers particularly for handling events from AWS managed services and building a continuous deployment flow with Terraform.

 - To handle an event from an AWS managed service, you need to parse the event to extract a message, format the message and send it to Slack. What you need to do is the same in each project and environment. However, what you want to show for each project and environment on Slack is slightly different. As a result, we needed to have shared codes and deploy it in multiple projects. AWS announced “AWS Lambda Layers” at AWS re:Invent in November, 2018. It allows us to share Python libraries between multiple AWS Lambda Functions instead of packaging them for each AWS Lambda Function so that we can reuse code as much as possible. For example, I will firstly show you how we utilize AWS Lambda Layers to avoid repeated work for event handling. Since Python source code directory will matter to use AWS Lambda Layers and build continuous deployment flow, I will also show you how we organize Python source code.

 - As many tools for serverless application are emerging these days, there are various ways to deploy an AWS Lambda Function. While these tools focus on deploying AWS resources related to AWS Lambda Functions (Amazon API Gateway, Amazon SNS, etc...), we want to manage all AWS resources that are glued with AWS Lambda Functions (AWS IAM Role and AWS managed services that send events) as code in a project. I will show you one option how you can manage AWS resources including packaging Python libraries for AWS Lambda Functions and layers with Terraform. With this approach, you can continuously deploy your Python libraries and infrastructure as your AWS infrastructure changes.

Through this talk, you will see how we share Python libraries between multiple AWS Lambda Functions to avoid repeated work for event handling with Lambda Layers and continuously deploy them as a part of our AWS infrastructure with Terraform for multiple projects and environments, which I assume cloud infrastructure engineers often need.

Tags: [u'#infrastructure-as-code', u'#cloud-service', u'#terraform', u'#serverless', u'#aws', u'#continuous-deploymemt']