Hello! It takes a few pieces to assemble a working lambda action for CodePipeline. I like to start from a simple example and build up to what I need. Here's the code I use as a starting point. First, a few notes: My pipeline lambda functions are usually small, often only a few dozen lines … Continue reading CodePipeline lambda Function: Complete Example
Category: Boto3
CloudFormation Custom Resource: Complete Example
Hello! It takes a few pieces to assemble a working CloudFormation Custom Resource. I like to start from a simple example and build up to what I need. Here's the code I use as a starting point. First, a few notes: My custom resources are usually small, often only a few dozen lines (more than … Continue reading CloudFormation Custom Resource: Complete Example
Boto3: NoRegionError
Hello! Read on if your boto3 code is throwing this error: But first, two reasons why you might only get this error in some situations: AWS lambda sets the region automatically. Code that works in lambda may still throw NoRegionError locally. Some AWS services are global (don't have regions). Like S3: This code will work … Continue reading Boto3: NoRegionError
Don’t Import requests From botocore.vendored
Hello! I've seen this anti-pattern scattered around plenty DevOps code, especially in AWS lambda functions: Vendoring libraries like requests into other libraries like botocore is arguably an anti-pattern in general, but reaching in to botocore and importing it in your own code is definitely one. Here are some of the reasons: The maintainers may un-vendor … Continue reading Don’t Import requests From botocore.vendored
CloudWatch Logs Structured as JSON with Python Lambda Functions
Hello! If you're setting up JSON logging in a script and instead of a lambda function, check out this instead. The pattern is a little simpler. Recently, I've been switching to logs structured as JSON. That means output like this: Instead of this: This way, your log processor can reference keys in a JSON object … Continue reading CloudWatch Logs Structured as JSON with Python Lambda Functions
CodePipeline: Python AWS Lambda Functions Without Timeouts
Hello! If you're new to CodePipeline lambda actions check out this complete example first. There’s a gotcha when writing CodePipeline lambda functions that’s easy to miss and if you miss it your pipeline can get stuck in timeout loops that you can't cancel. Here’s how to avoid that. This article assumes you're familiar with CodePipeline … Continue reading CodePipeline: Python AWS Lambda Functions Without Timeouts
How to Paginate in boto3: Use Collections Instead
Hello! When working with boto3, you'll often find yourself looping. Like if you wanted to get the names of all the objects in an S3 bucket, you might do this: But, methods like list_objects_v2 have limits on how many objects they'll return in one call (up to 1000 in this case). If you reach that … Continue reading How to Paginate in boto3: Use Collections Instead
Boto3 Best Practices: Assert to Stop Silent Failures
Good morning! Today's post covers a pattern I use to increase my confidence that my infrastructure code is working. It turns silent errors into loud ones. I've handled plenty of code that runs without errors but still ends up doing the wrong thing, so I'm never really sure if it's safe to go to sleep … Continue reading Boto3 Best Practices: Assert to Stop Silent Failures
Python boto3 Logging
Hello! If you're writing a lambda function, check out this article instead. The best way to log output from boto3 is with Python's logging library. The core docs have a nice tutorial. If you use print() statements for output, all you'll get from boto is what you capture and print yourself. But, boto does a lot of internal … Continue reading Python boto3 Logging
Lambda: boto3 CloudWatch Logs
Good morning! If you're writing a regular script (i.e. not a lambda function), check out this article. This pattern outputs traditional delimited strings. If you want to upgrade that into output structured as JSON objects, check out this article. For those custom cases that don't fit into Terraform or CloudFormation, a little bit of Python and some … Continue reading Lambda: boto3 CloudWatch Logs