Hello! I've seen this anti-pattern scattered around plenty DevOps code, especially in AWS lambda functions: Vendoring libraries like requests into other libraries like botocore is arguably an anti-pattern in general, but reaching in to botocore and importing it in your own code is definitely one. Here are some of the reasons: The maintainers may un-vendor … Continue reading Don’t Import requests From botocore.vendored
Category: Boto3
CloudWatch Logs Structured as JSON with Python Lambda Functions
Hello! If you're setting up JSON logging in a script and instead of a lambda function, check out this instead. The pattern is a little simpler. Recently, I've been switching to logs structured as JSON. That means output like this: Instead of this: This way, your log processor can reference keys in a JSON object … Continue reading CloudWatch Logs Structured as JSON with Python Lambda Functions
CodePipeline: Python AWS Lambda Functions Without Timeouts
Hello! Today we're going to cover how to add Python AWS lambda functions to CodePipeline, and specifically how to do that without getting stuck in timeout loops you can't cancel. Copy/pastable code first, details below. Replace the two highlighted lines with the code you actually need to run in the pipeline. The commented raise is … Continue reading CodePipeline: Python AWS Lambda Functions Without Timeouts
How to Paginate in boto3: Use Collections Instead
Hello! When working with boto3, you'll often find yourself looping. Like if you wanted to get the names of all the objects in an S3 bucket, you might do this: But, methods like list_objects_v2 have limits on how many objects they'll return in one call (up to 1000 in this case). If you reach that … Continue reading How to Paginate in boto3: Use Collections Instead
Boto3 Best Practices: Assert to Stop Silent Failures
Good morning! Today's post covers a pattern I use to increase my confidence that my infrastructure code is working. It turns silent errors into loud ones. I've handled plenty of code that runs without errors but still ends up doing the wrong thing, so I'm never really sure if it's safe to go to sleep … Continue reading Boto3 Best Practices: Assert to Stop Silent Failures
Simple Python boto3 Logging
Hello! If you're writing a lambda function, check out this article instead. The best way to log output from boto3 is with Python's logging library. The core docs have a nice tutorial. If you use print() statements for output, all you'll get from boto is what you capture and print yourself. But, boto does a lot of internal … Continue reading Simple Python boto3 Logging
Lambda: Simple boto3 CloudWatch Logs
Good morning! If you're writing a regular script (i.e. not a lambda function), check out this article. This pattern outputs traditional delimited strings. If you want to upgrade that into output structured as JSON objects, check out this article. For those custom cases that don't fit into Terraform or CloudFormation, a little bit of Python and some … Continue reading Lambda: Simple boto3 CloudWatch Logs
The Better Alexa Quick Start
You've seen the Life Coach. That was my second Alexa project. The one I used to learn the platform. I began with Amazon's quick start tutorial (update August 2019: broken link to quick start removed, the AWS tutorials have evolved since this article was written), but I didn't like the alexa-skills-kit-color-expert-python example code. It feels like … Continue reading The Better Alexa Quick Start