Lambda: Building Python 3 Packages

AWS Lambda, Python

Hello!

This is a guide for building lambda zip packages with pip dependencies in Python 3. It expands the AWS guide to:

  • Build your code if it’s either a single-file Python module or a pip-installable Python package (e.g. contains a setup.py).
  • Track pip dependencies for single-file Python modules in the standard requirements.txt file.
  • Show the file tree for each case and the lambda handler path that results from it.
  • Provide a copy/paste-able script for each case.

Notes:

  • Each package.sh can be run from any directory, but will always produce a function.zip in the same directory as itself.
  • Shell tested in the Apple Mac OS X 10.14.6 terminal.
  • Lambda packages tested in Python 3.7 functions.

Single-File Modules

File tree:

.
├── function.py  (defines a 'lambda_handler' function)
├── package.sh
└── requirements.txt

Lambda handler path: function.lambda_handler

package.sh:

#!/usr/bin/env bash

# Builds a lambda package from a single Python 3 module with pip dependencies.
# This is a modified version of the AWS packaging instructions:
# https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html#python-package-dependencies

# https://stackoverflow.com/a/246128
SCRIPT_DIRECTORY="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

pushd $SCRIPT_DIRECTORY > /dev/null

rm -rf .package function.zip
mkdir .package

pip install --target .package --requirement requirements.txt

pushd .package > /dev/null
zip --recurse-paths ${SCRIPT_DIRECTORY}/function.zip .
popd > /dev/null

zip --grow function.zip function.py

popd > /dev/null

Pip-Installable Python Packages

In lambda with Python 2 this used to be harder, but in lambda with Python 3 it’s as easy as a flat module. If you’re still on Python 2, it’s time to upgrade.

File tree:

.
├── function
│   ├── __init__.py
│   └── function.py  (defines a 'lambda_handler' function)
├── package.sh
└── setup.py

Lambda handler path: function.function.lambda_handler

package.sh:

#!/usr/bin/env bash

# Builds a lambda package from a pip-installable Python 3 package.
# This is a modified version of the AWS packaging instructions:
# https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html#python-package-dependencies

# https://stackoverflow.com/a/246128
SCRIPT_DIRECTORY="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

pushd $SCRIPT_DIRECTORY > /dev/null

rm -rf .package function.zip
mkdir .package

pip install --target .package .

pushd .package > /dev/null
zip --recurse-paths ${SCRIPT_DIRECTORY}/function.zip .
popd > /dev/null

popd > /dev/null

That’s it! In the latest versions of lambda it’s pretty easy, you just have to do a little fiddling to make the scripts easy to use and to figure out the handler paths.

Happy automating!

Adam

If this was helpful and you want to save time by getting “copy and paste” patterns for Python in your inbox, subscribe here. If you don’t want to wait for the next one, check out these:

Published