Lambda: Building Python 3 Packages

AWS Lambda, Python

Hello!

This is a guide for building lambda zip packages with pip dependencies in Python 3. It expands the AWS guide to:

  • Build your code if it’s either a single-file Python module or a pip-installable Python package (e.g. contains a setup.py).
  • Track pip dependencies for single-file Python modules in the standard requirements.txt file.
  • Show the file tree for each case and the lambda handler path that results from it.
  • Provide a copy/paste-able script for each case.

Notes:

  • Each package.sh can be run from any directory, but will always produce a function.zip in the same directory as itself.
  • Shell tested in the Apple Mac OS X 10.14.6 terminal.
  • Lambda packages tested in Python 3.7 functions.

Single-File Modules

File tree:

.
├── function.py  (defines a 'lambda_handler' function)
├── package.sh
└── requirements.txt

Lambda handler path: function.lambda_handler

package.sh:

#!/usr/bin/env bash

# Builds a lambda package from a single Python 3 module with pip dependencies.
# This is a modified version of the AWS packaging instructions:
# https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html#python-package-dependencies

# https://stackoverflow.com/a/246128
SCRIPT_DIRECTORY="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

pushd $SCRIPT_DIRECTORY > /dev/null

rm -rf .package function.zip
mkdir .package

pip install --target .package --requirement requirements.txt

pushd .package > /dev/null
zip --recurse-paths ${SCRIPT_DIRECTORY}/function.zip .
popd > /dev/null

zip --grow function.zip function.py

popd > /dev/null

Pip-Installable Python Packages

In lambda with Python 2 this used to be harder, but in lambda with Python 3 it’s as easy as a flat module. If you’re still on Python 2, it’s time to upgrade.

File tree:

.
├── function
│   ├── __init__.py
│   └── function.py  (defines a 'lambda_handler' function)
├── package.sh
└── setup.py

Lambda handler path: function.function.lambda_handler

package.sh:

#!/usr/bin/env bash

# Builds a lambda package from a pip-installable Python 3 package.
# This is a modified version of the AWS packaging instructions:
# https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html#python-package-dependencies

# https://stackoverflow.com/a/246128
SCRIPT_DIRECTORY="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

pushd $SCRIPT_DIRECTORY > /dev/null

rm -rf .package function.zip
mkdir .package

pip install --target .package .

pushd .package > /dev/null
zip --recurse-paths ${SCRIPT_DIRECTORY}/function.zip .
popd > /dev/null

popd > /dev/null

That’s it! In the latest versions of lambda it’s pretty easy, you just have to do a little fiddling to make the scripts easy to use and to figure out the handler paths.

Happy automating!

Adam

Check out these related posts:

Published