Python DevOps Code Error Checking: Lint with Pyflakes

Hello!

For those unfamiliar with linting (static analysis), read Dan Bader’s introduction.

There are several linters for Python, but when I’m doing DevOps I use Pyflakes. I love the opening sentence of its design principals:

Pyflakes makes a simple promise: it will never complain about style, and it will try very, very hard to never emit false positives.

I’m not generally rigid about style. And, when I enforce it, I use the code review process and not a static analysis tool. The Python interpreter doesn’t care about style. Style is for humans; humans are the best tools to analyze it. Linters turn what should be a human process into something robotic.

Style is especially hard to enforce in DevOps, where you’re often working with a mix of languages and frameworks and tools that all have different style conventions. For example, lots of folks use Chef in AWS. Chef is a Ruby framework. They also need lambda helper functions, but lambda doesn’t support Ruby so they write those functions in Python and now half their code is Ruby and half is Python. And that’s if you ignore all the HCL in their terraform modules… You can go insane trying to configure your linters to keep up with the variation.

More than that, in DevOps you’re not usually writing much code. A helper lambda function. A little boto3 script. If you’re writing lots of code, you’re probably making a mistake and you should be looking for an existing tool that already implements whatever you’re doing (terraform, troposphere, Ansible, Salt, paramiko, whatever).

Pyflakes is great because it catches syntax errors before execution time but won’t suck you in to The Bog of Style Sorrow. It’ll quickly tell you if you misplaced a quote mark, and then it exits. So if you do this:

bad_variable = 'Oops I forgot to close the string.

You get an error:

pyflakes test.py
test.py:1:51: EOL while scanning string literal
bad_variable = 'Oops I forgot to close the string.
                                                  ^

You also get some handy stuff like checking for unused imports. So if you do this:

import logging
good_variable = 'Huzzah! I remembered to close the string.'

You also get an error:

pyflakes test.py
test.py:1: 'logging' imported but unused

Crucially, Pyflakes will pass if you do this:

list_style_one = ['a', 'b']
list_style_two = [ 'a', 'b' ]

It’s a little funky to do both those patterns right next to each other, and if I were writing that code myself I’d fix it, but I don’t want my linter to error. The code works fine and I can read it easily. I prefer consistency, but not to the point that I want a robot to generate build failures.

I recommend running Pyflakes on all your Python DevOps code because it’s a quick win. Pretty much anything it errors on you should fix before you try to use the code, and it’s usually faster to run Pyflakes than to deploy a new version of the code and see if it works. I like things that are fast. 😁

Happy automating!

Adam

If this was helpful and you want to save time by getting “copy and paste” patterns for Python DevOps in your inbox, subscribe here. If you don’t want to wait for the next one, check out these: