Create a Battle Tested Python3 CLI Project

Every time you start a new Python CLI project, you have to set up the same basic structure.

This project structure that is about to be bestowed upon you is extremely trusty.

It has a testing framework, a settings management system, a batteries included CLI framework, code formatting, and possibly the best part is the project can import itself.

Meaning, say goodby to all those relative import errors.

By the end of this tutorial you will a have project tree that looks like this.

├── Pipfile
├── README.md
├── pytest.ini
├── settings.toml
├── setup.py
├── src
│   └── pycli
│       ├── cli.py
│       ├── config.py
│       └── main.py
├── template.secrets.toml
└── tests
    └── test_config.py

Prerequisites

• Python 3.10+
• Pipenv
• Git + Gibhub
• Terminal or terminal emulator
• DJ Airhorn Sound Effect (Link provided upon request)

Create working directory

Create a new directory you want to work in.

mkdir python-cli && cd python-cli

Once your working directory has changed lets use this bash script to stub out all the parts of the project that we need.

#!/bin/bash

# Create main directories
mkdir -p src/pycli tests

# Create files in the root directory
touch Pipfile README.md pytest.ini settings.toml setup.py template.secrets.toml .gitignore

# Create files in src/pycli
touch src/pycli/cli.py src/pycli/config.py src/pycli/main.py

# Create test file
touch tests/test_config.py

Stubbing out the project is half the battle. Now we can start building the pieces.

Command Line Entrypoint

This src/pycli/cli.py will be our entry point for the CLI.

It comes with a Click CLI command group and project imports.

Normally you would need to dig through pages of documentation to figure out how to setup command groups that invoke code from project imports.

You are in luck becuase that knowledge is being shared as an appetizer today.

import click
from pycli import main


@click.group()
def cli():
    pass


@cli.command()
@click.option("--name", "-n", default="World", help="Name to greet")
def greeting(name):
    main.greeting(name)

Create config.py

The src/pycli/config.py file is a file generated by dynaconf.

Dynaconf is a popular settings management package for Python3. Imagine .env files on steroids. We are just scratching the surface with it’s capabilities.

import pathlib
from dynaconf import Dynaconf

settings = Dynaconf(
    envvar_prefix="DYNACONF",
    settings_files=["settings.toml", ".secrets.toml"],
)

settings.PROJECT_ROOT = pathlib.Path(__file__).parents[2]

# `envvar_prefix` = export envvars with `export DYNACONF_FOO=bar`.
# `settings_files` = Load these files in the order.

Create main.py

This file represents the core logic of the project.

For this example we will keep it simple. In the real world you would probably have a lot more going on here such as data fetching and processing AND you would likely have it broken out into different modules.

def greeting(name):
    print(f"Hello, {name}!")

Create test_config.py

This file is a simple test to make sure that our settings are working.

It is always gratifying to fire off a pytest and see a few tests pass before you even start building out the project.

Oh! Did you notice that we are importing project files from our package into the test file?

This is the most efficient way to build a project.

import pathlib
from pycli.config import settings


def test_project_root():
    assert isinstance(settings.PROJECT_ROOT, pathlib.Path)
    assert settings.PROJECT_ROOT.exists()

def test_timezone():
    assert settings.TIMEZONE == "UTC"

def test_api_key():
    assert settings.API_KEY == "1234567890"

Create setup.py

This file is the secret sauce.

The setup.py file is what gives our project the abiltiy to import itself without getting the dreaded relative import errors.

It also establishes the entry point for our CLI using pcmd but you can pick any name you want instead of pcmd.

You might see this file and be wondering where are the package dependencies?

We outsource that to a Pipfile in a future step.

from setuptools import setup, find_packages

setup(
    name="pycli",
    version="0.1.0",
    packages=find_packages(where="src"),
    package_dir={"": "src"},
    include_package_data=True,
    entry_points={
        'console_scripts': [
            "pcmd=pycli.cli:cli",
        ],
    }
)

Create settings.toml

This file is used for any settings that are not secret.

TIMEZONE = "UTC"

Create template.secrets.toml

This is the template file we will use for secrets.

API_KEY = "1234567890"

Create README.md

This is a simple README file to get you started.

It’s always a good idea to have a README.md. Your future self will thank you. Plus if you make it look nice people want to hire you.

# A CLI template for Python projects.

- rename template.secrets.toml to .secrets.toml
- pipenv install --dev
- pipenv run pytest
- pipenv run black src/pycli
- pipenv run pycli --help

Create pytest.ini

This step is optional (but recommended for the Lazy Ones).

If you want to include the pytest.ini file it can be used to set the default flags for pytest (and more). -s allows for the printing of stdout and -v allows for the printing of verbose output. --tb=short allows for the printing of tracebacks in a clean way.

By far these flags are the most common to set. If we set them in our pytests.ini file we won’t have to include them on every invocation of pytest.

[pytest]
addopts = -s -v --tb=short

.gitignore

This is a simple .gitignore file to ignore the Pipfile and Pipfile.lock. We want to ignore these because we will not be committing them to the repository.

# Ignore dynaconf secret files
.secrets.*

# Ignore package files created from pipenv install -e .
*.egg-info/

Create Pipfile

This is our virtual environment file. Pipenv virtual environments are the bomb.

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
click = "*"
dynaconf = "*"
pycli = {file = ".", editable = true}

[dev-packages]
black = "*"
pytest = "*"

Install Dependencies

pipenv install --dev

Run the tests

pipenv run pytest

Run the project

pipenv run pycli --help

Format the code

pipenv run black src/pycli

Initialize git

git init
git add .
git commit -m "Initial commit"

DJ Airhorn

Congratulations! You have just created a battle tested Python3 CLI project.