Command Line Python

How to develop and install a command line utility. The goal is to provide straightforward access to a custom command line utility, written in Python, on a single machine to which all intended users of the utility have SSH access.

Before writing any code, it is a good idea to think about the user’s interface to the program: its command line options. As the utility will be called from the command line by anyone on the shared system, it will be in everyone’s PATH (likely under ‘/usr/local/bin’), which is comparable to a ‘global namespace’ for command line utilities. To make the tool easier to find and use, and avoid polluting the PATH, it has a single entry point; a single command in the ‘global namespace’, which exposes all of the functionality of the tool. Most common command line tools work this way; for instance, ‘apt’ uses sub commands to access entirely different functions. If you already have a bunch of scripts implementing each sub command’s function, some refactoring may be needed to create a single entry point. Consider the name of the command line utility carefully. If you are writing for an organisation, it might be memorable for users, and mark out the utility as being custom to the organisation rather than provided by the system or a third party, to prepend a shortened version of the organisation name.

The build environment is easy to set up, but there are extra steps as opposed to a collection of scripts that are to be run purely from the build environment. Start with a directory for the project:

mkdir cliprogram
cd cliprogram
python3 -m venv .
source bin/activate

Create a source tree:

mkdir -p src/cliprogram

All source files will live in ‘src/cliprogram’. Now, create a few required files:

touch src/cliprogram/cli.py
touch src/cliprogram/__init__.py
touch src/cliprogram/__main__.py

‘cli.py’ will be the entry point for the program. Its filename doesn’t matter. The other two files are important for Python packaging; ‘__init__.py’ is empty, ‘__main__.py’ is a shim that calls the ‘cli()’ method. The complete content of ‘__main__.py’ is:

if __name__ == "__main__":
    from cliprogram.cli import cli
    cli()

Create a ‘pyproject.toml’ file in the project directory. The content of the file will look like this:

[project]
name = "cliprogram"
version = "0.1"
requires-python = ">=3.10"
dependencies = [
    "netmiko==4.6.0"
]
readme = "readme"
authors = [
    { name = "Your Name", email = "your.name@cliprogram" }
]

[project.scripts]
cliprogram = "cliprogram.cli:cli"

The ‘[project.scripts]’ section produces a global ‘cliprogram’ utility.

The project should be implemented in ‘cli.py’, and other files under ‘src/cliprogram/’. When importing other files under ‘src/cliprogram/’, remember to prefix the module name with ‘cliprogram.’, or they won’t be found after the program is installed.

Once the project is implemented, it can be run immedidately with ‘pipx’. ‘pipx’ manages isolated virtual environments for Python programs to run in without interfering with other Python programs. Note that, as pipx manages virtual environments itself, it is no longer necessary to run ‘source bin/activate’ to activate the local virtual environment. Run the local program with:

pipx run --spec . cliprogram

To install the program for the current user, run:

pipx install . --force

The ‘--force’ option installs the program even if a matching version is found. This is useful as it saves updating the version number in the ‘pyproject.toml’ file every time a change is made.

Finally, to install the program globally, for all users to use, switch to the root user and run:

PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install .

This installs the utility and all of its virtual environment into ‘/opt/pipx’ and places a shim in ‘/usr/local/bin/cliprogram’ to activate the virtual environment and call the command line utility.

Verify that the environment has been installed using ‘pipx list’. To verify virtual environments that have been installed globally, use the environment variables, as above:

PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx list

Only read access is required, so it is not necessary to beome root to run this command.