7. Python Package Index

The Python Package Index is known as PyPi.

This is where python packages are downloaded from when you use pip install <package-name> and the code installs (even though <package-name> is not the name of a folder in your machine).

7.1. PyPi

Navigate to https://pypi.org/ and search for numpy. On the page you can see the current version of the package that is downloaded by default when you run pip install numpy.

The webpage also gives you the pip install command you need to use, and on the left if you click on download files, you will see things that look familiar:

The source distribution (a tar.gz file) containing the source code of the package.
The wheel distribution (a .whl file) containing a binary version of the package.

We saw both of these and how to build them in the previous parts of the course.

In the case of numpy you see that there are many wheels. This is because numpy is a truly compiled pakage that is not just pure Python (it uses C extensions). We will see more of this later in the course, when we talk about multi-language programming and cibuildwheel.

When you click on release history you will see the many different versions of the package that have been released and the dates they were released. You can click on any of the version to see details and how to install that specific version. For instance, if you click on numpy-1.26.4 you will see that the installation command is pip install numpy==1.26.4.

7.2. Installing a package from PyPi

To install a package from PyPi do, in a terminal:

pip install <package-name>

For example:

[1]:

pip install numpy

Requirement already satisfied: numpy in /Users/boris/opt/miniconda3/lib/python3.9/site-packages (2.0.2)
Note: you may need to restart the kernel to use updated packages.

Above, you see that the package is already installed. and you see the version number in brackets at the end.

To uninstall a package do:

pip uninstall <package-name>

For example:

[2]:

pip uninstall numpy

Found existing installation: numpy 2.0.2
Uninstalling numpy-2.0.2:
  Would remove:
    /Users/boris/opt/miniconda3/bin/f2py
    /Users/boris/opt/miniconda3/bin/numpy-config
    /Users/boris/opt/miniconda3/lib/python3.9/site-packages/numpy-2.0.2.dist-info/*
    /Users/boris/opt/miniconda3/lib/python3.9/site-packages/numpy/*
Proceed (Y/n)?

and you are prompted with proceed, to which you can press y to confirm. After which, you would see:

Successfully uninstalled numpy-2.0.2

To install a specific version of a package from PyPi do:

pip install <package-name>==<version>

For example:

[5]:

pip install numpy==2.0.2

Collecting numpy==2.0.2
  Using cached numpy-2.0.2-cp39-cp39-macosx_14_0_arm64.whl (5.3 MB)
Installing collected packages: numpy
Successfully installed numpy-2.0.2
Note: you may need to restart the kernel to use updated packages.

As you can see, it is always the wheel distribution that is downloaded and installed (or was downloaded, in which case it uses cache). PyPi automatically selects the wheel that is appropriate for your operating system. Only when no wheel is available for your OS or Python version, the source distribution is downloaded and installed (and this can take a while and risks failing).

Good practice when you distribute a package is to provide all possible wheels for all major operating systems and Python versions.

7.3. Installing multiple packages

You can install and uninstall multiple packages at the same time:

pip install <package-name1> <package-name2>

or

pip uninstall <package-name1> <package-name2>

7.3.1. Package dependencies

Package dependencies (as set in the configuration file of the package) are automatically installed when you install a package.

For instance,

[1]:

pip install scikit-learn

Requirement already satisfied: scikit-learn in /Users/boris/opt/miniconda3/lib/python3.9/site-packages (1.5.2)
Requirement already satisfied: threadpoolctl>=3.1.0 in /Users/boris/opt/miniconda3/lib/python3.9/site-packages (from scikit-learn) (3.1.0)
Requirement already satisfied: numpy>=1.19.5 in /Users/boris/opt/miniconda3/lib/python3.9/site-packages (from scikit-learn) (2.0.2)
Collecting scipy>=1.6.0
  Using cached scipy-1.13.1-cp39-cp39-macosx_12_0_arm64.whl (30.3 MB)
Requirement already satisfied: joblib>=1.2.0 in /Users/boris/opt/miniconda3/lib/python3.9/site-packages (from scikit-learn) (1.2.0)
Installing collected packages: scipy
Successfully installed scipy-1.13.1
Note: you may need to restart the kernel to use updated packages.

in this example, scikit-learn found all its dependencies (requirements) already installed and did not install them again, but it didn’t find scipy and installed it.

If for some reasons you don’t want dependencies to be installed automatically, you can use the --no-deps flag:

pip install <package-name> --no-deps

7.3.2. Upgrading a package

To upgrade a package that is already installed, you can use the --upgrade flag:

pip install <package-name> --upgrade

7.3.4. Keeping track of packages

Remember the command

pip list

which lists all the packages installed in the current environment.

7.4. Uploading your package to PyPi

Create a PyPi account if you don’t have one already.

Uploading your package to PyPi is simple. Follow these instructions.

The essential steps are:

build your package with python -m build. This creates the dist folder with source and wheel distributions.
upload the package to PyPi with twine upload dist/*

That’s it.

Note that local version segments are not allowed. If you have used setuptools_scm for versioning, you will need to remove the local version segment before uploading. A systematic way to do this is to add the following two lines to your pyproject.toml file:

[tool.setuptools_scm]
version_scheme = "post-release"
local_scheme = "no-local-version"

If the project name is already taken you will get an error and need to modify it.

When successful, you will see something like:

Uploading distributions to https://upload.pypi.org/legacy/
Uploading company_package-0.0.0b0.post12-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.5/19.5 kB • 00:00 • 14.9 MB/s
Uploading company_package-0.0.0b0.post12.tar.gz
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 kB • 00:00 • 5.0 MB/s

View at:
https://pypi.org/project/company-package/0.0.0b0.post12/

and the package becomes available on PyPi.

You can then test the installation by running

pip install company_package

from within a fresh virtual environment or on another machine.

7.5. Versioning

Versioning for python packages is handled automatically. We do it with setuptools_scm and the version is automatically written to <package_folder>/version.py.

Relevant configuration in the pyproject.toml file is:

[tool.setuptools_scm]
version_scheme = "post-release"
local_scheme = "no-local-version"
write_to = "<package_folder>/version.py"  # Where to write the dynamic version

With this configuration, when you make changes and build the package, the version is automatically updated and increments as 0.0.0.post1, 0.0.0.post2, etc. for each build that has changed compared to the previous one.

To increment other parts of the version, you tag with a version number using git.

For example,

git tag v0.0.1

will update the version to 0.0.1 (you dont need to worry about the post part).

To push the tags to the remote repository, you run

git push --tags

and you can check they are there on the GitHub, like at this page.

Exercise: Upload the pygmb package you created in the example class to PyPi. You can deleted afterward if you want. To delete the project from PyPi, navigate to the PyPi page of your project, go to Settings, scroll down and click delete.

7. Python Package Index

7.1. PyPi

7.2. Installing a package from PyPi

7.3. Installing multiple packages

7.3.1. Package dependencies

7.3.2. Upgrading a package

7.3.3. Sharing your Python environment

7.3.4. Keeping track of packages

7.4. Uploading your package to PyPi

7.5. Versioning