Publishing package distribution releases using GitHub Actions CI/CD workflows

GitHub Actions CI/CD allows you to run a series of commands whenever an event occurs on the GitHub platform. One popular choice is having a workflow that’s triggered by a push event. This guide shows you how to publish a Python distribution whenever a tagged commit is pushed. It will use the pypa/gh-action-pypi-publish GitHub Action for publishing. It also uses GitHub’s upload-artifact and download-artifact actions for temporarily storing and downloading the source packages.

Увага

This guide assumes that you already have a project that you know how to build distributions for and it lives on GitHub. This guide also avoids details of building platform specific projects. If you have binary components, check out cibuildwheel’s GitHub Action examples.

Configuring trusted publishing

This guide relies on PyPI’s trusted publishing implementation to connect to GitHub Actions CI/CD. This is recommended for security reasons, since the generated tokens are created for each of your projects individually and expire automatically. Otherwise, you’ll need to generate an API token for both PyPI and TestPyPI. In case of publishing to third-party indexes like devpi, you may need to provide a username/password combination.

Since this guide will demonstrate uploading to both PyPI and TestPyPI, we’ll need two trusted publishers configured. The following steps will lead you through creating the «pending» publishers for your new PyPI project. However it is also possible to add trusted publishing to any pre-existing project, if you are its owner.

Увага

If you followed earlier versions of this guide, you have created the secrets PYPI_API_TOKEN and TEST_PYPI_API_TOKEN for direct PyPI and TestPyPI access. These are obsolete now and you should remove them from your GitHub repository and revoke them in your PyPI and TestPyPI account settings in case you are replacing your old setup with the new one.

Let’s begin! 🚀

1. Go to https://pypi.org/manage/account/publishing/.

2. Fill in the name you wish to publish your new PyPI project under (the name value in your setup.cfg or pyproject.toml), the GitHub repository owner’s name (org or user), and repository name, and the name of the release workflow file under the .github/ folder, see Creating a workflow definition. Finally, add the name of the GitHub Environment (pypi) we’re going set up under your repository. Register the trusted publisher.

3. Now, go to https://test.pypi.org/manage/account/publishing/ and repeat the second step, but this time, enter testpypi as the name of the GitHub Environment.

4. Your «pending» publishers are now ready for their first use and will create your projects automatically once you use them for the first time.

   Примітка

   If you don’t have a TestPyPI account, you’ll need to create it. It’s not the same as a regular PyPI account.

   Увага

   For security reasons, you must require manual approval on each run for the pypi environment.

Creating a workflow definition

GitHub CI/CD workflows are declared in YAML files stored in the .github/workflows/ directory of your repository.

Let’s create a .github/workflows/publish-to-test-pypi.yml file.

Start it with a meaningful name and define the event that should make GitHub run this workflow:

name: Publish Python 🐍 distribution 📦 to PyPI and TestPyPI

on: push

Checking out the project and building distributions

We will have to define two jobs to publish to PyPI and TestPyPI respectively, and an additional job to build the distribution packages.

First, we’ll define the job for building the dist packages of your project and storing them for later use:

jobs:
  build:
    name: Build distribution 📦
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: "3.x"

This will download your repository into the CI runner and then install and activate the newest available Python 3 release.

And now we can build the dists from source and store them. In this example, we’ll use the build package. So add this to the steps list:

    - name: Install pypa/build
      run: >-
        python3 -m
        pip install
        build
        --user
    - name: Build a binary wheel and a source tarball
      run: python3 -m build
    - name: Store the distribution packages
      uses: actions/upload-artifact@v3
      with:
        name: python-package-distributions
        path: dist/

Defining a workflow job environment

Now, let’s add initial setup for our job that will publish to PyPI. It’s a process that will execute commands that we’ll define later. In this guide, we’ll use the latest stable Ubuntu LTS version provided by GitHub Actions. This also defines a GitHub Environment for the job to run in its context and a URL to be displayed in GitHub’s UI nicely. Additionally, it allows acquiring an OpenID Connect token that the pypi-publish actions needs to implement secretless trusted publishing to PyPI.

  publish-to-pypi:
    name: >-
      Publish Python 🐍 distribution 📦 to PyPI
    if: startsWith(github.ref, 'refs/tags/')  # only publish to PyPI on tag pushes
    needs:
    - build
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/<package-name>  # Replace <package-name> with your PyPI project name
    permissions:
      id-token: write  # IMPORTANT: mandatory for trusted publishing

This will also ensure that the PyPI publishing workflow is only triggered if the current commit is tagged.

Publishing the distribution to PyPI

Finally, add the following steps at the end:

    steps:
    - name: Download all the dists
      uses: actions/download-artifact@v3
      with:
        name: python-package-distributions
        path: dist/
    - name: Publish distribution 📦 to PyPI
      uses: pypa/gh-action-pypi-publish@release/v1

This step uses the pypa/gh-action-pypi-publish GitHub Action: after the stored distribution package has been downloaded by the download-artifact action, it uploads the contents of the dist/ folder into PyPI unconditionally.

Signing the distribution packages

The following job signs the distribution packages with Sigstore, the same artifact signing system used to sign CPython.

Firstly, it uses the sigstore/gh-action-sigstore-python GitHub Action to sign the distribution packages. In the next step, an empty GitHub Release from the current tag is created using the gh CLI. Note this step can be further customised. See the gh release documentation as a reference.

Порада

You may need to manage your GITHUB_TOKEN permissions to enable creating the GitHub Release. See the GitHub documentation for instructions. Specifically, the token needs the contents: write permission.

Finally, the signed distributions are uploaded to the GitHub Release.

  github-release:
    name: >-
      Sign the Python 🐍 distribution 📦 with Sigstore
      and upload them to GitHub Release
    needs:
    - publish-to-pypi
    runs-on: ubuntu-latest

    permissions:
      contents: write  # IMPORTANT: mandatory for making GitHub Releases
      id-token: write  # IMPORTANT: mandatory for sigstore

    steps:
    - name: Download all the dists
      uses: actions/download-artifact@v3
      with:
        name: python-package-distributions
        path: dist/
    - name: Sign the dists with Sigstore
      uses: sigstore/gh-action-sigstore-python@v1.2.3
      with:
        inputs: >-
          ./dist/*.tar.gz
          ./dist/*.whl
    - name: Create GitHub Release
      env:
        GITHUB_TOKEN: ${{ github.token }}
      run: >-
        gh release create
        '${{ github.ref_name }}'
        --repo '${{ github.repository }}'
        --notes ""
    - name: Upload artifact signatures to GitHub Release
      env:
        GITHUB_TOKEN: ${{ github.token }}
      # Upload to GitHub Release using the `gh` CLI.
      # `dist/` contains the built packages, and the
      # sigstore-produced signatures and certificates.
      run: >-
        gh release upload
        '${{ github.ref_name }}' dist/**
        --repo '${{ github.repository }}'

Примітка

This is a replacement for GPG signatures, for which support has been removed from PyPI. However, this job is not mandatory for uploading to PyPI and can be omitted.

Separate workflow for publishing to TestPyPI

Now, repeat these steps and create another job for publishing to the TestPyPI package index under the jobs section:

  publish-to-testpypi:
    name: Publish Python 🐍 distribution 📦 to TestPyPI
    needs:
    - build
    runs-on: ubuntu-latest

    environment:
      name: testpypi
      url: https://test.pypi.org/p/<package-name>

    permissions:
      id-token: write  # IMPORTANT: mandatory for trusted publishing

    steps:
    - name: Download all the dists
      uses: actions/download-artifact@v3
      with:
        name: python-package-distributions
        path: dist/
    - name: Publish distribution 📦 to TestPyPI
      uses: pypa/gh-action-pypi-publish@release/v1
      with:
        repository-url: https://test.pypi.org/legacy/

Порада

Requiring manual approvals in the testpypi GitHub Environment is typically unnecessary as it’s designed to run on each commit to the main branch and is often used to indicate a healthy release publishing pipeline.

The whole CI/CD workflow

This paragraph showcases the whole workflow after following the above guide.

Click here to display the entire GitHub Actions CI/CD workflow definition

name: Publish Python 🐍 distribution 📦 to PyPI and TestPyPI

on: push

jobs:
  build:
    name: Build distribution 📦
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: "3.x"
    - name: Install pypa/build
      run: >-
        python3 -m
        pip install
        build
        --user
    - name: Build a binary wheel and a source tarball
      run: python3 -m build
    - name: Store the distribution packages
      uses: actions/upload-artifact@v3
      with:
        name: python-package-distributions
        path: dist/

  publish-to-pypi:
    name: >-
      Publish Python 🐍 distribution 📦 to PyPI
    if: startsWith(github.ref, 'refs/tags/')  # only publish to PyPI on tag pushes
    needs:
    - build
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/<package-name>  # Replace <package-name> with your PyPI project name
    permissions:
      id-token: write  # IMPORTANT: mandatory for trusted publishing

    steps:
    - name: Download all the dists
      uses: actions/download-artifact@v3
      with:
        name: python-package-distributions
        path: dist/
    - name: Publish distribution 📦 to PyPI
      uses: pypa/gh-action-pypi-publish@release/v1

  github-release:
    name: >-
      Sign the Python 🐍 distribution 📦 with Sigstore
      and upload them to GitHub Release
    needs:
    - publish-to-pypi
    runs-on: ubuntu-latest

    permissions:
      contents: write  # IMPORTANT: mandatory for making GitHub Releases
      id-token: write  # IMPORTANT: mandatory for sigstore

    steps:
    - name: Download all the dists
      uses: actions/download-artifact@v3
      with:
        name: python-package-distributions
        path: dist/
    - name: Sign the dists with Sigstore
      uses: sigstore/gh-action-sigstore-python@v1.2.3
      with:
        inputs: >-
          ./dist/*.tar.gz
          ./dist/*.whl
    - name: Create GitHub Release
      env:
        GITHUB_TOKEN: ${{ github.token }}
      run: >-
        gh release create
        '${{ github.ref_name }}'
        --repo '${{ github.repository }}'
        --notes ""
    - name: Upload artifact signatures to GitHub Release
      env:
        GITHUB_TOKEN: ${{ github.token }}
      # Upload to GitHub Release using the `gh` CLI.
      # `dist/` contains the built packages, and the
      # sigstore-produced signatures and certificates.
      run: >-
        gh release upload
        '${{ github.ref_name }}' dist/**
        --repo '${{ github.repository }}'

  publish-to-testpypi:
    name: Publish Python 🐍 distribution 📦 to TestPyPI
    needs:
    - build
    runs-on: ubuntu-latest

    environment:
      name: testpypi
      url: https://test.pypi.org/p/<package-name>

    permissions:
      id-token: write  # IMPORTANT: mandatory for trusted publishing

    steps:
    - name: Download all the dists
      uses: actions/download-artifact@v3
      with:
        name: python-package-distributions
        path: dist/
    - name: Publish distribution 📦 to TestPyPI
      uses: pypa/gh-action-pypi-publish@release/v1
      with:
        repository-url: https://test.pypi.org/legacy/

That’s all, folks!

Now, whenever you push a tagged commit to your Git repository remote on GitHub, this workflow will publish it to PyPI. And it’ll publish any push to TestPyPI which is useful for providing test builds to your alpha users as well as making sure that your release pipeline remains healthy!

Увага

If your repository has frequent commit activity and every push is uploaded to TestPyPI as described, the project might exceed the PyPI project size limit. The limit could be increased, but a better solution may constitute to use a PyPI-compatible server like pypiserver in the CI for testing purposes.

Примітка

It is recommended to keep the integrated GitHub Actions at their latest versions, updating them frequently.