Updating bioconda-utils#

This section documents the steps required to update bioconda-utils and get the updated version working over on bioconda-recipes, working on CI platforms and building packages.

Developing from source#

Development dependency management is defined by pixi.toml. From a bioconda-utils checkout, use the repository’s Pixi-backed Just tasks:

pixi install
just install
just check

Do not install development dependencies into the checkout with pip. The just install task builds the project inside the Pixi environment and makes that CLI available from ~/.local/bin.

bioconda-utils is currently using Release Please to manage updates, changelogs, and versioning. This works in a specific way, so the steps below walk through the process if you’re not already familiar.

Prefix the PR title, as well as at least one commit in the PR, with one of the following change types. Some special types will change the bioconda-utils version number, as noted below.

  • <type>! (that is, any of the types below ending an exclamation point) indicates a breaking change. PRs with this title will result in a new MAJOR VERSION

  • feat: a new feature. PRs with this title will result in a new MINOR VERSION

  • fix: fixes a bug. PRs with this title will result in a new PATCH VERSION

  • test: changes related to tests

  • chore: for maintenance changes. E.g. dependency version changes (should use a ! to indicate breaking change in this case)

  • ci: for changes related to CI of bioconda-utils

  • docs: a change that only affects documentation (ReST, comments, docstrings)

  • refactor: a change in code that neither fixes a bug nor adds a feature

  • style: whitespace, formatting, etc

The general workflow is:

  1. Open a pull request to bioconda-utils repo, being sure to use conventional commit messages in your title.

    Why do I need to pay attention to the PR title?

    Why do I need to pay attention to the PR title?

    This is part of Release Please, which uses the PR titles (as well as individual commits within the PR) to decide on semantic version bumps. The bioconda-utils repo has a GitHub Action that will check for conventional commit message in the PR title, and the PR will fail without a properly-formatted title.

    Note that if you update the PR title to address a failing check, you may need to push an additional commit to trigger the check again (or possibly close and then reopen the PR).

  2. Merge to master branch

    Won't the changes be immediately used?

    Won't the changes be immediately used?

    Our infrastructure no longer points directly to the master branch of bioconda-utils. Instead, the infrastructure points to specific releases. Merging to master branch does not create a release – see below for how releases are created.

    Release Please monitors the master branch to determine what to add to the special release PR.

  3. Allow Release Please to automatically create a release PR

    What's a release PR?

    What's a release PR?

    A release PR is a special PR automatically create by the Release Please GitHub Action running in the bioconda-utils repo. The release PR will keep track of accumulated changes since the last release. The version in the title of the PR will reflect semantic versioning to use based on the accumulated changes. Here’s an example. Merging the special release PR will create a release.

  4. Merge the release PR to automatically create a new GitHub release of bioconda-utils

    I'm done, right?

    I'm done, right?

    Not done yet…our infrastructure uses the conda package of bioconda-utils, which is in turn hosted on bioconda-recipes. So simply creating a new GitHub release of bioconda-utils is insufficient to use it on our infrastructure. We still need to build the conda package, which happens over on bioconda-recipes.

  5. Allow the autobump bot to detect the new release and create a new PR over on bioconda-recipes to create an updated conda package.

    How are dependencies kept consistent?

    How are dependencies kept consistent?

    pixi.toml is the source of truth for runtime and development dependencies. The shipped bioconda_utils/bioconda_utils-requirements.txt contains only the conda runtime dependencies needed by Dockerfiles and runtime build helpers; it is generated from pixi.toml and must not be edited directly.

    After changing runtime dependencies, run:

    pixi run regenerate-requirements
pixi run check-requirements

    just check also runs the generated-file check. When the release is packaged in bioconda-recipes, review its requirements: run section against the generated runtime requirements and update the recipe where necessary.

  6. Once tests pass, treat it as a typical package: get approval, and then merge.

    What version is used to build the package?

    What version is used to build the package?

    That new conda package is built using the previous version of bioconda-utils since that’s what’s running on our infrastructure. Merge into bioconda-recipes when tests pass.

  7. Open a PR on bioconda-recipes bumping the utils-ref input default in .github/actions/setup-bioconda-utils/action.yml to the new tag (e.g. v4.5.0). If the release changes the CLI surface used by the workflows, include those workflow changes in the same PR – the pin and its call sites now land atomically, which is the point of the action. To trial an unreleased utils revision, a recipes PR can pass it explicitly via the same input instead of moving the default.

    Where is that pin file used?

    Where is that pin file used?

    The setup-bioconda-utils composite action (.github/actions/setup-bioconda-utils) checks out bioconda-utils at the pinned ref and provisions its pixi environment. Every GitHub Actions workflow (PR checks, master uploads, bulk, nightly, build-failure reports, mulled manifests) goes through that action, so bumping the file rolls the new version out everywhere. The bulk branch carries its own value of the file, which is how bulk runs a different version.

At this point, the next workflow run provisions the new version of bioconda-utils via pixi (setup-pixi caches the environment keyed on the utils pixi.lock). bioconda-recipes is now using the updated version..

How do I check?

How do I check?

You can keep an eye on new bioconda-recipe PRs, or maybe close and then reopen an existing one. Look in the PR check logs for the “Set up pixi” step and the installed bioconda-utils version to ensure it matches the pin.