CI Inventory

Changed on 2023-05-02: Use mergify instead of AutoMerge.yml GitHub Action

Changed on 2023-06-06: Start using build-failures workflow

Changed on 2024-03-04: Start using linux-aarch64 in bulk and recipe tests

Changed on 2024-07-05: Start using osx-arm64 in bulk and recipe tests

This page documents the various moving parts that, together, make Bioconda work. We rely on a mixture of free services to spread the workload and to maintain flexibility over the long term in case a service becomes unusable.

Note

For the platform column, we use the conda designations when conda packages are built. Otherwise we use operating system. See Understanding platform nomenclature for more info on this.

Name

Runs on

Controlled from

Trigger

Code references

Platform

Description

Recipe tests

Azure Pipelines, CircleCI

bioconda-recipes

on push

azure-pipeline.yml (linux-64, osx-64); config.yml (linux-aarch64, osx-arm64)

linux-64, osx-64, linux-aarch64, osx-arm64

These are the most-run tests: these are what run on every change on pull requests to bioconda-recipes, and they must pass before the recipe is merged into the master branch.

Comment responder

GitHub Actions

bioconda-recipes

@BiocondaBot mentions

CommentResponder.yml

Linux

Runs the bioconda-bot container (quay.io/bioconda/bot) with different image tags (merge, comment, update, repost) in response to comments. This allows fast response time (rather than, say, restoring a cache each time).

Master branch tests

Azure Pipelines, CircleCI

bioconda-recipes

push to master (bioconda-recipes)

azure-pipeline-master.yml (linux-64, osx-64); config.yml (linux-aarch64, osx-arm64)

linux-64, osx-64, linux-aarch64, osx-arm64

Runs when a PR is merged to the master branch. The already-built artifacts are retrieved from the PR and uploaded to the bioconda channel. Container(s) are uploaded to quay.io/biocontainers. If artifacts are not found, it will build the recipe(s) as a fallback.

Mergify

mergify

bioconda-recipes

Successful tests on an autobumped recipe

.mergify.yml

Linux

If a recipe was autobumped, the tests passed, and it’s been more than 3 days, then automatically merge the updated recipe.

Autobump bot

CircleCI

bioconda-utils

hourly

config.yml

Linux

Checks upstream repository for version updates, if so, creates a new bioconda-recipes recipe with the updated version and an updated hash, and opens a new pull request with various templated info and with the “autobump” label applied.

Build failures

GitHub Actions

bioconda-recipes

daily

build-failures.yaml

Linux

Runs bioconda-utils list-build-failures to generate and publish a wiki page listing prioritized failures.

Docs

GitHub Actions

bioconda-docs

daily, or on push

docs.yml

Linux

Build sphinx documentation (including updated READMEs for every recipe) and pushes the changes to [bioconda.github.io](https://github.com/bioconda/bioconda.github.io).

bioconda-utils tests

GitHub Actions

bioconda-utils

on push

GithubActionTests.yml

linux-64, osx-64

Unit tests and functional tests for bioconda-utils.

Docker image test

GitHub Actions

bioconda-utils

on push

build-image.yml, Dockerfile

Linux

Ensures that bioconda-utils can run inside a newly-built container.

Check public containers

GitHub Actions

bioconda-utils

manually

changevisibility.yml

Linux

Checks quay.io to see if any containers are mistakenly private; if so makes them public

Conventional PRs

GitHub Actions

bioconda-utils

bioconda-utils PR open/reopen/edit/sync

conventional-prs.yml

Linux

Enforces “conventional commit” tags in the title of PRs, like “docs:”, “fix:”, “ci:”, and so on in bioconda-utils. This enables better automation of releases.

Release-please

GitHub Actions

bioconda-utils

push to master on bioconda-utils

release-please.yml

Linux

Collects PRs that have been merged to master since the last release into a separate, special PR. Merging that special PR is what triggers a new release.

Nightly maintenance

Azure Pipelines

bioconda-recipes

daily

azure-pipeline-nightly.yml

Linux, macOS

Various maintenance tasks: build and upload the bioconda-repodata-patches package; try to build and upload any remaining packages (runs bioconda-utils build on all recipes)

Bulk

GitHub Actions

bioconda-recipes

push to bulk branch of bioconda-utils

Bulk.yml (linux-64, osx-64, osx-arm64); config.yml (linux-aarch64)

linux-64, osx-64, linux-aarch64, osx-arm64

If pushing to the special bulk branch, this workflow will run. It uses special bioconda-utils functionality to split the full DAG into sub-DAGs and submits them to independent parallel jobs. When recipes sucessfully build, they are immediately uploaded. Use with caution. Typically used when migrating (e.g., bioconductor updates, pinning updates)

Compile stats for plots

GitHub Actions

bioconda-stats

several times a day

packages-anaconda-org.yml

Linux

Builds a zip file containing TSVs for each package, which are inspected and compiled into json files in bioconda-plots.

Generate plots

GitHub Actions

bioconda-plots

daily

generate-plots.yml; test-plots.yml runs minor tests on the plot-generating code.

Linux

Using stats compiled in bioconda-stats, builds json files for each package that are used by plotting code on bioconda.github.io recipe pages.