Linting¶

In order to ensure high-quality packages, we now perform routine checks on each recipe (called linting).

Linting is executed as a GitHub Check on all PRs and as a guarding stage on all builds. It can also be executed locally using the bioconda-utils lint commeand.

Skipping a lint check¶

The linter may occasionally create false positives. Lint checks can therefore be disabled individually if there is good reason to do so.

Skipping persistently on a per-recipe basis¶

Recipes may contain a section listing lint checks to be disabled:

extra:
  skip-lints:
    - uses_setuptools  # uses pkg_resoures during run time

For example, uses_setuptools will trigger if a recipe requires setuptools in its run section. While the vast majority of tools only need setuptools during installation, there are some exceptions to this rule. A tool accessing the pkg_resources module which is part of setuptools actually does need setuptools present during run time. In this case, add skip as shown above and add a comment describing why the recipe needs to ignore that particular lint check.

Skipping on a per-commit basis¶

While only recommended in very special cases, it is possible to skip specific linting tests on a commit by using special text in the commit message, [lint skip $FUNCTION for $RECIPE] where $FUNCTION is the name of the function to skip and $RECIPE is the path to the recipe directory for which this test should be skipped.

For example, if the linter reports a uses_setuptools issue for recipes/mypackage, but you are certain the package really needs setuptools, you can add [lint skip uses_setuptools for recipes/mypackage] to the commit message and this linting test will be skipped on CircleCI. Multiple tests can be skipped by adding additional special text. For example, [lint skip uses_setuptools for recipes/pkg1] [lint skip in_other_channels for recipes/pkg2/0.3.5]. Note in the latter case that the second recipe has a subdirectory for an older version.

Technically we check for the regular expression \[\s*lint skip (?P<func>\w+) for (?P<recipe>.*?)\s*\] in the commit message of the HEAD commit. However, often we want to test changes locally without committing. When running locally for testing, you can add the same special text to a temporary environment variable LINT_SKIP. The same example above could be tested locally like this without having to make a commit:

LINT_SKIP="[lint skip uses_setuptools for recipes/mypackage]" circleci build

Lint Checks¶

Below, each lint check executed is shown and described. Lints are grouped into a few sections, depending on what type of issue they aim at preventing.

Incomplete Recipe ¶

These are basic checks, making sure that the recipe is not missing anything essential.

“`missing_home`”¶

The recipe is missing a homepage URL

Please add:

about:
   home: <URL to homepage>

We want to make sure users can get additional information about a package, and it saves a separate search for the tool. Furthermore some tools with name collisions have to be renamed to fit into the conda channel and the homepage is an unambiguous original source.

“`missing_summary`”¶

The recipe is missing a summary

Please add:

about:
  summary: One line briefly describing package

This is the “title” for the package. It will also be shown in package listings. Keep it brief and informative. One line does not mean one line on a landscape poster.

“`missing_license`”¶

The recipe is missing the about/license key

Please add:

about:
   license: <name of license>

We need to ensure that adding the package to bioconda does not violate the license. No license means we have no permission to distribute the package, so we need to have one.

If the license is not a standard FOSS license, please read it to make sure that we are actually permitted to distribute the software.

“`missing_tests`”¶

The recipe is missing tests

Please add:

test:
    commands:
       - some_command

and/or:

test:
    imports:
       - some_module

and/or any file named run_test.py`, ``run_test.sh or run_test.pl executing tests.

In order to provide a collection of actually working tools, some basic testing is required. The tests should make sure the tool was built (if applicable) and installed correctly, and guard against things breaking with e.g. a version update.

The tests can include commands to be run or modules to import. You can also use a script (run_test.sh, run_test.py or run_test.pl) to execute tests (which must exit with exit status 0).

See Tests for more information.

“`missing_hash`”¶

The recipe is missing a checksum for a source file

Please add:

source:
  sha256: checksum-value

The hash or checksum ensures the integrity of the downloaded source archive. It guards both against broken or incomplete downloads and against the source file’s content changing unexpectedly.

While conda allows md5, sha1 in addition to sha256, we prefer the latter.

See Hashes for more info.

“`missing_version_or_name`”¶

The recipe is missing name and/or version

Please make sure the recipe has at least:

package:
  name: package_name
  version: 0.12.34

For obvious reasons, each package must at least have a name and a version.

“`empty_meta_yaml`”¶

The recipe has an empty meta.yaml!?

Please check if you forgot to commit its contents.

Intentionally left blank. Or not?

“`missing_meta_yaml`”¶

The recipe is missing a meta.yaml file

Most commonly, this is because the file was accidentally named meta.yml. If so, rename it to meta.yaml.

“`missing_build`”¶

The recipe is missing a build section

Please add:

build:
  number: 0

“`missing_build_number`”¶

The recipe is missing a build number

Please add:

build:
    number: 0

The build number should be 0 for any new version and incremented with each revised build published.

Noarch or not noarch ¶

Not all packages are platform specific. Pure Python packages for example will usually work on any platform without modification. For this reason, we have a third “subdir” (in conda lingo) in addition to linux-64 and osx-64 called noarch. Packages marked with noarch: True or noarch: python will be built only once and can then be used on both supported target platforms.

There are some conda idiosyncracies to be aware of when marking a package as noarch:

A noarch package cannot use skip. Packages that are noarch but require specific Python versions should use pinning (e.g. - python >3).
Packages that are not noarch must use skip and must not pin Python (it will simply not work as expected).

“`should_be_noarch_python`”¶

The recipe should be build as noarch

Please add:

build:
  noarch: python

Python packages that don’t require a compiler to build are normally architecture independent and go into the noarch subset of packages.

Python packages that do not include compiled modules (e.g. Cython build modules) are architecture independent and only need to be built (packaged) once, saving time and space.

“`should_be_noarch_generic`”¶

The recipe should be build as noarch

Please add:

build:
  noarch: generic

Packages that don’t require a compiler to build are normally architecture independent and go into the noarch subset of packages.

Packages that contain no platform specific binaries, e.g. Java packages, and are not Python packages, should be marked as noarch: generic. This saves build time and space on our hosters.

“`should_not_be_noarch_compiler`”¶

The recipe uses a compiler but is marked noarch

Recipes using a compiler should not be marked noarch.

Please remove the build: noarch: section.

Packages containing platform specific binaries should not be marked noarch. Use of a compiler was detected, which generally indicates that platform specific binaries were built.

“`should_not_be_noarch_source`”¶

The recipe uses per platform sources and cannot be noarch

You are downloading different upstream sources for each platform. Remove the noarch section or use just one source for all platforms.

Packages downloading different sources for each platform cannot be marked noarch.

“`should_not_be_noarch_skip`”¶

The recipe uses skip: True but is marked noarch

Recipes marked as noarch cannot use skip.

“`should_not_use_skip_python`”¶

The recipe should be noarch and not use python based skipping

Please use:

requirements:
   build:
     - python >3  # or <3
   run:
     - python >3  # or <3

The build: skip: True feature only works as expected for packages built specifically for each “platform” (i.e. Python version and OS). This package should be noarch and not use skips.

The skip mechanism works by not creating packages for some of our target platforms and interpreters (= Python versions). It therefore does not work in conjunction with noarch.

If the recipe has a skip only for specific Python versions (e.g. skip: True # [py2k]), use pinning instead:

requirements:
  run:
    python >3

Policy ¶

These checks enforce some of our “policy” decisions for keeping Bioconda and it’s recipe repository consistent and clean.

“`uses_vcs_url`”¶

The recipe downloads source from a VCS

Please build from source archives and don’t use the git_url, svn_url or hg_url feature of conda.

While conda technically supports downloading sources directly from a versioning system, we strongly discourage doing so.

There are a number of reasons for this:

Making a release expresses an author’s intent that the software is at a stable point suitable for distribution. Distributing a specific, unreleased revision makes it unnecessarily difficult for upstream authors to help with bugs users might encounter.
For reproducibility, we keep a backup of all source files used to build packages on Bioconda, but cannot (currently) do so for git/svn/hg repositories.
Git uses checksums (hashes) as revision labels. These have no implicit order, and would require assigning a pseudo version number to the package to allow knowing whether another release is newer or older than a git revision based one.

With the exception of old, orphaned projects, upstream authors will usually be happy to create a release if asked kindly. Most hosting sites allow “tagging” a release via their web interface, making the process simple.

“`folder_and_package_name_must_match`”¶

The recipe folder and package name do not match

For clarity, the name of the folder the meta.yaml resides, in and the name of the toplevel package should match.

It’s just way simpler to find the “recipe” building a “package” if you can expect that the recipe building “samtools” is found in the “samtools” folder (and not hiding in “new_release” or “bamtools”).

If you are using outputs to split a single upstream distribution into multiple packages, try to make sure each output package name contains the name of the tool you are packaging

“`gpl_requires_license_distributed`”¶

The recipe packages GPL software but is missing copy of license

The GPL requires that a copy of the license accompany all distributions of the software. Please add:

about:
    license_file: name_of_license_file

If the upstream tar ball does not include a copy, please ask the authors of the software to add it to their distribution archive.

While many upstream authors are not aware of this when they grant license to use and distribute their work under the GPL, the GPL says that we must include a copy of the GPL with every package we distribute. It can be annoying and feel redundant, but it simply is what we must do to be permitted to distribute the software.

“`should_not_use_fn`”¶

The recipe uses source/fn

There is no need to specify the filename as the URL should give a name and it will in most cases be unpacked automatically.

The fn is really only needed if you have multiple url s that share a filename, which is a somewhat constructed scenario.

“`has_windows_bat_file`”¶

The recipe directory contains a bat file

Bioconda does not currently build packages for Windows (and has at this time no plans to change this), so these files cannot be tested.

Please remove any *.bat files generated by conda skeleton from the recipe directory.

The skeleton commands (e.g. conda skeleton pypi) create this file automatically, but we do not build for windows, making this file merely clutter needlessly increasing the size of our git repository.

“`long_summary`”¶

The summary line is rather long

Consider using the description field for longer text:

about:
  summary: Fancy Read Simulator (makes drinks)
  description: |
    XYZ is a very fancy read simulator that will not just make coffee
    while you are waiting but prepare all kinds of exquisite caffeeinated
    beverages from freshly roasted, single source beans ground to match
    ambient humidity.

This will fit better into the templates listing and describing recipes, which assume the summary to be a title and the description to be one or more paragraphs.

Recipes have a summary and description field. The recipe/package description pages at anaconda.org/bioconda and here are designed to use the summary as a title line and provide a separate section for multi-paragraph descriptions filled with content from the description field. The summary is also used for package listings with only one row per package.

It just looks better if the summary fits into one line.

“`cran_packages_to_conda_forge`”¶

CRAN packages not depending on Bioconda should go to Conda-Forge

This recipe builds a CRAN package and does not depend on packages from Bioconda. It should therefore be moved to Conda-Forge.

Conda-Forge has a very active R community planning to eventually package all of CRAN. For that reason, we only allow CRAN packages on Bioconda if they depend on other Bioconda packages.

“`version_starts_with_v`”¶

The version string should not start with a “v” character

Version numbers in Conda recipes need to follow PEP 386

Version numbers in Conda recipes need to follow PEP 386 and may not start with a “v”. With a “v”, the uploaded package displays incorrectly on the Anaconda website.

Syntax ¶

These checks ensure that the extra section conforms to our “schema”.

“`extra_identifiers_not_list`”¶

The extra/identifiers section must be a list

Example:

extra:
   identifiers:
      - doi:123

“`extra_identifiers_not_string`”¶

Each item in the extra/identifiers section must be a string

Example:

extra:
   identifiers:
      - doi:123

Note that there is no space around the colon

“`extra_identifiers_missing_colon`”¶

Each item in the extra/identifiers section must be of form type:value

Example:

extra:
   identifiers:
      - doi:123

Ensure that the section is of the following format:

extra:
  identifiers:
    - doi:10.1093/bioinformatics/bts480
    - biotools:Snakemake

In particular, ensure that each identifier starts with a type (doi, biotools, …), followed by a colon and the identifier. Whitespace is not allowed.

“`extra_skip_lints_not_list`”¶

The extra/skip-lints section must contain a list

Example:

extra:
   skip-lints:
      - should_use_compilers

The recipe is trying to skip a lint with an unknown name. Check the list here for the correct name.

“`version_constraints_missing_whitespace`”¶

Packages and their version constraints must be space separated

Example:

host:
    python >=3

Version constraints in the meta.yaml must have spaces between the package name and the constraint.

Recipe Parsing ¶

These lints happen implicitly during recipe parsing. They cannot be skipped!

“`duplicate_key_in_meta_yaml`”¶

The recipe meta.yaml contains a duplicate key

This is invalid YAML, as it’s unclear what the structure should become. Please merge the two offending sections

Say you have two requirements: build: sections, should the second replace the list in the first, or should it append to it? The YAML standard does not specify this. Instead, it just says that sections cannot occur twice. Some YAML parsers allow duplicate keys, but it’s often unclear what the result should be, and it’s easy to miss another section further down in the recipe, so we don’t.

“`unknown_selector`”¶

The recipe failed to parse due to selector lines

Please request help from @bioconda/core.

The recipe uses an # [abc] selector that is not understood by bioconda-utils. If you actually do encounter this, ping @bioconda/core at it is very likely a bug.

“`conda_render_failure`”¶

The recipe was not understood by conda-build

Please request help from @bioconda/core.

There was an error in conda-build “rendering” the recipe. Please contact @bioconda/core for help.

“`jinja_render_failure`”¶

The recipe could not be rendered by Jinja2

Check if you are missing quotes or curly braces in Jinja2 template expressions. (The parts with {{ something }} or {% set var="value" %}).

Conda recipes are technically not YAML, but Jinja YAML templates. The Jinja template engine turning the recipe text into YAML complained about a part in your recipe.

Most frequently, this is due to unbalanced or missing braces, parentheses or quotes.

“`unknown_check`”¶

Something went wrong inside the linter

Please request help from @bioconda/core

You broke it!!! Congratulations, you found a bug in the linter. Ping @bioconda/core to figure out what’s going on.

Repository ¶

“`in_other_channels`”¶

A package of the same name already exists in another channel

Bioconda and Conda-Forge occupy the same name space and have agreed not to add packages if a package of the same name is already present in the respective other channel.

If this is a new package, pease choose a different name (e.g. append -bio).

If you are updating a package, please continue in the package’s new home at conda-forge.

The package exists in another dependent channel (currently conda-forge and defaults). This often happens when a general-use package was added to bioconda first but was subsequently added to one of the more general channels. In this case we’d prefer it to be in the general channel.

We want to minimize duplicated work. If a package already exists in another dependent channel, it doesn’t need to be maintained in the bioconda channel.

In special cases this can be overridden, for example if a bioconda-specific patch is required. However it is almost always better to fix or update the recipe in the other channel. Note that the package in the bioconda channel will remain in order to maintain reproducibility.

Sometimes when adding or updating a recipe in a pull request to conda-forge the conda-forge linter will warn that a recipe with the same name already exists in bioconda. When this happens, usually the best thing to do is:

Submit – but don’t merge yet! – a PR to bioconda that removes the recipe. In that PR, reference the conda-forge/staged-recipes PR.
Merge the conda-forge PR adding or updating the recipe
Merge the bioconda PR deleting the recipe

“`build_number_needs_bump`”¶

The recipe build number should be incremented

A package with the same name and version and a build number at least as high as specified in the recipe already exists in the channel. Please increase the build number.

Every time you change a recipe, you should assign a new (incremented) build number. Otherwise, the package may not be built (it will be built only if the “build string” is different, which might happen only on one architecture and not the other).

“`build_number_needs_reset`”¶

The recipe build number should be reset to 0

No previous build of a package of this name and this version exists, the build number should therefore be 0.

If no previous build exists for a package/version combination, the build number should be 0.

“`recipe_is_blacklisted`”¶

The recipe is currently blacklisted and will not be built

If you are intending to repair this recipe, remove it from the build fail blacklist.

We maintain a list of packages that are “blacklisted”. Recipes are added to this list if they fail to rebuild and would block automatic build processes. Just remove your package from this list and proceed as normal. Hopefully, you can fix whatever got the recipe on the blacklist!

Deprecations ¶

These checks catch packages or recipe idioms we no longer use or that no longer work and need to be changed.

“`uses_perl_threaded`”¶

The recipe uses perl-threaded

Please use perl instead.

Previously, Bioconda used perl-threaded as a dependency for Perl packages, but now we are using perl instead. When one of these older recipes is updated, it will fail this check.

“`uses_javajdk`”¶

The recipe uses java-jdk

Please use openjdk instead.

Previously, Bioconda used java-jdk as a dependency for Java packages, but now we are using openjdk instead. When one of those older recipes is updated, it will fail this check.

“`deprecated_numpy_spec`”¶

The recipe contains a deprecated numpy spec

Please remove the x.x - pinning is now handled automatically.

Originally, conda used the numpy x.x syntax to enable pinning for numpy. This way of pinning has been superceded by the conda_build_config.yaml way of automatic pinning. You can now just write numpy (without any special string appended).

“`uses_matplotlib`”¶

The recipe uses matplotlib, but matplotlib-base is recommended

The matplotlib dependency should be replaced with matplotlib-base unless the package explicitly needs the PyQt interactive plotting backend.

The matplotlib package has been split into matplotlib and matplotlib-base. The only difference is that matplotlib contains an additional dependency on pyqt, which pulls in many other dependencies. In most cases, using matplotlib-base is sufficient.

Build helpers ¶

“`missing_run_exports`”¶

Recipe should have a run_exports statement that ensures correct pinning in downstream packages

This ensures that the package is automatically pinned to a compatible version if it is used as a dependency in another recipe. This is a conservative strategy to avoid breakage. We came to the conclusion that it is better to require this little overhead instead of trying to fix things when they break later on. This holds for compiled packages (in particular those with shared libraries) but also for e.g. Python packages, as those might also introduce breaking changes in their APIs or command line interfaces.

We distinguish between four cases.

Case 1: If the software follows semantic versioning (or it has at least a normal version string (like 1.2.3) and the actual strategy of the devs is unknown), add run_exports to the recipe like this:

build:
  run_exports:
    - {{ pin_subpackage('myrecipe', max_pin="x") }}

with myrecipe being the name of the recipe (you can also use the name variable). This will by default pin the package to >=1.2.0,<2.0.0 where 1.2.0 is the version of the package at build time of the one depending on it and <2.0.0 constrains it to be less than the next major (i.e. potentially not backward compatible) version.

Case 2: If the software version starts with a 0 (e.g. 0.3.2) semantic versioning allows breaking changes in minor releases. Hence, you should use:

build:
  run_exports:
    - {{ pin_subpackage('myrecipe', max_pin="x.x") }}

Case 3: If the software has a normal versioning (like 1.2.3) but does reportedly not follow semantic versioning, please choose the max_pin argument such that it captures the potential next version that will introduce a breaking change. E.g. if you expect breaking changes to occur with the next minor release, choose max_pin="x.x", if they even can occur with the next patch release, choose max_pin="x.x.x".

Case 4: If the software does have a non-standard versioning (e.g. calendar versioning like 20220602), we cannot really protect well against breakages. However, we can at least pin to the current version as a minimum and skip the max_pin constraint. This works by setting max_pin=None.

In the recipe depending on this one, one just needs to specify the package name and no version at all.

Also check out the possible arguments of pin_subpackage here: https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#export-runtime-requirements

Since this strategy can lead to potentially more conflicts in dependency pinnings between tools, it is advisable to additionally set a common version of very frequently used packages for all builds. This happens by specifying the version via a separate pull request in the project wide build configuration file here: https://github.com/bioconda/bioconda-utils/blob/master/bioconda_utils/bioconda_utils-conda_build_config.yaml

Finally, note that conda is unable to conduct such pinnings in case the dependency and the depending recipe are updated within the same pull request. Hence, the pull request adding the run_exports statement has to be merged before the one updating or creating the depending recipe is created.

Importantly note that this shall not be used to pin compatible versions of other recipes in the current recipe. Rather, those other recipes have to get their own run_exports sections. Usually, there is no need to use pin_compatible, just use pin_subpackage as shown above, and fix run_exports in upstream packages as well if needed.

“`should_use_compilers`”¶

The recipe requires a compiler directly

Since version 3, conda-build uses a special syntax to require compilers for a given language matching the architecture for which a package is being build. Please use:

requirements:
   build:
     - {{ compiler('language') }}

Where language is one of c, cxx, fortran, rust, go or cgo. You can specify multiple compilers if needed.

There is no need to add libgfortran, libgcc, or toolchain to the dependencies as this will be handled by conda-build itself.

The recipe uses a compiler directly. Since version 3, conda-build has a special syntax for compilers, e.g.:

build:
  - {{ compiler('cxx') }}

This will select the appropriate C++ compiler (clang on MacOS and g++ on Linux) automatically, inject apropriate environment variables (CXX, CXXFLAGS, LDFLAGS, …) into the build environment and create the right dependencies (e.g. libgcc). Which compilers are used is configured via conda_build_config.yaml, which we “inherit” from conda-forge.

Packages no longer needed are toolchain, libgfortran, libgcc. The compilers gcc, llvm, go, and cgo don’t need to be installed directly, instead specify c, cxx, fortran, go or cgo as language using the compiler syntax.

One exception from this is llvm-openmp # [osx] which still needs to be added manually if your package makes use of OpenMP.

See Compiler tools for details.

“`uses_setuptools`”¶

The recipe uses setuptools in run depends

Most Python packages only need setuptools during installation. Check if the package really needs setuptools (e.g. because it uses pkg_resources or setuptools console scripts).

setuptools is typically used to install dependencies for Python packages but most of the time this is not needed within a conda package as a run dependency.

Some packages do need setuptools, in which case this check can be overridden. setuptools may be required, e.g., if a package loads resources via pkg_resources which is part of setuptools. That dependency can also be introduced implicitly when setuptools-created console scripts are used. To avoid this, make sure to carry console_scripts entry points from setup.py over to meta.yaml to replace them with scripts created by conda/conda-build which don’t require pkg_resources. Recipes generated via conda skeleton pypi already include the required section.

“`setup_py_install_args`”¶

The recipe uses setuptools without required arguments

Please use:

$PYTHON setup.py install --single-version-externally-managed --record=record.txt

The parameters are required to avoid setuptools trying (and failing) to install certifi when a package this recipe requires defines entrypoints in its setup.py.

When a package depends on setuptools, we have to disable some parts of setuptools during installation to make it work correctly with conda. In particular, it seems that packages depend on other packages that specify entry points (e.g., pyfaidx) will cause errors about how setuptools is not allowed to install certifi in a conda package.

Change the line in either in build.sh or the build:script key in meta.yaml from:

$PYTHON setup.py install

to:

$PYTHON setup.py install --single-version-externally-managed --record=record.txt

“`compilers_must_be_in_build`”¶

The recipe requests a compiler in a section other than build

Please move the {{ compiler('language') }} line into the requirements: build: section.

A {{ compiler('xyz') }} variable was found, but not in the build: section. Move {{ compiler() }} variables to the build: section.

“`cython_must_be_in_host`”¶

Cython should be in the host section

Move cython to host:

requirements:
  host:
    - cython

Cython should match the Python version, so we keep it in the host section for now.

“`cython_needs_compiler`”¶

Cython generates C code, which will need to be compiled

Add the compiler to the recipe:

requirements:
  build:
    - {{ compiler('c') }}

Cython is compiled into C code, which then needs to be compiled. You almost certainly want to have a C compiler in your recipe.

Linter Errors ¶

“`linter_failure`”¶

An unexpected exception was raised during linting

Please file an issue at the bioconda-utils repo

Developer docs¶

See bioconda_utils.lint for information on how to write additional checks.