Recipe Linter

Writing additional checks

Lint checks are defined in bioconda_utils.lint.checks as subclasses of LintCheck. It might be easiest to have a look at that module and the already existing checks and go from there.

Briefly, each class becomes a check by:

• The class name becomes the name of the check in the documentation and when skipping lints. The convention is to use lower case separated by underscore.

• The docstring is used to describe the check on the command line, on Github when the check fails and in the documentation.

  The first line is considered a title or one line summary used where a brief description is needed. The remainder is a long desription and should include brief help on how to fix the detected issue.

• The class property severity defaults to ERROR but can be set to INFO or WARNING for informative checks that should not cause linting to fail.

• The class property requires may contain a list of other check classes that are required to have passed before this check is executed. Use this to avoid duplicate errors presented, or to ensure that asumptions made by your check are met by the recipe.

• Each class is instantiated once per linting run. Do slow preparation work in the constructor. E.g. the recipe_is_blacklisted check loads and parses the blacklist here.

• As each recipe is linted, your check will get called on three functions: check_recipe, check_deps and check_source.

  The first is simply passed the Recipe object, which you can use to inspect the recipe in question. The latter two are for convenience and performance. The check_deps call receives a mapping of all dependencies mentioned in the recipe to a list of locations in which the dependency occurred. E.g.:

  {'setuptools': ['requirements/run',
                  'outputs/0/requirements/run']}
  

  The check_sources is called for each source listed in the recipe, whether source: is a dict or a source, eliminating the need to repeat handling of these two cases in each check. The value source passed is a dictionary of the source, the value section a path to the source section.

• When a check encounters an issue, it should use self.message() to record the issue. The message is commonly modified with a path in the meta data to point to the part of the meta.yaml that lead to the error. This will be used to position annotations on github.

  If a check never calls self.message(), it is assumed to have passed (=no errors generated).

  You can also provide an alternate filename (fname) and/or specify the line number directly (line).

Module Autodocs

Environment Variables

LINT_SKIP

If set, will be parsed instead of the most recent commit. To skip checks for specific recipes, add strings of the following form:

[lint skip <check_name> for <recipe_name>]

check_build_help	Build tool usage
check_completeness	Completeness
check_deprecation	Deprecated packages and syntax
check_noarch	Use of noarch and skip
check_policy	Policy compliance
check_repo	Repository checks
check_syntax	Syntax checks

Functions

get_checks()

Loads and returns the available lint checks

Classes

LintCheck(_linter)	Base class for lint checks
LintCheckMeta(name, bases, …], namespace, …)	Meta class for lint checks
LintMessage(recipe, check, severity, title, …)	Message issued by LintChecks
Linter(config, recipe_folder[, exclude, nocatch])	Lint executor
Severity(value)	Severities for lint checks
conda_render_failure(_linter)	The recipe was not understood by conda-build
duplicate_key_in_meta_yaml(_linter)	The recipe meta.yaml contains a duplicate key
empty_meta_yaml(_linter)	The recipe has an empty meta.yaml!?
jinja_render_failure(_linter)	The recipe could not be rendered by Jinja2
linter_failure(_linter)	An unexpected exception was raised during linting
missing_build(_linter)	The recipe is missing a build section.
missing_meta_yaml(_linter)	The recipe is missing a meta.yaml file
missing_version_or_name(_linter)	The recipe is missing name and/or version
unknown_check(_linter)	Something went wrong inside the linter
unknown_selector(_linter)	The recipe failed to parse due to selector lines

Documentation

class bioconda_utils.lint.LintCheck(_linter)

Base class for lint checks

check_deps(deps)

Execute check on recipe dependencies

Example format for deps:

{
  'setuptools': ['requirements/run',
                 'outputs/0/requirements/run/1'],
  'compiler_cxx': ['requirements/build/0']
}

You can use the values in the list directly as section parameter to self.message().

Parameters

deps (Dict[str, List[str]]) – Dictionary mapping requirements occurring in the recipe to their locations within the recipe.

Return type

None

check_recipe(recipe)

Execute check on recipe

Override this method in subclasses, using self.message() to issue LintMessage as failures are encountered.

Parameters

recipe (Recipe) – The recipe under test.

Return type

None

check_source(source, section)

Execute check on each source

Parameters

• source (Dict) – Dictionary containing the source section

• section (str) – Path to the section. Can be source or source/0 (1,2,3…).

Return type

None

fix(message, data)

Attempt to fix the problem

Return type

LintMessage

classmethod make_message(recipe, section=None, fname=None, line=None, canfix=False)

Create a LintMessage

Parameters

• section (Optional[str]) – If specified, a lint location within the recipe meta.yaml pointing to this section/subsection will be added to the message

• fname (Optional[str]) – If specified, the message will apply to this file, rather than the recipe meta.yaml

• line – If specified, sets the line number for the message directly

Return type

LintMessage

message(section=None, fname=None, line=None, data=None)

Add a message to the lint results

Also calls fix if we are supposed to be fixing.

Parameters

• section (Optional[str]) – If specified, a lint location within the recipe meta.yaml pointing to this section/subsection will be added to the message

• fname (Optional[str]) – If specified, the message will apply to this file, rather than the recipe meta.yaml

• line (Optional[int]) – If specified, sets the line number for the message directly

• data (Optional[Any]) – Data to be passed to fix. If check can fix, set this to something other than None.

Return type

None

messages: List[LintMessage]

Messages collected running tests

recipe: _recipe.Recipe

Recipe currently being checked

requires: List[LintCheck] = []

Checks that must have passed for this check to be executed.

run(recipe, fix=False)

Run the check on a recipe. Called by Linter

Parameters

• recipe (Recipe) – The recipe to be linted

• fix (bool) – Whether to attempt to fix the recipe

Return type

List[LintMessage]

severity: bioconda_utils.lint.Severity = 30

Severity of this check. Only ERROR causes a lint failure.

try_fix: bool

Whether we are supposed to fix

class bioconda_utils.lint.LintCheckMeta(name: str, bases: Tuple[type, …], namespace: Dict[str, Any], **kwargs)

Meta class for lint checks

Handles registry

Creates LintCheck classes

class bioconda_utils.lint.LintMessage(recipe: bioconda_utils.recipe.Recipe, check: LintCheck, severity: bioconda_utils.lint.Severity = <Severity.ERROR: 30>, title: str = '', body: str = '', start_line: int = 0, end_line: int = 0, fname: str = 'meta.yaml', canfix: bool = False)

Message issued by LintChecks

Create new instance of LintMessage(recipe, check, severity, title, body, start_line, end_line, fname, canfix)

property body

Message body to be presented to user

property canfix

Whether the problem can be auto fixed

property check

The check issuing the message

property end_line

Line at which problem ends

property fname

Name of file in which error was found

get_level()

Return level string as required by github

property recipe

The recipe this message refers to

property severity

The severity of the message

property start_line

Line at which problem begins

property title

Message title to be presented to user

class bioconda_utils.lint.Linter(config, recipe_folder, exclude=None, nocatch=False)

Lint executor

Parameters

• config (Dict) – Configuration dict as provided by utils.load_config().

• recipe_folder (str) – Folder which recipes are located.

• exclude (Optional[List[str]]) – List of function names in registry to skip globally. When running on CI, this will be merged with anything else detected from the commit message or LINT_SKIP environment variable using the special string “[skip lint <function name> for <recipe name>]”. While those other mechanisms define skipping on a recipe-specific basis, this argument can be used to skip tests for all recipes. Use sparingly.

• nocatch (bool) – Don’t catch exceptions in lint checks and turn them into linter_error lint messages. Used by tests.

clear_messages()

Clears the lint messages stored in linter

get_blacklist()

Loads the blacklist as per linter configuration

Return type

Set[str]

get_messages()

Returns the lint messages collected during linting

Return type

List[LintMessage]

lint(recipe_names, fix=False)

Run linter on multiple recipes

Lint messages are collected in the linter. They can be retrieved with get_messages and the list cleared with clear_messages.

Parameters

• recipe_names (List[str]) – List of names of recipes to lint

• fix (bool) – Whether checks should attempt to fix detected issues

Return type

bool

Returns

True if issues with errors were found

lint_one(recipe_name, fix=False)

Run the linter on a single recipe

Parameters

• recipe_name (str) – Mames of recipe to lint

• fix (bool) – Whether checks should attempt to fix detected issues

Return type

List[LintMessage]

Returns

List of collected messages

load_skips()

Parses lint skips

If LINT_SKIP or the most recent commit contains [ lint skip <check_name> for <recipe_name> ], that particular check will be skipped.

class bioconda_utils.lint.Severity(value)

Severities for lint checks

ERROR = 30

Checks of this severity must be fixed and will fail a recipe.

INFO = 10

Checks of this severity are purely informational

WARNING = 20

Checks of this severity indicate possible problems

class bioconda_utils.lint.conda_render_failure(_linter)

The recipe was not understood by conda-build

Please request help from @bioconda/core.

class bioconda_utils.lint.duplicate_key_in_meta_yaml(_linter)

The recipe meta.yaml contains a duplicate key

This is invalid YAML, as it’s unclear what the structure should become. Please merge the two offending sections

class bioconda_utils.lint.empty_meta_yaml(_linter)

The recipe has an empty meta.yaml!?

Please check if you forgot to commit its contents.

bioconda_utils.lint.get_checks()

Loads and returns the available lint checks

class bioconda_utils.lint.jinja_render_failure(_linter)

The recipe could not be rendered by Jinja2

Check if you are missing quotes or curly braces in Jinja2 template expressions. (The parts with {{ something }} or {% set var="value" %}).

class bioconda_utils.lint.linter_failure(_linter)

An unexpected exception was raised during linting

Please file an issue at the bioconda-utils repo

class bioconda_utils.lint.missing_build(_linter)

The recipe is missing a build section.

Please add:

build:
  number: 0

class bioconda_utils.lint.missing_meta_yaml(_linter)

The recipe is missing a meta.yaml file

Most commonly, this is because the file was accidentally named meta.yml. If so, rename it to meta.yaml.

class bioconda_utils.lint.missing_version_or_name(_linter)

The recipe is missing name and/or version

Please make sure the recipe has at least:

package:
  name: package_name
  version: 0.12.34

bioconda_utils.lint.recipe_error_to_lint_check = {<class 'bioconda_utils.recipe.DuplicateKey'>: <class 'bioconda_utils.lint.duplicate_key_in_meta_yaml'>, <class 'bioconda_utils.recipe.MissingKey'>: <class 'bioconda_utils.lint.missing_version_or_name'>, <class 'bioconda_utils.recipe.EmptyRecipe'>: <class 'bioconda_utils.lint.empty_meta_yaml'>, <class 'bioconda_utils.recipe.MissingBuild'>: <class 'bioconda_utils.lint.missing_build'>, <class 'bioconda_utils.recipe.HasSelector'>: <class 'bioconda_utils.lint.unknown_selector'>, <class 'bioconda_utils.recipe.MissingMetaYaml'>: <class 'bioconda_utils.lint.missing_meta_yaml'>, <class 'bioconda_utils.recipe.CondaRenderFailure'>: <class 'bioconda_utils.lint.conda_render_failure'>, <class 'bioconda_utils.recipe.RenderFailure'>: <class 'bioconda_utils.lint.jinja_render_failure'>}

Maps _recipe.RecipeError to LintCheck

class bioconda_utils.lint.unknown_check(_linter)

Something went wrong inside the linter

Please request help from @bioconda/core

class bioconda_utils.lint.unknown_selector(_linter)

The recipe failed to parse due to selector lines

Please request help from @bioconda/core.