Skip to main content

CEP 2 - Add plugin architecture to conda

Title Add plugin architecture to conda
Status Accepted
Author(s) Jannis Leidel <jleidel@anaconda.com>
Created Jun 29, 2021
Updated Jun 30, 2021
Discussion https://github.com/conda/ceps/pull/1
Implementation (listed below)

Motivation​

Conda consists of many moving parts, both in the conda/conda-build projects and even 3rd party projects. It relies on loose coupling with various technical and workflow mechanisms.

Conda would profit from a generic plugin architecture because it would

  • cover use cases that are underserved at the moment,
  • support a better distribution of maintenance in the community,
  • add the ability to extend conda internals via official APIs
  • and lower the barrier for contributions from other stakeholders in the conda ecosystem.

Implementation​

  • Integrate pluggy or HookMan to host a set of hooks in conda and allow external packages to provide plugins in addition to the default implementations shipped with conda.

  • Especially for performance critical code paths, the plugin architecture would make it possible to work on innovative solutions instead of risking changes to the existing stable implementation.

Rationale​

Conda's architecture was not designed to extend it programmatically with an official plugin API. It would profit from learning from other tools in the Python ecosystem that have had plugins for years (e.g. pytest, tox) and have a thriving well-maintained ecosystem.

At the same time, conda's special role as a tool written in Python but existing in an ecosystem that tends to use the best tool for the job, means that the plugin architecture should eventually allow handling programming languages and technologies other than Python.

It's also in the best interest for the health of the conda project to provide the means for the community to create innovative plugins for conda to cater to the ever-evolving Scientific community. Third party contributors should use official APIs instead of having to divert to workarounds and wrappers.

The CLI for example can already be extended with any command on the PATH that follows the naming scheme conda-([\w\-]+)$ (or conda-([\w\-]+)\.(exe|bat)$ on Windows). Unfortunately, that does not suffice to have a low-level integration with conda internals.

So for deeper integrated functionality it's imperative to instead provide code-level plugin hooks in conda, e.g. where the solver functionality that is currently implemented as a static mapping for the libraries pycosat, pycryptosat and pysat.

Other example areas where this could be useful (not part of this CEP):

  • shell integration

  • dependency solver backend

  • network adapter

  • virtual package discovery

  • build system integration

  • language support (R, Julia, etc.)

Backward Compatibility​

  • Any plugin hook added to conda should default to the way conda works in the latest major release as of accepting its related CEP.

  • Future updates to the defaults of the conda plugin hooks should be reviewed for backward compatibility and follow clear a compatibility policy, e.g. only changed in major versions.

References​

All CEPs are explicitly CC0 1.0 Universal.