Integrations#

Integrations span across clusters, secrets, environment variables and how an execution environment is created.

They provide easy ways to configure the execution environment of your runs and properly set up dependencies.

For example, a Weights and Biases integration may uniquely identify a run and surface links to easily find the run within W&B (while handling the project, organization, and run naming for you).

Integrations are in Beta

Our supported integrations are currently in Beta and the exact usage for them is subject to change over time.

How to Specify Integrations#

Integrations are specified as a list in the MCLI YAML. Each integration in the list specifies a valid integration_type (see below for supported types) along with any parameters relevant to that integration type as shown below:

integrations:
  - integration_type: <insert_integration_type>
    <integration_param>: <integration_value>

For example:

integrations:
  - integration_type: "git_repo"
    git_repo: mosaicml/composer
    git_branch: v0.7.1
    pip_install: .
  - integration_type: "apt_packages"
    packages:
      - htop
  - integration_type: "pip_packages"
    packages:
      - numpy
  - integration_type: "wandb"
    project: ben_bitdiddle_llms
    entity: mosaicml
  - integration_type: "local"
    directories:
      - folder1
      - folder2
    push_image: myrepo/example

Note: Each integration_type has different available parameters that configure how it affects your run environment. For information on each specific integration type, visit the page for that integration.

Supported Integration Types#

Currently our supported integration types include: