Nextflow config ¶

All nf-lamin configuration lives in the lamin {} scope of your nextflow.config.

Best-practice config¶

A recommended starting point for nf-core-style pipelines. The key idea is to exclude all output files by default (exclude_pattern = '.*'), then use type = 'include' rules to opt-in to the files that matter. Pre-define rules for optional outputs with enabled = false so users can turn them on without writing new patterns.

plugins {
  id 'nf-lamin'
}

lamin {
  // Link all artifacts, runs, and transforms to a project
  project_uids = ['projXXXXXXXXXXXXXX']

  // Track input artifacts
  input_artifacts {
    rules {
      // Explicitly track files not staged into Nextflow processes
      samplesheet { include_paths = { params.input }; kind = 'dataset' }
      fastq_reads { pattern = '.*\\.fastq(\\.gz)?$'; kind = 'dataset' }
      reference   { pattern = '.*\\.(fasta|fa)(\\.gz)?$'; kind = 'dataset' }
      annotation  { pattern = '.*\\.(gtf|gff)(\\.gz)?$'; kind = 'dataset' }
    }
  }

  // Track output artifacts
  output_artifacts {
    exclude_pattern = '.*'
    rules {
      // Enabled by default
      reports      { type = 'include'; pattern = '.*\\.html$'; kind = 'report' }
      mapped_reads { type = 'include'; pattern = '.*\\.bam$'; kind = 'dataset' }
      // Disabled (opt-in)
      bam_index    { type = 'include'; enabled = false; pattern = '.*\\.bai$'; kind = 'dataset' }
    }
  }
}

To enable the optional BAM index tracking, a user could modify the config above, or create a new config file with just the override:

lamin.output_artifacts.rules.bam_index.enabled = true

The sections below document each setting in detail.

`lamin` - top-level settings¶

Setting	Type	Default	Env variable	Description
`instance`	String	env var	`LAMIN_CURRENT_INSTANCE`	LaminDB instance (`owner/name`)
`api_key`	String	env var	`LAMIN_API_KEY`	LaminHub API key
`project_uids`	List	`null`	`LAMIN_CURRENT_PROJECT`	Project UIDs to link to all records
`ulabel_uids`	List	`null`		ULabel UIDs to link to all records
`space_uid`	String	`null`		Space UID
`branch_uid`	String	`null`		Branch UID
`env`	String	`'prod'`	`LAMIN_ENV`	Environment (`'prod'` or `'staging'`)
`dry_run`	Boolean	`false`	`LAMIN_DRY_RUN`	Validate config without creating records
`transform_uid`	String	`null`	`LAMIN_TRANSFORM_UID`	Override the auto-generated transform UID
`run_uid`	String	`null`	`LAMIN_RUN_UID`	Override the auto-generated run UID

Experimental: UID fields (project_uids, ulabel_uids, space_uid, branch_uid) also accept named references: '?name' (lookup by name, omit if missing), '!name' (lookup, error if missing), '+name' (create if missing). This is an experimental feature and may be removed in a future release.

`lamin.run` / `lamin.transform` - record-specific metadata¶

Attach ULabel UIDs specifically to runs or transforms. These are merged with the root-level ulabel_uids.

Setting	Type	Default
`ulabel_uids`	List	`[]`

lamin {
  run       { ulabel_uids = ['ulab-run-specific'] }
  transform { ulabel_uids = ['ulab-transform-specific'] }
}

Artifact tracking¶

Control which files are tracked and what metadata is attached. Configure tracking either globally (artifacts) or separately for inputs and outputs (input_artifacts / output_artifacts). These two approaches are mutually exclusive.

Artifact config options¶

Apply to artifacts, input_artifacts, or output_artifacts:

Setting	Type	Default	Description
`enabled`	Boolean	`true`	Enable/disable tracking
`exclude_work_dir`	Boolean	`true`	Ignore paths in the Nextflow work directory
`exclude_assets_dir`	Boolean	`true`	Ignore paths in the Nextflow assets directory
`include_pattern`	String	`null`	Regex; only matching files are tracked
`exclude_pattern`	String	`null`	Regex; matching files are skipped
`ulabel_uids`	List	`null`	ULabel UIDs for matched artifacts
`kind`	String	`null`	Artifact kind (e.g. `'dataset'`, `'report'`)
`include_paths`	String / List / Closure	`null`	Paths to explicitly track (see below)
`rules`	Map	`{}`	Pattern-based rules (see below)

Explicit paths (`include_paths`)¶

Some files are never staged into Nextflow process input channels (e.g. samplesheets parsed by Groovy helpers like nf-schema’s samplesheetToList). Use include_paths to explicitly track these files.

include_paths accepts a string, a list of strings, or a closure returning a string or list:

On an artifact config:

input_artifacts {
  include_paths = { params.input }
}

On a rule (to attach metadata like kind):

input_artifacts {
  rules {
    samplesheet {
      include_paths = { params.input }
      kind = 'dataset'
    }
  }
}

Note

Use closures for params.* references

When the lamin {} config scope is first evaluated, not all Nextflow params may be available yet. For example, params set by a profile (-profile test) or pulled in via includeConfig are resolved later. Using a closure defers the evaluation until params are fully resolved:

// safe: evaluated after all params are resolved
include_paths = { params.input }

// may fail if the param isn't available yet
include_paths = params.input

Input paths are resolved at the beginning of the workflow (onFlowBegin), and output paths just before finalizing the run. Resolved paths go through the same deduplication, metadata linking, and rule evaluation as auto-detected artifacts.

Rules¶

Rules apply different settings based on file patterns or explicit paths. Each rule is a named block. Either pattern or include_paths (or both) must be specified.

Setting	Type	Default	Description
`pattern`	String	`null`	Java regex to match file paths
`include_paths`	String / List / Closure	`null`	Paths to explicitly track
`enabled`	Boolean	`true`	Enable/disable this rule
`type`	String	`'include'`	`'include'` or `'exclude'`
`direction`	String	`'both'`	`'input'`, `'output'`, or `'both'`
`order`	Integer	`100`	Priority (lower = evaluated first)
`ulabel_uids`	List	`null`	ULabel UIDs for matched artifacts
`kind`	String	`null`	Override artifact kind

Evaluation order:

Global include_pattern / exclude_pattern are checked first
Rules are evaluated by order (lower first)
All matching rules are applied; later rules can override earlier ones
ULabel UIDs from all matching rules are merged (deduplicated)

Patterns are Java regular expressions. Backslashes must be escaped in Groovy: \\. not \.

Example: direction-specific tracking¶

lamin {
  input_artifacts {
    rules {
      samplesheet { include_paths = { params.input }; kind = 'dataset'; order = 1 }
      reference   { pattern = '.*\\.(fasta|gtf)$'; kind = 'dataset' }
      fastqs      { pattern = '.*\\.fastq\\.gz$'; kind = 'dataset' }
    }
  }

  output_artifacts {
    exclude_pattern = '.*\\.(log|tmp)$'
    rules {
      exclude_intermediate { type = 'exclude'; pattern = '.*intermediate.*'; order = 1 }
      bam_files  { pattern = '.*\\.bam$'; kind = 'dataset'; order = 2 }
      vcf_files  { pattern = '.*\\.vcf\\.gz$'; kind = 'dataset'; order = 3 }
    }
  }
}

Example: disable all artifact tracking¶

lamin {
  output_artifacts { enabled = false }
  input_artifacts  { enabled = false }
}

`lamin.features` - Feature flags¶

Optional toggles for plugin features.

Setting	Type	Default	Description
`manage_s3_credentials`	Boolean	`true`	Enable automatic credential federation for S3 (see `lamin://` URIs)
`enable_artifact_hashing`	Boolean	`true`	Compute the size and hash of `s3://` artifacts in the plugin (see Artifact hashing)

Example — disable credential federation if it causes issues in your environment:

lamin {
  features {
    manage_s3_credentials = false
  }
}

When manage_s3_credentials = false, the plugin resolves lamin:// URIs to their underlying s3:// paths and lets Nextflow handle authentication via the standard credential provider chain (environment variables, AWS credentials file, instance profile, etc.). This applies to publishing as well as reading.

`space_uid` and publish targets¶

A publish target that names a space (lamin://owner/instance?space=<uid>) decides the space of the artifacts published to it, because the storage location it resolves to is the one LaminDB associates with that space. The same holds for a ?storage=<uid> that sits in a space. lamin.space_uid applies to everything else, including publish targets whose storage location is not in a space.

Artifact hashing¶

LaminDB stores the size, hash and file count of an artifact. When the Lamin Hub cannot read a storage location (for example a private bucket that is only registered for provenance), it cannot compute them. By default the plugin computes them itself for s3:// and lamin-s3:// artifacts, the same way lamindb does, and passes them along when the artifact is created:

a single object: one HeadObject call; the ETag gives the md5 hash, or md5-<N> for a multipart upload
a directory-like prefix: a paginated ListObjectsV2 call; the per-object hashes are combined into an md5-d hash, with the summed size and the file count

Other locations (gs://, az://, https://) are left to LaminDB.

Credentials — lamin-s3:// artifacts use the federated credentials of their storage location. Plain s3:// artifacts use the aws scope of nextflow.config (accessKey / secretKey or profile, region, and client.endpoint / client.s3PathStyleAccess / client.anonymous for S3-compatible stores such as MinIO), else the default AWS credential provider chain, else anonymous access for public buckets. Assumed roles and SSO logins are not supported; when the plugin cannot read an object it silently leaves the size and hash to LaminDB.

Older instances — a LaminDB instance older than laminlabs/lamindb#3995 rejects the request. The plugin then warns once, retries without the size and hash and stops sending them for the rest of the run.

Set enable_artifact_hashing = false to skip the extra S3 calls altogether:

lamin {
  features {
    enable_artifact_hashing = false
  }
}

`lamin.api` - API connection (Advanced settings)¶

Setting	Type	Default	Env variable
`supabase_api_url`	String	`null`	`SUPABASE_API_URL`
`supabase_anon_key`	String	`null`	`SUPABASE_ANON_KEY`
`max_retries`	Integer	`3`	`LAMIN_MAX_RETRIES`
`retry_delay`	Integer	`100`	`LAMIN_RETRY_DELAY`
`max_workers`	Integer	`8`	`LAMIN_MAX_WORKERS`

Only needed for custom LaminHub deployments or to tune retry and parallelism behavior. max_workers controls the size of the thread pool used to create artifacts in parallel.

Troubleshooting¶

`ERROR ~ Unknown config attribute` when using `params` in config¶

ERROR ~ Unknown config attribute `lamin.input_artifacts.rules.samplesheet.params.input` -- check config file

When the lamin {} config scope is evaluated, not all Nextflow params are necessarily available yet. Params set by a profile (-profile test), pulled in via includeConfig, or resolved only after plugin loading may be absent at that point. A bare params.input reference then creates an unresolved placeholder that triggers this error.

Fix: wrap params.* references in a closure so evaluation is deferred until all params have been fully resolved:

// Safe: evaluated after all params (CLI, profiles, includeConfig) are resolved
include_paths = { params.input }
include_paths = { ["${params.outdir}/samplesheet/samplesheet.csv"] }

// May fail if the param is not yet available when this line is evaluated
include_paths = params.input

Nextflow config ¶

Best-practice config¶

lamin - top-level settings¶

lamin.run / lamin.transform - record-specific metadata¶