Skip to content

Add vepyr/annotate module - #13001

Open
mwiewior wants to merge 4 commits into
nf-core:masterfrom
mwiewior:add-vepyr-annotate
Open

mwiewior wants to merge 4 commits into
nf-core:masterfrom
mwiewior:add-vepyr-annotate

Conversation

@mwiewior

Copy link
Copy Markdown

Add vepyr/annotate to annotate VCFs against a vepyr Parquet cache and emit a bgzip VCF with its tabix index. The module uses Bioconda vepyr 0.7.0 and htslib 1.24, accepts reference indexes and an optional plugin cache, and uses task CPUs for annotation parallelism with a single-worker fallback for unindexed input.

The real annotation test uses 1,000 HG002 chr22 records, the Ensembl 116 cache and GRCh38 reference now published by nf-core/test-datasets#2270. It extracts the cache through UNTAR and snapshots the --everything annotation digest, output names and tool versions. A second test covers stub output. Docker/Singularity container metadata and generated Conda lock files are included for both supported architectures.

Validation in a fresh checkout of nf-core/modules:

  • nf-core modules lint vepyr/annotate: 80 passed, 0 warnings, 0 failures.
  • Repository pre-commit hooks: passed, including schemas, formatting and Nextflow lint.
  • nf-test test modules/nf-core/vepyr/annotate/tests --profile=+docker,arm64 --ci: 2/2 passed on Nextflow 25.10.2 (the CI version) and 26.04.6, with nf-test 0.9.5 and no test-data URL override.
  • The published-data output also matches the recorded Ensembl VEP 116 record-body MD5 1d4a92b815eb6193e6f4546f4ab35978 for all 1,000 records.

Snapshots were generated from the published data on ARM64 and remained unchanged during both upstream test runs. Native x64, Conda and Singularity execution still need upstream CI validation. This PR contains only the module; the normalisation subworkflow will follow separately after the module is merged.

Generated by Codex

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR. Published in Add test data for the vepyr/annotate module (Ensembl 116, HG002 chr22) test-datasets#2270.
  • Remove all TODO statements.
  • Broadcast software version numbers to topic: versions.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label.
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • Docker: equivalent direct nf-test command above, using the ARM64 profile.
    • nf-core modules test vepyr/annotate --profile singularity
    • nf-core modules test vepyr/annotate --profile conda

Annotate VCFs against a Parquet VEP cache with Bioconda vepyr 0.7.0.
Include version outputs, platform container metadata, generated lock files,
and real/stub nf-tests against the published HG002 chr22 fixture.

Import the module from biodatageeks/vepyr with its verified published-data
snapshot.

Generated by Codex
@mwiewior
mwiewior marked this pull request as ready for review September 20, 2026 04:49
@SPPearce

Copy link
Copy Markdown
Contributor

Can you please join the nf-core organisation via the github-invitations channel on slack

Comment thread modules/nf-core/vepyr/annotate/main.nf Outdated
prefix = task.ext.prefix ?: "${meta.id}"
// vepyr opens the reference through its .fai and does not build one, so the
// index must be staged alongside the FASTA or --everything/--hgvsc fail. A
// bgzip FASTA also needs its .gzi: pass [ fai, gzi ] in the fai slot.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be explicitly incorporated into the input channel then.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 69cbfa9: the reference tuple is now tuple val(meta3), path(fasta), path(fai), path(gzi), with gzi documented as its own optional input in meta.yml ([] for an uncompressed FASTA). Tests updated; outputs and the snapshot are unchanged.

A bgzip FASTA's .gzi was passed as [ fai, gzi ] in the fai slot. Declare it
as its own path(gzi) element of the reference tuple, as samtools/merge
does; pass [] for an uncompressed FASTA. Outputs and the snapshot are
unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Comment thread modules/nf-core/vepyr/annotate/main.nf Outdated
Comment thread modules/nf-core/vepyr/annotate/main.nf Outdated
Comment thread modules/nf-core/vepyr/annotate/main.nf Outdated
…none

- Drop the input/ staging: the output defaults to ${meta.id}_vepyr.vcf.gz
  and the module errors when the prefix equals the input's name.
- Build a tabix index when none is supplied so --fork can use task.cpus;
  a plain .vcf, or a .gz that is not BGZF, runs on a single pipeline.
- Remove the fai/gzi staging comment.
- New test: bgzip input without an index. Snapshot: file names only, the
  variantsMD5 is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
mwiewior added a commit to biodatageeks/vepyr that referenced this pull request Sep 20, 2026
…the input when it has none

Second review round on nf-core/modules#13001:

- Drop the non-standard input/ staging. The output defaults to
  ${meta.id}_vepyr.vcf.gz and the usual nf-core guard errors when the
  prefix equals the input's name.
- When no index is supplied, build one with tabix so --fork can use
  task.cpus. A plain .vcf, or a .gz that tabix rejects as not BGZF, still
  runs on a single pipeline rather than failing.
- Remove the fai/gzi staging comment; meta.yml already says it.

New module test: bgzip input without an index, same variantsMD5 as the
indexed run. The dev parity test printed a hardcoded input/ path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@mwiewior

Copy link
Copy Markdown
Author

hey @SPPearce - could you please check if it's ready for merging now ?

SPPearce

This comment was marked as duplicate.

vepyr annotates with --everything, which needs the FASTA, so the module
now stops with the reason instead of starting a task vepyr rejects, and
always passes --fasta. meta.yml marks the FASTA and .fai as required; a
new test covers the failure.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@mwiewior

Copy link
Copy Markdown
Author

Pushed 3811c74: the module now requires the reference FASTA. vepyr annotates with --everything, which needs it, so VEPYR_ANNOTATE stops with an explicit error when channel 3 carries none, instead of starting a task that vepyr rejects. --fasta is always passed, meta.yml marks fasta/fai as required, and a new test (no fasta - fails) covers it. The existing tests and snapshots are unchanged (nf-test --ci passes locally).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants