Standalone BQSR crashing but full FQ2BAM succeeding

Healthcare & Life Sciences Parabricks
1

I’m noticing curious behavior where an alignment will crash if I run fq2bam --no-markdups and then a standalone BQSR while the same dataset, reference genome, and dbSNP will succeed if I let fq2bam run every stage. I’ve noticed this behavior is particular to my mouse alignments using the reference genome Mus_musculus.GRCm38.dna.toplevel.fa and the dbSNP mus_musculus.sort.vcf.gz. My alignments using human or rat references work just fine in the same pipeline. I wish to avoid duplicate marking for libraries that haven’t been PCR-amplified so it’s not ideal to have to execute a full run of fq2bam as this theoretically results in unexpected changes to the BQSR recalibration. This is the full fq2bam run code:


  singularity exec instance://"$inst" \
    pbrun fq2bam \
    --num-gpus 2 \
    --ref $refgenome \
    "${known_args[@]}" \
    --bwa-options "-K 5000000" \
    "${fq_args[@]}" \
    --out-bam $SCRATCH_BASE/4-parabricks/$samplename-pe.rg.bam \
    --out-recal-file $SCRATCH_BASE/4-parabricks/$samplename-pe.recal.grp \
    --out-duplicate-metrics $SCRATCH_BASE/4-parabricks/duplicates_$samplename.txt \
    --memory-limit 150

And this is the split fq2bam + standalone bqsr command:


  singularity exec instance://"$inst" \
    pbrun fq2bam \
    --no-markdup \
    --num-gpus 2 \
    --ref $refgenome \
    --bwa-options "-K 5000000" \
    "${fq_args[@]}" \
    --out-bam $SCRATCH_BASE/4-parabricks/$samplename-pe.rg.bam \
    --memory-limit 150 \
    --gpusort \
    --gpuwrite

  singularity exec instance://"$inst" \
    pbrun bqsr \
    --num-gpus 1 \
    --verbose \
    --ref $refgenome \
    "${known_args[@]}" \
    --in-bam $SCRATCH_BASE/4-parabricks/$samplename-pe.rg.bam \
    --out-recal-file $SCRATCH_BASE/4-parabricks/$samplename-pe.recal.grp

For both runs, the known_args[@] variable contained a value of

--knownSites mus_musculus.sort.vcf.gz

I am using Parabricks 4.6.0-2

The error log from the crashed run was as follows:

[PB Info 2026-Mar-05 13:22:23] ------------------------------------------------------------------------------
[PB Info 2026-Mar-05 13:22:23] ||                 Parabricks accelerated Genomics Pipeline                 ||
[PB Info 2026-Mar-05 13:22:23] ||                              Version 4.6.0-2                             ||
[PB Info 2026-Mar-05 13:22:23] ||                                   bqsr                                   ||
[PB Info 2026-Mar-05 13:22:23] ------------------------------------------------------------------------------
[PB Info 2026-Mar-05 13:23:16] ProgressMeter -  Current-Locus   Elapsed-Minutes
[PB Info 2026-Mar-05 13:23:16] Checking if the index file exists for the input compressed file:/gs/gsfs0/users/murray-maslov-lab/reference/mouse/mus_musculus.sort.vcf.gz for better performance
[PB Info 2026-Mar-05 13:23:17] ProgressMeter -  :0      0.0
[PB Info 2026-Mar-05 13:23:18] ProgressMeter -  :0      0.0
[PB Info 2026-Mar-05 13:23:19] ProgressMeter -  :0      0.1
[PB Info 2026-Mar-05 13:23:20] ProgressMeter -  :0      0.1
[PB Info 2026-Mar-05 13:23:21] ProgressMeter -  :0      0.1
[PB Info 2026-Mar-05 13:23:22] ProgressMeter -  :0      0.1
[PB Info 2026-Mar-05 13:23:23] ProgressMeter -  :0      0.1
[PB Info 2026-Mar-05 13:23:24] ProgressMeter -  :0      0.1
[PB Info 2026-Mar-05 13:23:25] ProgressMeter -  :0      0.1
[PB Info 2026-Mar-05 13:23:26] ProgressMeter -  :0      0.2
[PB Info 2026-Mar-05 13:23:27] ProgressMeter -  :0      0.2
[PB Info 2026-Mar-05 13:23:28] ProgressMeter -  :0      0.2
[PB Info 2026-Mar-05 13:23:29] ProgressMeter -  :0      0.2
[PB Info 2026-Mar-05 13:23:30] ProgressMeter -  :0      0.2
[PB Info 2026-Mar-05 13:23:31] ProgressMeter -  :0      0.2
[PB Info 2026-Mar-05 13:23:32] ProgressMeter -  :0      0.3
[PB Info 2026-Mar-05 13:23:33] ProgressMeter -  :0      0.3
[PB Info 2026-Mar-05 13:23:34] ProgressMeter -  :0      0.3
[PB Info 2026-Mar-05 13:23:35] ProgressMeter -  :0      0.3
[PB Info 2026-Mar-05 13:23:36] ProgressMeter -  :0      0.3
[PB Info 2026-Mar-05 13:23:37] ProgressMeter -  :0      0.3
[PB Info 2026-Mar-05 13:23:38] ProgressMeter -  :0      0.4
[PB Info 2026-Mar-05 13:23:39] ProgressMeter -  :0      0.4
[PB Info 2026-Mar-05 13:23:40] ProgressMeter -  :0      0.4
[PB Info 2026-Mar-05 13:23:41] ProgressMeter -  :0      0.4
[PB Info 2026-Mar-05 13:23:42] ProgressMeter -  :0      0.4
[PB Info 2026-Mar-05 13:23:43] ProgressMeter -  :0      0.5
[PB Info 2026-Mar-05 13:23:44] ProgressMeter -  :0      0.5
[PB ESC[31mErrorESC[0m 2026-Mar-05 13:23:45][src/BaseRecalibrationEngine.cu:2629] cudaSafeCall() failed: invalid argument, exiting.
Process exited with failure. Please check the logs.

Is there some nuance to how the full fq2bam is transforming the files or some transitional output from markdup that impacts how the standalone bqsr performs?

2

I was able to resolve the issue by downloading a fresh version of the latest Ensemble release of the mm38 primary-assembly reference genome and its associated dbSNP and re-indexing them. I never did figure out why the crash was occurring as the logging is too vague but clearly something was more fragile in Parabrick’s standalone BQSR to cause the issue.

I’m happy to provide the offending reference genome and dbSNP combo if someone would like to try reproducing the issue