Mount the dreamdock-data PVC in the k8s profile, and probe for it

The PVC holding chembl_36.db and protein_seq.zarr was never mounted into task
pods: WES's injected profile.config supplies no storage config (verified across
a full run's logs). Without /mnt/dreamdock-data, sqlite3 raises 'unable to open
database file' and every GET_FINAL_METABOLITES_STATIC task exits 1.

Restore storageClaimName/storageMountPath, and add a probe that reports what is
visible before opening sqlite so the failure names its own cause.
This commit is contained in:
Olamide Isreal
2026-07-28 11:04:46 +01:00
parent 73e2eea4be
commit 724c7d8987
2 changed files with 41 additions and 6 deletions

View File

@@ -302,6 +302,30 @@ process GET_FINAL_METABOLITES_STATIC {
import numpy as np
import pandas as pd
import sqlite3
import os, sys
# --- diagnostic probe -------------------------------------------------
# The ChEMBL DB lives on the dreamdock-data PVC. If that PVC is not mounted
# into the task pod, sqlite3 fails with a generic "unable to open database
# file" that is indistinguishable from a corrupt/locked DB. Report what is
# actually visible before touching sqlite, so the failure names its cause.
_db = '$chembl_db'
print('[probe] chembl_db path : %s' % _db, file=sys.stderr)
print('[probe] exists : %s' % os.path.exists(_db), file=sys.stderr)
for _p in ('/mnt/dreamdock-data', os.path.dirname(_db)):
print('[probe] dir %-22s exists=%s' % (_p, os.path.isdir(_p)), file=sys.stderr)
if os.path.exists(_db):
print('[probe] size : %d bytes' % os.path.getsize(_db), file=sys.stderr)
print('[probe] readable : %s' % os.access(_db, os.R_OK), file=sys.stderr)
else:
sys.exit(
"[probe] FATAL: %s is not visible inside this task container.\n"
"[probe] The dreamdock-data PVC is not mounted. Check that the k8s\n"
"[probe] profile sets k8s.storageClaimName/storageMountPath and that\n"
"[probe] the WES-injected profile.config does not override them."
% _db
)
# --- end probe --------------------------------------------------------
# Connect to ChEMBL database
conn = sqlite3.connect('file:$chembl_db?mode=ro&immutable=1', uri=True)

View File

@@ -30,14 +30,25 @@ process {
}
profiles {
// Minimal k8s profile: WES injects its own profile.config via -C, which supplies the
// executor, namespace, service account, workDir and PVC mounts. Anything set here that
// WES also sets is either overridden or conflicts, so keep this deliberately sparse.
// Minimal k8s profile for WES. WES injects its own profile.config via -C, which
// supplies the executor, namespace, service account and workDir — so those are
// deliberately left out here to avoid conflicting with it.
//
// NOTE: the dreamdock-data PVC must be mounted by the WES-side profile for
// params.protein_zarr and params.chembl_db (/mnt/dreamdock-data/...) to resolve.
// The storage block is NOT optional. WES's injected profile does not mount any
// PVC (verified: a full experiment run produced 468 KB of logs with zero
// references to dreamdock/storageClaimName/volumeClaim). Without it,
// /mnt/dreamdock-data does not exist inside task pods and every process that
// reads params.chembl_db or params.protein_zarr fails immediately —
// GET_FINAL_METABOLITES_STATIC dies with
// sqlite3.OperationalError: unable to open database file
// and exit status 1, on every task, regardless of input.
k8s {
process.executor = 'k8s'
k8s {
storageClaimName = 'dreamdock-data'
storageMountPath = '/mnt/dreamdock-data'
}
}
// Full standalone k8s profile for running this pipeline directly (not via WES),