Mount the dreamdock-data PVC in the k8s profile, and probe for it
The PVC holding chembl_36.db and protein_seq.zarr was never mounted into task pods: WES's injected profile.config supplies no storage config (verified across a full run's logs). Without /mnt/dreamdock-data, sqlite3 raises 'unable to open database file' and every GET_FINAL_METABOLITES_STATIC task exits 1. Restore storageClaimName/storageMountPath, and add a probe that reports what is visible before opening sqlite so the failure names its own cause.
This commit is contained in:
@@ -302,11 +302,35 @@ process GET_FINAL_METABOLITES_STATIC {
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import sqlite3
|
||||
import os, sys
|
||||
|
||||
# --- diagnostic probe -------------------------------------------------
|
||||
# The ChEMBL DB lives on the dreamdock-data PVC. If that PVC is not mounted
|
||||
# into the task pod, sqlite3 fails with a generic "unable to open database
|
||||
# file" that is indistinguishable from a corrupt/locked DB. Report what is
|
||||
# actually visible before touching sqlite, so the failure names its cause.
|
||||
_db = '$chembl_db'
|
||||
print('[probe] chembl_db path : %s' % _db, file=sys.stderr)
|
||||
print('[probe] exists : %s' % os.path.exists(_db), file=sys.stderr)
|
||||
for _p in ('/mnt/dreamdock-data', os.path.dirname(_db)):
|
||||
print('[probe] dir %-22s exists=%s' % (_p, os.path.isdir(_p)), file=sys.stderr)
|
||||
if os.path.exists(_db):
|
||||
print('[probe] size : %d bytes' % os.path.getsize(_db), file=sys.stderr)
|
||||
print('[probe] readable : %s' % os.access(_db, os.R_OK), file=sys.stderr)
|
||||
else:
|
||||
sys.exit(
|
||||
"[probe] FATAL: %s is not visible inside this task container.\n"
|
||||
"[probe] The dreamdock-data PVC is not mounted. Check that the k8s\n"
|
||||
"[probe] profile sets k8s.storageClaimName/storageMountPath and that\n"
|
||||
"[probe] the WES-injected profile.config does not override them."
|
||||
% _db
|
||||
)
|
||||
# --- end probe --------------------------------------------------------
|
||||
|
||||
# Connect to ChEMBL database
|
||||
conn = sqlite3.connect('file:$chembl_db?mode=ro&immutable=1', uri=True)
|
||||
# conn = sqlite3.connect('$chembl_db')
|
||||
|
||||
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Load biotransformer output
|
||||
|
||||
@@ -30,14 +30,25 @@ process {
|
||||
}
|
||||
|
||||
profiles {
|
||||
// Minimal k8s profile: WES injects its own profile.config via -C, which supplies the
|
||||
// executor, namespace, service account, workDir and PVC mounts. Anything set here that
|
||||
// WES also sets is either overridden or conflicts, so keep this deliberately sparse.
|
||||
// Minimal k8s profile for WES. WES injects its own profile.config via -C, which
|
||||
// supplies the executor, namespace, service account and workDir — so those are
|
||||
// deliberately left out here to avoid conflicting with it.
|
||||
//
|
||||
// NOTE: the dreamdock-data PVC must be mounted by the WES-side profile for
|
||||
// params.protein_zarr and params.chembl_db (/mnt/dreamdock-data/...) to resolve.
|
||||
// The storage block is NOT optional. WES's injected profile does not mount any
|
||||
// PVC (verified: a full experiment run produced 468 KB of logs with zero
|
||||
// references to dreamdock/storageClaimName/volumeClaim). Without it,
|
||||
// /mnt/dreamdock-data does not exist inside task pods and every process that
|
||||
// reads params.chembl_db or params.protein_zarr fails immediately —
|
||||
// GET_FINAL_METABOLITES_STATIC dies with
|
||||
// sqlite3.OperationalError: unable to open database file
|
||||
// and exit status 1, on every task, regardless of input.
|
||||
k8s {
|
||||
process.executor = 'k8s'
|
||||
|
||||
k8s {
|
||||
storageClaimName = 'dreamdock-data'
|
||||
storageMountPath = '/mnt/dreamdock-data'
|
||||
}
|
||||
}
|
||||
|
||||
// Full standalone k8s profile for running this pipeline directly (not via WES),
|
||||
|
||||
Reference in New Issue
Block a user