omic/digital-trial
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Digital Trials pipeline configured for WES

Browse Source 
Source-only snapshot of the cluster branch for WES execution. Large
reference files (HPA/MANE/ensemble FASTA, model weights, ~597 MB) are
omitted: they are baked into the container images at build time and
mounted from the dreamdock-data PVC at runtime, and exceed the Gitea
request size limit.

Pipeline entry point is main.nf, which orchestrates the biotransformer,
conplex and tissue modules as a single workflow. Ligand inputs are read
from the eureka workspace; protein_zarr and chembl_db come from the
dreamdock-data PVC.
This commit is contained in:
Olamide Isreal
2026-07-27 21:59:52 +01:00
commit 9e75f44f1a
86 changed files with 10142 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

122
.nextflow.log.5 Normal file
View File

@@ -0,0 +1,122 @@
Feb-27 17:41:44.642 [main] DEBUG nextflow.cli.Launcher - $> nextflow run test.nf -params-file /data/dreamdock-data/caline-run6/digtrial/caline-run6.json
Feb-27 17:41:45.022 [main] DEBUG nextflow.cli.CmdRun - N E X T F L O W ~ version 25.04.3
Feb-27 17:41:45.053 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/omic/.nextflow/plugins; core-plugins: nf-amazon@2.15.0,nf-azure@1.16.0,nf-cloudcache@0.4.3,nf-codecommit@0.2.3,nf-console@1.2.1,nf-google@1.21.0,nf-k8s@1.0.0,nf-tower@1.11.3,nf-wave@1.12.1
Feb-27 17:41:45.091 [main] INFO o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Feb-27 17:41:45.092 [main] INFO o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Feb-27 17:41:45.096 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.12.0 in 'deployment' mode
Feb-27 17:41:45.109 [main] INFO org.pf4j.AbstractPluginManager - No plugins
Feb-27 17:41:45.142 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: /data/bugra/digital_trials/nextflow.config
Feb-27 17:41:45.146 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /data/bugra/digital_trials/nextflow.config
Feb-27 17:41:45.190 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /home/omic/.nextflow/secrets/store.json
Feb-27 17:41:45.196 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@6ef81f31] - activable => nextflow.secret.LocalSecretsProvider@6ef81f31
Feb-27 17:41:45.226 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Feb-27 17:41:45.999 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declaration
Feb-27 17:41:46.021 [main] DEBUG nextflow.cli.CmdRun - Launching `test.nf` [chaotic_leakey] DSL2 - revision: 5cb3581ad6
Feb-27 17:41:46.024 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Feb-27 17:41:46.025 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[nf-tower@1.11.3]
Feb-27 17:41:46.028 [main] DEBUG nextflow.plugin.PluginUpdater - Installing plugin nf-tower version: 1.11.3
Feb-27 17:41:46.042 [main] INFO org.pf4j.AbstractPluginManager - Plugin 'nf-tower@1.11.3' resolved
Feb-27 17:41:46.043 [main] INFO org.pf4j.AbstractPluginManager - Start plugin 'nf-tower@1.11.3'
Feb-27 17:41:46.062 [main] DEBUG nextflow.plugin.BasePlugin - Plugin started nf-tower@1.11.3
Feb-27 17:41:46.139 [main] DEBUG nextflow.Session - Session UUID: 6d6a967c-f1c2-433b-8c30-4f047e71a910
Feb-27 17:41:46.140 [main] DEBUG nextflow.Session - Run name: chaotic_leakey
Feb-27 17:41:46.140 [main] DEBUG nextflow.Session - Executor pool size: 80
Feb-27 17:41:46.153 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Feb-27 17:41:46.161 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=240; workQueue=LinkedBlockingQueue[-1]; allowCoreThreadTimeout=false
Feb-27 17:41:46.198 [main] DEBUG nextflow.cli.CmdRun -
Version: 25.04.3 build 5949
Created: 02-06-2025 20:56 UTC
System: Linux 6.11.0-26-generic
Runtime: Groovy 4.0.26 on OpenJDK 64-Bit Server VM 21.0.7+6-Ubuntu-0ubuntu124.04
Encoding: UTF-8 (UTF-8)
Process: 3934402@k8s-node23 [127.0.1.1]
CPUs: 80 - Mem: 251.6 GB (5 GB) - Swap: 0 (0)
Feb-27 17:41:46.237 [main] DEBUG nextflow.Session - Work-dir: /data/bugra/digital_trials/work [btrfs]
Feb-27 17:41:46.238 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /data/bugra/digital_trials/bin
Feb-27 17:41:46.254 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Feb-27 17:41:46.273 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Feb-27 17:41:46.279 [main] DEBUG nextflow.Session - Observer factory: TowerFactory
Feb-27 17:41:46.281 [main] DEBUG nextflow.Session - Observer factory (v2): LinObserverFactory
Feb-27 17:41:46.315 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Feb-27 17:41:46.328 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 81; maxThreads: 1000
Feb-27 17:41:46.574 [main] DEBUG nextflow.Session - Session start
Feb-27 17:41:47.067 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Feb-27 17:41:47.963 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-27 17:41:47.964 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-27 17:41:47.971 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Feb-27 17:41:47.981 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=80; memory=251.6 GB; capacity=80; pollInterval=100ms; dumpInterval=5m
Feb-27 17:41:47.984 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
Feb-27 17:41:48.018 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'HUMAN_TRANSFORMER': maxForks=0; fair=false; array=0
Feb-27 17:41:48.112 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-27 17:41:48.112 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-27 17:41:48.114 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'GET_FINAL_METABOLITES_STATIC': maxForks=0; fair=false; array=0
Feb-27 17:41:48.127 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:gpu_process` matches labels `gpu_process` for process with name PREPROCESS_PROTEIN
Feb-27 17:41:48.132 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-27 17:41:48.132 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-27 17:41:48.134 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'PREPROCESS_PROTEIN': maxForks=0; fair=false; array=0
Feb-27 17:41:48.193 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-27 17:41:48.193 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-27 17:41:48.194 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'CONPLEX_ALL': maxForks=0; fair=false; array=0
Feb-27 17:41:48.226 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-27 17:41:48.226 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-27 17:41:48.227 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'NETWORK_ENRICHMENT': maxForks=0; fair=false; array=0
Feb-27 17:41:48.235 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-27 17:41:48.235 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-27 17:41:48.236 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'TISSUE_DISTRIBUTION': maxForks=0; fair=false; array=0
Feb-27 17:41:48.247 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-27 17:41:48.248 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-27 17:41:48.248 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'BIO_METRICS': maxForks=0; fair=false; array=0
Feb-27 17:41:48.251 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: HUMAN_TRANSFORMER, METABOLITES_BY_MASS, GET_FINAL_METABOLITES_STATIC, CONPLEX_ALL, NETWORK_ENRICHMENT, GET_FINAL_METABOLITES, CONPLEX, BIO_METRICS, SUPER_TRANSFORMER, TISSUE_DISTRIBUTION, MERGE_DRUG, MERGE_INTERACTIONS, PREPROCESS_PROTEIN, ORDERED_SEQUENCE
Feb-27 17:41:48.254 [main] DEBUG nextflow.Session - Igniting dataflow network (11)
Feb-27 17:41:48.261 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > HUMAN_TRANSFORMER
Feb-27 17:41:48.277 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > GET_FINAL_METABOLITES_STATIC
Feb-27 17:41:48.278 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > PREPROCESS_PROTEIN
Feb-27 17:41:48.279 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > CONPLEX_ALL
Feb-27 17:41:48.279 [PathVisitor-3] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /mnt/dreamdock-data/digital_trials/input/blank/; pattern: *.fasta; options: [:]
Feb-27 17:41:48.279 [PathVisitor-1] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /data/dreamdock-data/caline-run6/digtrial/input/; pattern: *.csv; options: [:]
Feb-27 17:41:48.280 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > NETWORK_ENRICHMENT
Feb-27 17:41:48.280 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > TISSUE_DISTRIBUTION
Feb-27 17:41:48.281 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BIO_METRICS
Feb-27 17:41:48.282 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
Script_c432c676a3e7c0cb: /data/bugra/digital_trials/test.nf
Script_e39e54e7aa543fd0: /data/bugra/digital_trials/main_biotransformer.nf
Script_d345b7a65a8f3255: /data/bugra/digital_trials/main_conplex.nf
Script_ec01f2a38db15aa5: /data/bugra/digital_trials/main_tissue.nf
Feb-27 17:41:48.282 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination
Feb-27 17:41:48.282 [main] DEBUG nextflow.Session - Session await
Feb-27 17:41:48.510 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Feb-27 17:41:48.514 [Task submitter] INFO nextflow.Session - [9c/1a79a1] Submitted process > HUMAN_TRANSFORMER (1)
Feb-27 17:41:56.073 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: HUMAN_TRANSFORMER (1); status: COMPLETED; exit: 0; error: -; workDir: /data/bugra/digital_trials/work/9c/1a79a1afef130a28212413d1c010fd]
Feb-27 17:41:56.075 [Task monitor] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'TaskFinalizer' minSize=10; maxSize=240; workQueue=LinkedBlockingQueue[-1]; allowCoreThreadTimeout=false
Feb-27 17:41:56.102 [TaskFinalizer-1] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'PublishDir' minSize=10; maxSize=240; workQueue=LinkedBlockingQueue[-1]; allowCoreThreadTimeout=false
Feb-27 17:41:56.117 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Feb-27 17:41:56.118 [Task submitter] INFO nextflow.Session - [3b/c06172] Submitted process > GET_FINAL_METABOLITES_STATIC (1)
Feb-27 17:42:01.333 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: GET_FINAL_METABOLITES_STATIC (1); status: COMPLETED; exit: 0; error: -; workDir: /data/bugra/digital_trials/work/3b/c061720cd41597e8b90414cc4a66b0]
Feb-27 17:42:01.387 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Feb-27 17:42:01.388 [Task submitter] INFO nextflow.Session - [18/8af203] Submitted process > CONPLEX_ALL (1)
Feb-27 17:42:14.026 [SIGINT handler] DEBUG nextflow.Session - Session aborted -- Cause: SIGINT
Feb-27 17:42:14.046 [SIGINT handler] DEBUG nextflow.Session - The following nodes are still active:
[process] NETWORK_ENRICHMENT
status=ACTIVE
port 0: (queue) OPEN ; channel: interactions
port 1: (cntrl) - ; channel: $
[process] TISSUE_DISTRIBUTION
status=ACTIVE
port 0: (queue) OPEN ; channel: interaction
port 1: (cntrl) - ; channel: $
[process] BIO_METRICS
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
Feb-27 17:42:14.050 [main] DEBUG nextflow.Session - Session await > all processes finished
Feb-27 17:42:14.051 [main] DEBUG nextflow.Session - Session await > all barriers passed
Feb-27 17:42:14.051 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Feb-27 17:42:14.062 [main] WARN n.processor.TaskPollingMonitor - Killing running tasks (1)
Feb-27 17:42:14.084 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=2; failedCount=0; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=1; succeedDuration=12.6s; failedDuration=0ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=1; peakCpus=1; peakMemory=5 GB; ]
Feb-27 17:42:14.099 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Feb-27 17:42:14.172 [main] INFO org.pf4j.AbstractPluginManager - Stop plugin 'nf-tower@1.11.3'
Feb-27 17:42:14.173 [main] DEBUG nextflow.plugin.BasePlugin - Plugin stopped nf-tower
Feb-27 17:42:14.175 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye