omic/digital-trial
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Digital Trials pipeline configured for WES

Browse Source 
Source-only snapshot of the cluster branch for WES execution. Large
reference files (HPA/MANE/ensemble FASTA, model weights, ~597 MB) are
omitted: they are baked into the container images at build time and
mounted from the dreamdock-data PVC at runtime, and exceed the Gitea
request size limit.

Pipeline entry point is main.nf, which orchestrates the biotransformer,
conplex and tissue modules as a single workflow. Ligand inputs are read
from the eureka workspace; protein_zarr and chembl_db come from the
dreamdock-data PVC.
This commit is contained in:
Olamide Isreal
2026-07-27 21:59:52 +01:00
commit 9e75f44f1a
86 changed files with 10142 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

79
.nextflow.log Normal file
View File

@@ -0,0 +1,79 @@
Apr-03 17:30:46.267 [main] DEBUG nextflow.cli.Launcher - $> nextflow run test_string_only.nf -params-file /data/runs/R01-caline-nac_similar/J15-string/params.json
Apr-03 17:30:46.687 [main] DEBUG nextflow.cli.CmdRun - N E X T F L O W ~ version 25.04.3
Apr-03 17:30:46.758 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/omic/.nextflow/plugins; core-plugins: nf-amazon@2.15.0,nf-azure@1.16.0,nf-cloudcache@0.4.3,nf-codecommit@0.2.3,nf-console@1.2.1,nf-google@1.21.0,nf-k8s@1.0.0,nf-tower@1.11.3,nf-wave@1.12.1
Apr-03 17:30:46.876 [main] INFO o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Apr-03 17:30:46.878 [main] INFO o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Apr-03 17:30:46.882 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.12.0 in 'deployment' mode
Apr-03 17:30:46.898 [main] INFO org.pf4j.AbstractPluginManager - No plugins
Apr-03 17:30:46.926 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: /data/bugra/digital_trials/nextflow.config
Apr-03 17:30:46.930 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /data/bugra/digital_trials/nextflow.config
Apr-03 17:30:46.985 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /home/omic/.nextflow/secrets/store.json
Apr-03 17:30:46.991 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@6ef81f31] - activable => nextflow.secret.LocalSecretsProvider@6ef81f31
Apr-03 17:30:47.022 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Apr-03 17:30:47.826 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declaration
Apr-03 17:30:47.850 [main] DEBUG nextflow.cli.CmdRun - Launching `test_string_only.nf` [deadly_faggin] DSL2 - revision: 8e3db5067a
Apr-03 17:30:47.852 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Apr-03 17:30:47.853 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[nf-tower@1.11.3]
Apr-03 17:30:47.855 [main] DEBUG nextflow.plugin.PluginUpdater - Installing plugin nf-tower version: 1.11.3
Apr-03 17:30:47.877 [main] INFO org.pf4j.AbstractPluginManager - Plugin 'nf-tower@1.11.3' resolved
Apr-03 17:30:47.877 [main] INFO org.pf4j.AbstractPluginManager - Start plugin 'nf-tower@1.11.3'
Apr-03 17:30:47.962 [main] DEBUG nextflow.plugin.BasePlugin - Plugin started nf-tower@1.11.3
Apr-03 17:30:48.062 [main] DEBUG nextflow.Session - Session UUID: 4c59e728-8373-423d-b172-5781faf18d0f
Apr-03 17:30:48.062 [main] DEBUG nextflow.Session - Run name: deadly_faggin
Apr-03 17:30:48.063 [main] DEBUG nextflow.Session - Executor pool size: 80
Apr-03 17:30:48.073 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Apr-03 17:30:48.081 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=240; workQueue=LinkedBlockingQueue[-1]; allowCoreThreadTimeout=false
Apr-03 17:30:48.208 [main] DEBUG nextflow.cli.CmdRun -
Version: 25.04.3 build 5949
Created: 02-06-2025 20:56 UTC
System: Linux 6.11.0-26-generic
Runtime: Groovy 4.0.26 on OpenJDK 64-Bit Server VM 21.0.7+6-Ubuntu-0ubuntu124.04
Encoding: UTF-8 (UTF-8)
Process: 2896370@k8s-node23 [127.0.1.1]
CPUs: 80 - Mem: 251.6 GB (184.8 GB) - Swap: 0 (0)
Apr-03 17:30:48.264 [main] DEBUG nextflow.Session - Work-dir: /data/bugra/digital_trials/work [btrfs]
Apr-03 17:30:48.265 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /data/bugra/digital_trials/bin
Apr-03 17:30:48.294 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Apr-03 17:30:48.318 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Apr-03 17:30:48.323 [main] DEBUG nextflow.Session - Observer factory: TowerFactory
Apr-03 17:30:48.325 [main] DEBUG nextflow.Session - Observer factory (v2): LinObserverFactory
Apr-03 17:30:48.355 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Apr-03 17:30:48.372 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 81; maxThreads: 1000
Apr-03 17:30:48.619 [main] DEBUG nextflow.Session - Session start
Apr-03 17:30:48.966 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Apr-03 17:30:49.582 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-03 17:30:49.583 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-03 17:30:49.590 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Apr-03 17:30:49.601 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=80; memory=251.6 GB; capacity=80; pollInterval=100ms; dumpInterval=5m
Apr-03 17:30:49.604 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
Apr-03 17:30:49.637 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'NETWORK_ENRICHMENT': maxForks=0; fair=false; array=0
Apr-03 17:30:49.680 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: CONPLEX, MERGE_DRUG, MERGE_INTERACTIONS, NETWORK_ENRICHMENT, PREPROCESS_PROTEIN
Apr-03 17:30:49.682 [main] DEBUG nextflow.Session - Igniting dataflow network (2)
Apr-03 17:30:49.684 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > NETWORK_ENRICHMENT
Apr-03 17:30:49.685 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
Script_98e4d45ae72cf7d4: /data/bugra/digital_trials/test_string_only.nf
Script_158116c0b930c4dd: /data/bugra/digital_trials/main_conplex.nf
Apr-03 17:30:49.685 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination
Apr-03 17:30:49.685 [main] DEBUG nextflow.Session - Session await
Apr-03 17:30:49.974 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-03 17:30:49.979 [Task submitter] INFO nextflow.Session - [eb/7f494f] Submitted process > NETWORK_ENRICHMENT (1)
Apr-03 17:30:49.996 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-03 17:30:49.997 [Task submitter] INFO nextflow.Session - [e9/27e18e] Submitted process > NETWORK_ENRICHMENT (2)
Apr-03 17:30:50.004 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-03 17:30:50.004 [Task submitter] INFO nextflow.Session - [d2/7834f1] Submitted process > NETWORK_ENRICHMENT (3)
Apr-03 17:31:00.726 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: NETWORK_ENRICHMENT (3); status: COMPLETED; exit: 0; error: -; workDir: /data/bugra/digital_trials/work/d2/7834f119d9556ee05e00daf44d874b]
Apr-03 17:31:00.728 [Task monitor] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'TaskFinalizer' minSize=10; maxSize=240; workQueue=LinkedBlockingQueue[-1]; allowCoreThreadTimeout=false
Apr-03 17:31:00.850 [TaskFinalizer-1] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'PublishDir' minSize=10; maxSize=240; workQueue=LinkedBlockingQueue[-1]; allowCoreThreadTimeout=false
Apr-03 17:31:02.158 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: NETWORK_ENRICHMENT (2); status: COMPLETED; exit: 0; error: -; workDir: /data/bugra/digital_trials/work/e9/27e18e08e53e0c4d45c5407deebea2]
Apr-03 17:31:02.185 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: NETWORK_ENRICHMENT (1); status: COMPLETED; exit: 0; error: -; workDir: /data/bugra/digital_trials/work/eb/7f494fae594fe3be1f36cd54af9382]
Apr-03 17:31:02.198 [main] DEBUG nextflow.Session - Session await > all processes finished
Apr-03 17:31:02.288 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Apr-03 17:31:02.288 [main] DEBUG nextflow.Session - Session await > all barriers passed
Apr-03 17:31:02.291 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'TaskFinalizer' shutdown completed (hard=false)
Apr-03 17:31:02.292 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'PublishDir' shutdown completed (hard=false)
Apr-03 17:31:02.307 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=3; failedCount=0; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=35s; failedDuration=0ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=3; peakCpus=3; peakMemory=3 GB; ]
Apr-03 17:31:02.512 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Apr-03 17:31:02.595 [main] INFO org.pf4j.AbstractPluginManager - Stop plugin 'nf-tower@1.11.3'
Apr-03 17:31:02.595 [main] DEBUG nextflow.plugin.BasePlugin - Plugin stopped nf-tower
Apr-03 17:31:02.597 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'FileTransfer' shutdown completed (hard=false)
Apr-03 17:31:02.597 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye