omic/digital-trial
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Digital Trials pipeline configured for WES

Browse Source 
Source-only snapshot of the cluster branch for WES execution. Large
reference files (HPA/MANE/ensemble FASTA, model weights, ~597 MB) are
omitted: they are baked into the container images at build time and
mounted from the dreamdock-data PVC at runtime, and exceed the Gitea
request size limit.

Pipeline entry point is main.nf, which orchestrates the biotransformer,
conplex and tissue modules as a single workflow. Ligand inputs are read
from the eureka workspace; protein_zarr and chembl_db come from the
dreamdock-data PVC.
This commit is contained in:
Olamide Isreal
2026-07-27 21:59:52 +01:00
commit 9e75f44f1a
86 changed files with 10142 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

96
.nextflow.log.8 Normal file
View File

@@ -0,0 +1,96 @@
Feb-20 12:45:31.438 [main] DEBUG nextflow.cli.Launcher - $> nextflow run test.nf -params-file /data/dreamdock-data/gpr55-caline/digtrial/gpr55-2.json
Feb-20 12:45:31.816 [main] DEBUG nextflow.cli.CmdRun - N E X T F L O W ~ version 25.04.3
Feb-20 12:45:31.845 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/omic/.nextflow/plugins; core-plugins: nf-amazon@2.15.0,nf-azure@1.16.0,nf-cloudcache@0.4.3,nf-codecommit@0.2.3,nf-console@1.2.1,nf-google@1.21.0,nf-k8s@1.0.0,nf-tower@1.11.3,nf-wave@1.12.1
Feb-20 12:45:31.879 [main] INFO o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Feb-20 12:45:31.880 [main] INFO o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Feb-20 12:45:31.884 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.12.0 in 'deployment' mode
Feb-20 12:45:31.897 [main] INFO org.pf4j.AbstractPluginManager - No plugins
Feb-20 12:45:31.922 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: /data/bugra/digital_trials/nextflow.config
Feb-20 12:45:31.925 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /data/bugra/digital_trials/nextflow.config
Feb-20 12:45:31.961 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /home/omic/.nextflow/secrets/store.json
Feb-20 12:45:31.966 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@6ef81f31] - activable => nextflow.secret.LocalSecretsProvider@6ef81f31
Feb-20 12:45:31.995 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Feb-20 12:45:32.749 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declaration
Feb-20 12:45:32.771 [main] DEBUG nextflow.cli.CmdRun - Launching `test.nf` [mad_jepsen] DSL2 - revision: 5cb3581ad6
Feb-20 12:45:32.773 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Feb-20 12:45:32.775 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[nf-tower@1.11.3]
Feb-20 12:45:32.777 [main] DEBUG nextflow.plugin.PluginUpdater - Installing plugin nf-tower version: 1.11.3
Feb-20 12:45:32.790 [main] INFO org.pf4j.AbstractPluginManager - Plugin 'nf-tower@1.11.3' resolved
Feb-20 12:45:32.790 [main] INFO org.pf4j.AbstractPluginManager - Start plugin 'nf-tower@1.11.3'
Feb-20 12:45:32.808 [main] DEBUG nextflow.plugin.BasePlugin - Plugin started nf-tower@1.11.3
Feb-20 12:45:32.881 [main] DEBUG nextflow.Session - Session UUID: 9b7d6ce5-46a6-43c0-accf-fd12d7d33e5d
Feb-20 12:45:32.882 [main] DEBUG nextflow.Session - Run name: mad_jepsen
Feb-20 12:45:32.883 [main] DEBUG nextflow.Session - Executor pool size: 80
Feb-20 12:45:32.896 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Feb-20 12:45:32.905 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=240; workQueue=LinkedBlockingQueue[-1]; allowCoreThreadTimeout=false
Feb-20 12:45:32.940 [main] DEBUG nextflow.cli.CmdRun -
Version: 25.04.3 build 5949
Created: 02-06-2025 20:56 UTC
System: Linux 6.11.0-26-generic
Runtime: Groovy 4.0.26 on OpenJDK 64-Bit Server VM 21.0.7+6-Ubuntu-0ubuntu124.04
Encoding: UTF-8 (UTF-8)
Process: 851238@k8s-node23 [127.0.1.1]
CPUs: 80 - Mem: 251.6 GB (56.5 GB) - Swap: 0 (0)
Feb-20 12:45:32.977 [main] DEBUG nextflow.Session - Work-dir: /data/bugra/digital_trials/work [btrfs]
Feb-20 12:45:32.978 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /data/bugra/digital_trials/bin
Feb-20 12:45:32.997 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Feb-20 12:45:33.016 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Feb-20 12:45:33.022 [main] DEBUG nextflow.Session - Observer factory: TowerFactory
Feb-20 12:45:33.025 [main] DEBUG nextflow.Session - Observer factory (v2): LinObserverFactory
Feb-20 12:45:33.059 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Feb-20 12:45:33.073 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 81; maxThreads: 1000
Feb-20 12:45:33.221 [main] DEBUG nextflow.Session - Session start
Feb-20 12:45:33.684 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Feb-20 12:45:34.592 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-20 12:45:34.592 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-20 12:45:34.600 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Feb-20 12:45:34.611 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=80; memory=251.6 GB; capacity=80; pollInterval=100ms; dumpInterval=5m
Feb-20 12:45:34.614 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
Feb-20 12:45:34.649 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'HUMAN_TRANSFORMER': maxForks=0; fair=false; array=0
Feb-20 12:45:34.755 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-20 12:45:34.755 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-20 12:45:34.757 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'GET_FINAL_METABOLITES_STATIC': maxForks=0; fair=false; array=0
Feb-20 12:45:34.772 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:gpu_process` matches labels `gpu_process` for process with name PREPROCESS_PROTEIN
Feb-20 12:45:34.778 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-20 12:45:34.779 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-20 12:45:34.781 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'PREPROCESS_PROTEIN': maxForks=0; fair=false; array=0
Feb-20 12:45:34.850 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-20 12:45:34.850 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-20 12:45:34.851 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'CONPLEX_ALL': maxForks=0; fair=false; array=0
Feb-20 12:45:34.887 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-20 12:45:34.887 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-20 12:45:34.888 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'NETWORK_ENRICHMENT': maxForks=0; fair=false; array=0
Feb-20 12:45:34.897 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-20 12:45:34.897 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-20 12:45:34.899 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'TISSUE_DISTRIBUTION': maxForks=0; fair=false; array=0
Feb-20 12:45:34.927 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Feb-20 12:45:34.927 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Feb-20 12:45:34.928 [main] DEBUG nextflow.processor.TaskProcessor - Creating process 'BIO_METRICS': maxForks=0; fair=false; array=0
Feb-20 12:45:34.933 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: HUMAN_TRANSFORMER, METABOLITES_BY_MASS, GET_FINAL_METABOLITES_STATIC, CONPLEX_ALL, NETWORK_ENRICHMENT, GET_FINAL_METABOLITES, CONPLEX, BIO_METRICS, SUPER_TRANSFORMER, TISSUE_DISTRIBUTION, MERGE_DRUG, MERGE_INTERACTIONS, PREPROCESS_PROTEIN, ORDERED_SEQUENCE
Feb-20 12:45:34.935 [main] DEBUG nextflow.Session - Igniting dataflow network (11)
Feb-20 12:45:34.942 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > HUMAN_TRANSFORMER
Feb-20 12:45:34.944 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > GET_FINAL_METABOLITES_STATIC
Feb-20 12:45:34.945 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > PREPROCESS_PROTEIN
Feb-20 12:45:34.946 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > CONPLEX_ALL
Feb-20 12:45:34.946 [PathVisitor-3] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /mnt/dreamdock-data/digital_trials/input/blank/; pattern: *.fasta; options: [:]
Feb-20 12:45:34.946 [PathVisitor-1] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /data/dreamdock-data/gpr55-caline/digtrial/input/ligands2/; pattern: *.csv; options: [:]
Feb-20 12:45:34.947 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > NETWORK_ENRICHMENT
Feb-20 12:45:34.947 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > TISSUE_DISTRIBUTION
Feb-20 12:45:34.948 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BIO_METRICS
Feb-20 12:45:34.949 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
Script_c432c676a3e7c0cb: /data/bugra/digital_trials/test.nf
Script_e39e54e7aa543fd0: /data/bugra/digital_trials/main_biotransformer.nf
Script_d345b7a65a8f3255: /data/bugra/digital_trials/main_conplex.nf
Script_ec01f2a38db15aa5: /data/bugra/digital_trials/main_tissue.nf
Feb-20 12:45:34.949 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination
Feb-20 12:45:34.949 [main] DEBUG nextflow.Session - Session await
Feb-20 12:45:34.951 [PathVisitor-1] DEBUG nextflow.file.PathVisitor - No such file or directory: /data/dreamdock-data/gpr55-caline/digtrial/input/ligands2/ -- Skipping visit
Feb-20 12:45:34.966 [main] DEBUG nextflow.Session - Session await > all processes finished
Feb-20 12:45:35.028 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Feb-20 12:45:35.028 [main] DEBUG nextflow.Session - Session await > all barriers passed
Feb-20 12:45:35.037 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=0; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=0ms; failedDuration=0ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=0; peakCpus=0; peakMemory=0; ]
Feb-20 12:45:35.083 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Feb-20 12:45:35.175 [main] INFO org.pf4j.AbstractPluginManager - Stop plugin 'nf-tower@1.11.3'
Feb-20 12:45:35.175 [main] DEBUG nextflow.plugin.BasePlugin - Plugin stopped nf-tower
Feb-20 12:45:35.179 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'FileTransfer' shutdown completed (hard=false)
Feb-20 12:45:35.180 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye