Cgpipe Alternatives
Similar projects and alternatives to cgpipe
-
Kedro
Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
common-workflow-language
Repository for the CWL standards. Use https://cwl.discourse.group/ for support 😊
cgpipe reviews and mentions
-
Nextflow: Data-Driven Computational Pipelines
I do too.. and have similar opinions. I wrote my own tool years back for pipelines because it was always frustrating (started roughly around the same time as Nextflow).
Allowing for files to be marked as transient (temp) and re-running from arbitrary time points are definitely one of the things I support... as is conditional logic within the pipeline for job definition and resource usage. For me though, one of the biggest things is that I like having composable pipelines, so each part of the larger workflow can be developed independently. They can interact with each other (DAG) and use existing dependencies, but they don't have to exist in the same document/script. I work on large WGS datasets, so 1000's of jobs per patient isn't uncommon.
Happy to talk more if you're interested.
https://github.com/compgen-io/cgpipe
(And yes, you can dry run the entire thing. It will write out a bash script if you want to see exactly what is going to run without submitting jobs.)
Stats
compgen-io/cgpipe is an open source project licensed under BSD 3-clause "New" or "Revised" License which is an OSI approved license.
The primary programming language of cgpipe is Java.
Popular Comparisons
Sponsored