Show HN: Togomak – declarative pipeline orchestrator based on HCL and Terraform

• just

  165 17,053 9.1 Rust

  🤖 Just a command runner

One primary design goal togomak had from the beginning was concurrency. All tasks run concurrently, unless a `depends_on` argument is mentioned. `just` didn't support that when I was initially building togomak, but there is a feature coming in soon which I am looking forward to: https://github.com/casey/just/pull/1562 .

While I was building togomak, I read through Dagger [1], Earthly [2], Concourse CI [3], Jest and Make along with the stuff I was already working with - Jenkins, GitHub actions and GitLab CI. Dagger [1] is really great, I like its design - it supports writing pipelines in Python, Typescript, Go and a few more languages. togomak tries to abstract away a lot of it. Such as dependency management (in the case of python, the requirement of a python interpreter, and its package managers, etc). togomak is just a single statically-linked binary.

[1]: https://dagger.io/

• dagger

  92 10,228 9.9 Go

  Application Delivery as Code that Runs Anywhere (by dagger)

  

One primary design goal togomak had from the beginning was concurrency. All tasks run concurrently, unless a `depends_on` argument is mentioned. `just` didn't support that when I was initially building togomak, but there is a feature coming in soon which I am looking forward to: https://github.com/casey/just/pull/1562 .

While I was building togomak, I read through Dagger [1], Earthly [2], Concourse CI [3], Jest and Make along with the stuff I was already working with - Jenkins, GitHub actions and GitLab CI. Dagger [1] is really great, I like its design - it supports writing pipelines in Python, Typescript, Go and a few more languages. togomak tries to abstract away a lot of it. Such as dependency management (in the case of python, the requirement of a python interpreter, and its package managers, etc). togomak is just a single statically-linked binary.

[1]: https://dagger.io/

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• togomak

  1 141 9.2 Go

  A declarative pipeline orchestrator with the magic of HCL as a configuration language, inspired from Terraform's architecture.

• act

  145 50,182 9.2 Go

  Run your GitHub Actions locally 🚀

  

Yes, I have been taking a look at https://github.com/nektos/act which allows running github actions workflow locally. I am investigating if I can integrate it directly in togomak, so that users can reuse existing github action in their pipeline. For the most bit, these pipelines run in node docker containers. I'm hoping to add a concept called "operator" where you can specify or use multiple build backends like Google Cloud Build, or GitHub Actions.

• go-plugin

  30 4,960 6.7 Go

  Golang plugin system over RPC.

  

I am looking forward to a mix of both - I am hoping to add a concept called "operator" which would be a go-plugin [1], just like terraform providers, but build backends. So, someone would be able to, say, write a Slack plugin (in Go, or anything over RPC) which sends a message once a build is complete - like Jenkins/GitHub actions, or just scripts that we can reuse like GitLab CI through `modules`.

Perhaps a new registry where we can push custom modules and providers (operators in this case), I'm curious to know about if we have any existing implementations we could reuse for the registry.

[1]: https://github.com/hashicorp/go-plugin

• Concourse

  47 7,172 9.0 Go

  Concourse is a container-based continuous thing-doer written in Go.

  

• hcl

  40 5,060 8.2 Go

  HCL is the HashiCorp configuration language.

  

HCL has a JSON representation [1], internally, objects behave that way. so it should be possible to write a Jsonnet wrapper around it. Terraform can currently parse json pipelines too.

[1]: https://github.com/hashicorp/hcl/blob/main/json/spec.md

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• Pulumi

  178 19,705 9.9 Go

  Pulumi - Infrastructure as Code in any programming language. Build infrastructure intuitively on any cloud using familiar languages 🚀

  

Would it make sense to say Dagger is to Pulumi [1], as Terraform is to Togomak?

[1]: https://www.pulumi.com/

• bcl

  7 10 8.9 Go

  Basic Configuration Language

I agree with you that HCL sucks when it comes to variables.

Other thing that is funny: no user-defined functions, being unable to use function calls in string interpolations, but allowing variables... so it is like saying: we have this parser and at some points it allows expressions, at some other point not. This seems wrong.

At the same time I agree or at least understand original author's intent to squeeze HCL to maxinum. There is something appealing in HCL visual form, at least when defining resources. Maybe it's just (almost) simplest form of defining such structures that can exists.

This is why I started to work on my own format for configuration, visually similar but with different model of evaluation.

Here is the first attempt: https://github.com/wkhere/bcl

Disclaimer: I named it BCL, 'B' stands for Basic, to somehow relate to HCL and make it easily pronounced. But later I discovered that another BCL is used as Google to configure the Borg platform and seems to be massively hated ;) So I look for the better name..

• nickel

  46 2,137 9.5 Rust

  Better configuration for less

  

Also look at nickel which is an evolution of nix. It's my favorite in this space.

nickel-lang.org

https://github.com/tweag/nickel

Suggest a related project