go-plugin VS starlark-go

I am looking forward to a mix of both - I am hoping to add a concept called "operator" which would be a go-plugin [1], just like terraform providers, but build backends. So, someone would be able to, say, write a Slack plugin (in Go, or anything over RPC) which sends a message once a build is complete - like Jenkins/GitHub actions, or just scripts that we can reuse like GitLab CI through `modules`.

Perhaps a new registry where we can push custom modules and providers (operators in this case), I'm curious to know about if we have any existing implementations we could reuse for the registry.

[1]: https://github.com/hashicorp/go-plugin

I'd recommend looking into a different approach for plugins such as hashicorp/go-plugin (which uses multiple process PIDs and RPC communication between them) or traefik/yaegi (which implements a Go-compatible scripting language that can be interpreted at runtime and which still supports most Go modules).

You'll never fully protect your code from someone who's dead-set on reverse-engineering it, however, you can use https://github.com/hashicorp/go-plugin or a similar RPC technique, which will let you ship binary plugins and will also be less fragile and janky compared to something made with `-buildmode=plugin`.

The plugin system that hashicorp uses for all their projects works very well. It's essentially a local RPC implementation. https://github.com/hashicorp/go-plugin

can you try older go-plugin versions? The only major change in v1.4.4 was a bugfix for automtls. https://github.com/hashicorp/go-plugin/blob/master/CHANGELOG.md

Starlark in go https://github.com/google/starlark-go is a great way to combine the best of both, the ease of use of Python and the simplicity of go.

I have been building a platform for deploying internal web applications using this approach https://github.com/claceio/clace. Use Starlark to configure the application, the platform itself is built in go.

The big unknown is your task definition: what does user-defined logic look like? If you're expecting go code, that's gonna need some cleverness because of the compiled nature of it. There's a node runtime implemented in go if you want to provide sandboxed javascript (check the source of k6.io, it's the main one I know that uses it). If you want to provide building blocks and let them compose them, starlark might be a good choice.

Another one of those (with broader language support) is the Starlark language, which has a Go implementation: https://github.com/google/starlark-go

Every config format gets as complex to be touring complete in the end. We had similar problems and eventually got rid of that complexity and switched to starlark (the bazel config language), was a huge benefit for the tools. https://github.com/google/starlark-go "Starlark is a dialect of Python intended for use as a configuration language. Like Python, it is an untyped dynamic language with high-level data types, first-class functions with lexical scope, and garbage collection."

Direct link to the Go implementation of Starlark: https://github.com/google/starlark-go

For a Python-like syntax, https://github.com/google/starlark-go is the language used in Babel. It's very mature, but since it is used in a massive mature project with a specific purpose, it doesn't move fast or drift from the spec of its Java-based sibling. It doesn't have exception try except blocks or some other features you might expect, but for short extension logic, it might be exactly what you want with the stability you can depend upon.

Context: https://github.com/google/starlark-go/blob/cfacd890221418a2dc2c736f7b5e3476c38709b1/starlark/int_posix64.go

Thus envd is more like Dockerfile, while it uses a simplified python dialect starlark https://github.com/google/starlark-go as the build language.

Use Go (mostly for starlark-go)