Ask HN: A Better Docker Compose?

• Juju

  14 2,300 9.9 Go

  Orchestration engine that enables the deployment, integration and lifecycle management of applications at any scale, on any infrastructure (Kubernetes or otherwise).

  

https://juju.is/

Each app is packaged in a charm which seems to be a yaml declaring inputs, dependencies and other meta data and optional python code that can respond to certain lifecycle hooks

https://discourse.charmhub.io/t/implementing-relations/1051

   name: my-node-app

• dhall-lang

  113 4,129 6.0 Dhall

  Maintainable configuration files

  

At the risk of sounding clichè/unhelpful.

> “you can solve every problem by adding or removing an abstraction layer”

Have you considered generating your YAML/JSON config with something that composes?

If you are open to it you might be interested in dhall [1] as it’s a config language with variables, functions and imports.

I have used it for pet projects and I could see how it could offer some tidy encapsulation patterns for larger, more complicated production applications.

[1]: https://dhall-lang.org/

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• dagger

  92 10,190 9.9 Go

  Application Delivery as Code that Runs Anywhere (by dagger)

  

I find this really annoying. There's also no concept of a package like Helm Charts which means you can distribute all your images on Docker Hub but then when it comes to compose files and configurations with templating your up @$%^ creek.

I was hoping something like https://dagger.io/ might solve this but they currently don't support compose files, or maybe https://www.acorn.io/ ?

It does feel like there is a legitimate space for a multi-container service application standard that isn't just kubernetes, unless perhaps things like k3d and docker desktop end up so good at running tiny k8s that docker compose dies.

• ecs-blueprints

  2 217 7.8 Python

  Configure and deploy complete ECS solutions with Terraform or CDK

  

I’ve been spending a week trying to learn how to deploy a collection of containers (my web app, a Postgres DB, and some microservices) to AWS and I am still so lost.

The first solution I happened upon was serverless. Specifically SST, which is written with AWS CDK, but you must develop on live services and I just can’t justify paying to develop.

Then I found Serverless Framework, which is an abstraction on CloudFormation, but the offline solutions like localstack get a lot of flack for being buggy and localstack charges for some services. I also looked into Architect but the documentation is abysmal.

Then I figured serverful might be the easier way to go. I found that docker compose has a built in integration with AWS ECS where it transforms your yaml into Cloudformation to provision the right services. However, it seems to just be missing key parts like custom domain and SSL certificate provisioning which seems to defeat the IaC ethos.

Then I figured I might go with Terraform and I found some seemingly good starters like https://github.com/aws-ia/terraform-aws-ecs-blueprints https://github.com/cloudposse/terraform-aws-ecs-web-app https://github.com/turnerlabs/terraform-ecs-fargate but the examples are just lacking. They don’t have any examples for multiple containers that can access each others’ resources that I can find. Reading these templates has at least given me a better idea of the resources I need to provision in AWS but the networking and configuration still frighten me. Like do I need to configure nginx with a reverse proxy myself? How do I orchestrate that container with the others? And apparently services can crash and just not restart? And I need to make sure to configure volumes for data that needs to persist. And setting up the CI/CD seems daunting.

I’ve also heard about docker swarm, kubernetes, pulumi, AWS SAM, etc but it’s a lot to learn. When I go on Discords for web frameworks, mostly everyone including the devs of these frameworks use 2nd tier managed providers like Vercel, Fly, Netlify, Supabase, Cloudflare, etc. But many of those are just not as reliable as core cloud providers and the cost is way higher. Glad to see I’m not alone in a very reasonable expectation of a simple way to orchestrate multiple containers on AWS, what must be the most common use case web developers have

• terraform-aws-ecs-web-app

  2 222 3.1 HCL

  Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more.

  

I’ve been spending a week trying to learn how to deploy a collection of containers (my web app, a Postgres DB, and some microservices) to AWS and I am still so lost.

The first solution I happened upon was serverless. Specifically SST, which is written with AWS CDK, but you must develop on live services and I just can’t justify paying to develop.

Then I found Serverless Framework, which is an abstraction on CloudFormation, but the offline solutions like localstack get a lot of flack for being buggy and localstack charges for some services. I also looked into Architect but the documentation is abysmal.

Then I figured serverful might be the easier way to go. I found that docker compose has a built in integration with AWS ECS where it transforms your yaml into Cloudformation to provision the right services. However, it seems to just be missing key parts like custom domain and SSL certificate provisioning which seems to defeat the IaC ethos.

Then I figured I might go with Terraform and I found some seemingly good starters like https://github.com/aws-ia/terraform-aws-ecs-blueprints https://github.com/cloudposse/terraform-aws-ecs-web-app https://github.com/turnerlabs/terraform-ecs-fargate but the examples are just lacking. They don’t have any examples for multiple containers that can access each others’ resources that I can find. Reading these templates has at least given me a better idea of the resources I need to provision in AWS but the networking and configuration still frighten me. Like do I need to configure nginx with a reverse proxy myself? How do I orchestrate that container with the others? And apparently services can crash and just not restart? And I need to make sure to configure volumes for data that needs to persist. And setting up the CI/CD seems daunting.

I’ve also heard about docker swarm, kubernetes, pulumi, AWS SAM, etc but it’s a lot to learn. When I go on Discords for web frameworks, mostly everyone including the devs of these frameworks use 2nd tier managed providers like Vercel, Fly, Netlify, Supabase, Cloudflare, etc. But many of those are just not as reliable as core cloud providers and the cost is way higher. Glad to see I’m not alone in a very reasonable expectation of a simple way to orchestrate multiple containers on AWS, what must be the most common use case web developers have

• terraform-ecs-fargate

  1 416 5.5 HCL

  A Terraform template used for provisioning web application stacks on AWS ECS Fargate

  

I’ve been spending a week trying to learn how to deploy a collection of containers (my web app, a Postgres DB, and some microservices) to AWS and I am still so lost.

The first solution I happened upon was serverless. Specifically SST, which is written with AWS CDK, but you must develop on live services and I just can’t justify paying to develop.

Then I found Serverless Framework, which is an abstraction on CloudFormation, but the offline solutions like localstack get a lot of flack for being buggy and localstack charges for some services. I also looked into Architect but the documentation is abysmal.

Then I figured serverful might be the easier way to go. I found that docker compose has a built in integration with AWS ECS where it transforms your yaml into Cloudformation to provision the right services. However, it seems to just be missing key parts like custom domain and SSL certificate provisioning which seems to defeat the IaC ethos.

Then I figured I might go with Terraform and I found some seemingly good starters like https://github.com/aws-ia/terraform-aws-ecs-blueprints https://github.com/cloudposse/terraform-aws-ecs-web-app https://github.com/turnerlabs/terraform-ecs-fargate but the examples are just lacking. They don’t have any examples for multiple containers that can access each others’ resources that I can find. Reading these templates has at least given me a better idea of the resources I need to provision in AWS but the networking and configuration still frighten me. Like do I need to configure nginx with a reverse proxy myself? How do I orchestrate that container with the others? And apparently services can crash and just not restart? And I need to make sure to configure volumes for data that needs to persist. And setting up the CI/CD seems daunting.

I’ve also heard about docker swarm, kubernetes, pulumi, AWS SAM, etc but it’s a lot to learn. When I go on Discords for web frameworks, mostly everyone including the devs of these frameworks use 2nd tier managed providers like Vercel, Fly, Netlify, Supabase, Cloudflare, etc. But many of those are just not as reliable as core cloud providers and the cost is way higher. Glad to see I’m not alone in a very reasonable expectation of a simple way to orchestrate multiple containers on AWS, what must be the most common use case web developers have

• Docker Compose

  383 32,312 9.6 Go

  Define and run multi-container applications with Docker

  

> Docker compose is a dead end AFAIK ..

What? I'm not involved and don't follow closely but pretty sure it's about as dead as docker itself. I.e. not dead. There was commits 8hrs ago -- https://github.com/docker/compose/. Not sure who did that if not "the community".

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• supabase

  761 65,456 10.0 TypeScript

  The open source Firebase alternative.

  

It’s not a dead end at all. It serves a specific purpose - if you need to have a packaged deliverable that consists of several (3-8) containers, and you want your end users to be able to easily deploy that deliverable, there is no better solution that I am aware of over docker-compose.

I’m not sure why OP feels that way, but I’ll just give two examples of great FOSS projects that live on the bleeding edge that I work with every day that offer docker compose deliverables as their #1 way to get going in prod:

1. Supabase: https://github.com/supabase/supabase/blob/master/docker/dock...

• Netmaker

  165 8,928 9.6 Go

  Netmaker makes networks with WireGuard. Netmaker automates fast, secure, and distributed virtual networks.

  

• agenix

  10 1,217 7.5 Nix

  age-encrypted secrets for NixOS and Home manager

I don't have a write-up, just my code in git. But it's not public. I'm not using anything out of the ordinary - Nix containers, modules, and functions, and the Agenix module with uses a private key to decrypt secrets at start. The Nix language is inherently composable. Here are some links that explain:

Containers:

https://nixos.wiki/wiki/NixOS_Containers

Modules:

https://nixos.wiki/wiki/NixOS_modules

Functions:

https://www.reddit.com/r/NixOS/comments/zzstun/please_help_m...

Agenix:

https://github.com/ryantm/agenix

• arion-compose

  2 561 6.8 Nix

  Run docker-compose with help from Nix/NixOS

  

• polycrate

  1 17 7.0 Go

  Polycrate is a framework that lets you package, integrate and automate complex applications and infrastructure.

  

https://github.com/polycrate/polycrate/blob/main/LICENSE.md#...

I think it's the new "JS framework syndrome" for everyone to invent their own license :-/

Suggest a related project