Our great sponsors
-
Ansible
Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy and maintain. Automate everything from code deployment to network configuration to cloud management, in a language that approaches plain English, using SSH, with no agents to install on remote systems. https://docs.ansible.com.
-
core
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web. (by vuejs)
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
architecture_decision_record
Architecture decision record (ADR) examples for software planning, IT leadership, and template documentation
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
rafiki
An open-source, comprehensive Interledger service for wallet providers, enabling them to provide Interledger functionality to their users.
Overall, I'm tempted to agree, whilst keeping in mind that you sometimes definitely need a little bit of novelty, which the author brings up at the end of the article as well. Here's a few bits of my personal experience, where some novelty helped greatly in these past few years.
Personally, I'd say that something like 12 Factor apps are a good and supposedly new approach (using mechanisms and approaches that have been around for a while) to let you look at software written in different languages pretty much the same from the outside. For example, you use environment variables for configuration, write lots to STDOUT, don't cripple your own horizontal scalability by always reaching for local storage (e.g. when S3 might be better suited) or local application memory (e.g. when Redis might be a good idea). It's nice to have those sorts of suggestions in one place and all of the sudden you escape XML or Tomcat setup hell, and can look at Python apps and Java apps similarly from the outside: https://12factor.net/
Similarly, adopting containers has been a good solution, both because it allows achieving what people historically didn't bother with when having the opportunity of using systemd, but also because all of the sudden your applications are like phone apps - that can be launched in a consistent format on any server that you need. And you get health checks, resource limits, automatic restarts, bind mounts, port mapping and internal DNS, all of which you will never build in environments where the DevOps knowledge or resources (time) are not there.
Note: Kubernetes might be too complex for some setups, as HN loves to point you, something like Nomad or even Docker Swarm also still exists and works: https://docs.docker.com/engine/swarm/ (just linking this in particular, because it's just a small step up from Docker Compose, the pinnacle of simplicity)
Speaking of which, infrastructure as code is great! Using something like Ansible is definitely a novel thing to do at first, but cutting off my team's write access to the servers and making them use GitOps with code review for the configuration changes has been a solid idea. No more wondering why some random configuration exists, or why it was changed N years ago, now you can just look at Git. No more fat fingering bad changes, and even if you did something like there's also code review. No more risks like Knight Capital of partial deploys and if something like that were to happen, you'd get a CI notification about what's wrong. Just describe what you need on the server and let those hundreds of actions execute every morning (or after every commit/merge) automatically, ensuring a mostly consistent state - and way more lazily than learning Nix/Guix: https://www.ansible.com/
Furthermore, adopting the "ingress pattern" where all of your apps are in some internal overlay network, but talk to the outside world through instances of Apache/Nginx/Caddy/Traefik is brilliant! No more wondering about how to set up SSL certificates in each of the different application runtimes or even framework versions. No more worrying about setting up rate limits for each application individually, no more worrying about context paths for how things are deployed - you can configure all of that in your web server, even if you don't use a Kubernetes Ingress controller.
Oh, and forget something like jQuery from the old days, especially when you'd integrate with numerous low quality plugins that wouldn't even work that well half of the time. Just use something like Vue with PrimeVue/PrimeFlex or any other premade component library, with Pinia for state management. You avoid the trouble of using React with Redux (though React Query is nice) or the complexity of Angular, while still getting the benefits of writing small, mostly self-contained application components. No more thousand line JavaScript controllers, no more messing around with global state, or god forbid using something like AngularJS. And with JSX, it actually ends up feeling more convenient to write front end code, in addition to Vue getting hooks right, better than React IMO: https://vuejs.org/
But the actual applications? Most of the time, they should be more boring. Using Java? Just go for Spring Boot or Dropwizard; something like Quarkus or Vert.X are nice, but not quite ready yet. Using Node? Look at Express.js, it does everything you need. Python? Django or Flask. PHP? Laravel or Symfony. Ruby? Rails. Every time I've seen someone go for a non-standard solution or actually writing their own framework, it's been an utter dumpsterfire. Good luck debugging some uncommented code that has not enough tests when there's a production outage and the few code comments that you might stumble upon are in Lithuanian or something.
Databases? As a rule of thumb, go for PostgreSQL or MariaDB/MySQL. If you need data storage, use something S3 compatible. If you need key-value storage, use Redis. If you need document storage, either store JSON in your RDBMS or cautiously go for MongoDB. In each of these spaces, there are one or two established options and using anything outside of those should only be done when you have person-years to throw at every problem, e.g. the expertise and the $$$ to innovate.
In most circumstances, just use what has worked for other people well, as long as their circumstances are similar to yours. (a bit long winded, but just felt like writing a bit today)
Overall, I'm tempted to agree, whilst keeping in mind that you sometimes definitely need a little bit of novelty, which the author brings up at the end of the article as well. Here's a few bits of my personal experience, where some novelty helped greatly in these past few years.
Personally, I'd say that something like 12 Factor apps are a good and supposedly new approach (using mechanisms and approaches that have been around for a while) to let you look at software written in different languages pretty much the same from the outside. For example, you use environment variables for configuration, write lots to STDOUT, don't cripple your own horizontal scalability by always reaching for local storage (e.g. when S3 might be better suited) or local application memory (e.g. when Redis might be a good idea). It's nice to have those sorts of suggestions in one place and all of the sudden you escape XML or Tomcat setup hell, and can look at Python apps and Java apps similarly from the outside: https://12factor.net/
Similarly, adopting containers has been a good solution, both because it allows achieving what people historically didn't bother with when having the opportunity of using systemd, but also because all of the sudden your applications are like phone apps - that can be launched in a consistent format on any server that you need. And you get health checks, resource limits, automatic restarts, bind mounts, port mapping and internal DNS, all of which you will never build in environments where the DevOps knowledge or resources (time) are not there.
Note: Kubernetes might be too complex for some setups, as HN loves to point you, something like Nomad or even Docker Swarm also still exists and works: https://docs.docker.com/engine/swarm/ (just linking this in particular, because it's just a small step up from Docker Compose, the pinnacle of simplicity)
Speaking of which, infrastructure as code is great! Using something like Ansible is definitely a novel thing to do at first, but cutting off my team's write access to the servers and making them use GitOps with code review for the configuration changes has been a solid idea. No more wondering why some random configuration exists, or why it was changed N years ago, now you can just look at Git. No more fat fingering bad changes, and even if you did something like there's also code review. No more risks like Knight Capital of partial deploys and if something like that were to happen, you'd get a CI notification about what's wrong. Just describe what you need on the server and let those hundreds of actions execute every morning (or after every commit/merge) automatically, ensuring a mostly consistent state - and way more lazily than learning Nix/Guix: https://www.ansible.com/
Furthermore, adopting the "ingress pattern" where all of your apps are in some internal overlay network, but talk to the outside world through instances of Apache/Nginx/Caddy/Traefik is brilliant! No more wondering about how to set up SSL certificates in each of the different application runtimes or even framework versions. No more worrying about setting up rate limits for each application individually, no more worrying about context paths for how things are deployed - you can configure all of that in your web server, even if you don't use a Kubernetes Ingress controller.
Oh, and forget something like jQuery from the old days, especially when you'd integrate with numerous low quality plugins that wouldn't even work that well half of the time. Just use something like Vue with PrimeVue/PrimeFlex or any other premade component library, with Pinia for state management. You avoid the trouble of using React with Redux (though React Query is nice) or the complexity of Angular, while still getting the benefits of writing small, mostly self-contained application components. No more thousand line JavaScript controllers, no more messing around with global state, or god forbid using something like AngularJS. And with JSX, it actually ends up feeling more convenient to write front end code, in addition to Vue getting hooks right, better than React IMO: https://vuejs.org/
But the actual applications? Most of the time, they should be more boring. Using Java? Just go for Spring Boot or Dropwizard; something like Quarkus or Vert.X are nice, but not quite ready yet. Using Node? Look at Express.js, it does everything you need. Python? Django or Flask. PHP? Laravel or Symfony. Ruby? Rails. Every time I've seen someone go for a non-standard solution or actually writing their own framework, it's been an utter dumpsterfire. Good luck debugging some uncommented code that has not enough tests when there's a production outage and the few code comments that you might stumble upon are in Lithuanian or something.
Databases? As a rule of thumb, go for PostgreSQL or MariaDB/MySQL. If you need data storage, use something S3 compatible. If you need key-value storage, use Redis. If you need document storage, either store JSON in your RDBMS or cautiously go for MongoDB. In each of these spaces, there are one or two established options and using anything outside of those should only be done when you have person-years to throw at every problem, e.g. the expertise and the $$$ to innovate.
In most circumstances, just use what has worked for other people well, as long as their circumstances are similar to yours. (a bit long winded, but just felt like writing a bit today)
> I've encountered a code written in the 12factor style of using environment variables for configuration, and in that particular case there was no validation nor documentation of the configuration options. Is this typical?
This just feels like bad development and isn't unlike being given a random .properties/.ini file with no explanations of what the values mean. Sounds like someone didn't do their job, or the processes to encourage (require) them to do so weren't in place.
> For onboarding new members, I would have thought it preferable to have a JSON configuration, where both documentation and validation of configuration options are provided by a JSON Schema file.
You know, this can work, but then you need your applications to be able to read that file and feeding it in through your container management solution (which many use for a variety of reasons) wouldn't be as easy. Even without containers, you'd still need to look out so you don't end up with 20 JSON files all of which might need to be changed for a new environment.
Honestly, something like JSON5 https://json5.org/ was pretty cool because of added comments, but otherwise JSON is a bit cumbersome to use. That said, some web servers like Caddy have gone for accepting JSON configuration as well, which lends itself nicely to automation, so it's still a valid approach: https://caddyserver.com/docs/json/
> I've encountered a code written in the 12factor style of using environment variables for configuration, and in that particular case there was no validation nor documentation of the configuration options. Is this typical?
I don't know about typical, it comes down to how your team values the code they write.
You can have a .env.example file commit to version control which explains every option in as much or as little detail as you'd like. For my own personal projects, I tend to document this file like this https://github.com/nickjj/docker-flask-example/blob/main/.en....
> Also everything was new at one time.
Hah, this is a good point, but in my eyes lots of things that were new... never really grew up and were just deprecated and died.
For example, if someone based their setup on IronFunctions, they might have run into a bit of a painful situation, seeing as the project has been largely abandoned: https://github.com/iron-io/functions
Same for a database solution like Clusterpoint, the support for which just ended and you were left to migrate away to something else: https://github.com/clusterpoint
Ergo, I'd say that it's good for others to suffer the consequences of being trend setters and making wild bets on new and risky products and to just reap the benefits of their efforts later yourself, when things are safer. If a project has survived for a reasonably long time, it's a good indicator that it'll probably keep surviving in the future as well (there was a name for this, sadly can't recall what that was).
The article mentions Architectural Design Records (ADR) which can be included as a folder in the git repo for the project, as a means of documenting the historical decisions that led to the project's current structure. Some of that seems overly complex or formalized (i.e. reinventing UML and all the problems that came with it), but having that history in some kind of constant format would likely help overcome any novelty issues and help people grasp what's going on. The simplest format discussed seems the best for most cases:
https://github.com/joelparkerhenderson/architecture-decision...
The article mentions Architectural Design Records (ADR) which can be included as a folder in the git repo for the project, as a means of documenting the historical decisions that led to the project's current structure. Some of that seems overly complex or formalized (i.e. reinventing UML and all the problems that came with it), but having that history in some kind of constant format would likely help overcome any novelty issues and help people grasp what's going on. The simplest format discussed seems the best for most cases:
https://github.com/joelparkerhenderson/architecture-decision...
I think I grossly oversold this thing because there's a lot of comments here asking for something.
I don't really have this concept written down anywhere like a number of other ideas I have. But, I guess the short version is, if I had to make an elevator pitch or something: No framework is a configuration (maybe "distro" in the linux sense) of concepts (maybe "packages" in the software sense). A concept is either something you might use a framework or library for (and usually it exists somewhere), or it is something you would want a linter to find, and it might even be something that you want to ensure was done correctly at code review. I think this last one is the most accurate idea of what a "concept" is.
Over time I have accumulated a small informal set of "packages" that can be implemented without a helper library in nearly the same amount of code as if you were to use that library anyway. The important part is that the running software doesn't depend on the third party code, but actually the developers depend on a rule book and anything that violates the rules should be treated the same as calling an third party package's API method that doesn't exist. In other words: the dependency remains entirely in concept-space, not disk space.
This link below is not "no framework" but it is something I wrote where you can see the result of "no framework thinking". The concepts are stole from people who are probably smarter than me, have decades of experience and written books on these topics. The only difference is instead of turning it into a library to depend on, it's turned into rules for humans (which I guess is also what the book authors originally did anyway). I combined them and made them into a "distro" and I called it "modular provider architecture" (not very engaging or entertaining, but it does what's on the label).
https://github.com/Incognito/python-architecture-linter/tree...
That text document is meant to be an example of how developers should write an application. By the way, it has a demo application here which does basically nothing:
https://github.com/Incognito/python-architecture-linter-demo...
It might be hard to see here because it's pretty silly example, but I managed a small/growing team of 3-5 developers who create over 15 different services following this pattern. They did end up using libraries to do things like send data to/from Kafka or a DB, but the Modular Provider Architecture's rules were always there.
Oh, by the way, that repo I linked to, https://github.com/Incognito/python-architecture-linter/ ... this is a proof of concept for a linter that could implement the "no framework" concept. It is a dev dependency of your project, meaning you have no production framework as a dependency. It is a tool that lets you configure "rules" for your project in the style of any linter you already know of. It's like a linter from hyperspace, you can "lint" rules like.... if a file is 3 levels deep, and depended on by methods anywhere in the project with the word "bob" in the method name name, but those methods don't have if-statements, and also the Afferent coupling of the module itself is less than 0.5 .... fail CI with an explanation why. It also has a feature for you to commit an exemption list.
I used this in my teams once I started managing multiple large teams, and I could do things like generate entire reports across all projects of these really complex metrics that most linters and tools aren't really set up for.
That code is in these files, sorry for the total mess, I was just hacking around and didn't really think of a nice way to structure the definition "API. My main goal was proving the concept.
I think I grossly oversold this thing because there's a lot of comments here asking for something.
I don't really have this concept written down anywhere like a number of other ideas I have. But, I guess the short version is, if I had to make an elevator pitch or something: No framework is a configuration (maybe "distro" in the linux sense) of concepts (maybe "packages" in the software sense). A concept is either something you might use a framework or library for (and usually it exists somewhere), or it is something you would want a linter to find, and it might even be something that you want to ensure was done correctly at code review. I think this last one is the most accurate idea of what a "concept" is.
Over time I have accumulated a small informal set of "packages" that can be implemented without a helper library in nearly the same amount of code as if you were to use that library anyway. The important part is that the running software doesn't depend on the third party code, but actually the developers depend on a rule book and anything that violates the rules should be treated the same as calling an third party package's API method that doesn't exist. In other words: the dependency remains entirely in concept-space, not disk space.
This link below is not "no framework" but it is something I wrote where you can see the result of "no framework thinking". The concepts are stole from people who are probably smarter than me, have decades of experience and written books on these topics. The only difference is instead of turning it into a library to depend on, it's turned into rules for humans (which I guess is also what the book authors originally did anyway). I combined them and made them into a "distro" and I called it "modular provider architecture" (not very engaging or entertaining, but it does what's on the label).
https://github.com/Incognito/python-architecture-linter/tree...
That text document is meant to be an example of how developers should write an application. By the way, it has a demo application here which does basically nothing:
https://github.com/Incognito/python-architecture-linter-demo...
It might be hard to see here because it's pretty silly example, but I managed a small/growing team of 3-5 developers who create over 15 different services following this pattern. They did end up using libraries to do things like send data to/from Kafka or a DB, but the Modular Provider Architecture's rules were always there.
Oh, by the way, that repo I linked to, https://github.com/Incognito/python-architecture-linter/ ... this is a proof of concept for a linter that could implement the "no framework" concept. It is a dev dependency of your project, meaning you have no production framework as a dependency. It is a tool that lets you configure "rules" for your project in the style of any linter you already know of. It's like a linter from hyperspace, you can "lint" rules like.... if a file is 3 levels deep, and depended on by methods anywhere in the project with the word "bob" in the method name name, but those methods don't have if-statements, and also the Afferent coupling of the module itself is less than 0.5 .... fail CI with an explanation why. It also has a feature for you to commit an exemption list.
I used this in my teams once I started managing multiple large teams, and I could do things like generate entire reports across all projects of these really complex metrics that most linters and tools aren't really set up for.
That code is in these files, sorry for the total mess, I was just hacking around and didn't really think of a nice way to structure the definition "API. My main goal was proving the concept.
Joran from TigerBeetle here.
On the contrary, Coil's Interledger [1] project is designing for TigerBeetle, and TigerBeetle will be the core ledger database for Rafiki [2], the open implementation of Interledger.
Joran from TigerBeetle here.
On the contrary, Coil's Interledger [1] project is designing for TigerBeetle, and TigerBeetle will be the core ledger database for Rafiki [2], the open implementation of Interledger.