adama-lang VS litestream

I'm just a tryhard. However, I've been coding since a child out of a weird love/obsession. Nothing super successful in the public space, but I retired at 40 to spend time building my cathedral: https://www.adama-platform.com/

my history: https://www.adama-platform.com/2024/01/28/euler.html

I'm in the camp of f = ui(state), and the reason for this is the extreme of streaming games where UI = frame buffer. I'm inventing my own framework for radically simplifying traditional Web apps via RxHTML which works great for crud apps. However, games requires more insight into state machines and what-not.

In terms of the logic, I wrote an entire platform to simplify multi-player board games which I'm evolving to tackle various businesses. https://www.adama-platform.com/

For context, I was not raised religious at all and kind of thought a bunch of it was hokum.

But then I did a heroic dose, and I went to the church of engineering where I saw the entire construction of the machines use from the transistor all the way up to the platform I'm building.

My faith is more a faith that humanity is worth it, and I've come to see "The Lord" as an appropriate metaphor for the collective conciseness of several billion people living their lives.

At core, I'm rejecting nihilism as a valid way to live, and I'm writing an essay to outline the way.

In this, I also reject cynicism, and this is because it offers nothing.

The way forward is to embrace the responsibility that comes with great knowledge and to build an organization that helps push technology forward. This is why my platform is open source ( https://www.adama-platform.com/ ), and I'm working very hard to get my handful of clients in a good place.

I'll take the bait and provide evidence on why you should write a programming language.

First, it's technically very difficult, but you will gain deeper insight into the art of the craft. So, if you are a TC chaser or career minded person, then spending half a year writing a language will help you master the coding aspect of the game. I've started many languages since I started college, and each one was instructional. (I'm now 40 and an early retiree)

Second, it's fun.

Third, it may turn into something new. If some people don't write a new programming language, then we are stuck with what we have. This advice basically admits that the status quo is good enough.

The authors saying that the language, as a project, is a lifetime appointment? Well, this reveals everything. I believe if you want to do a programming language, then you must be willing to invest at least a decade or two.

So, here, I am at forty preparing to launch a SaaS around a language that I designed ( http://www.adama-lang.org/ ). The kicker, I believe, is that a project like this requires wandering the desert alone for quite a while.

I'm preparing to launch, and I just started to load test my shiny new production cluster. Low and behold, it sucks. Fortunately, I have a tremendous number of dashboards and isolated it to how I'm interacting with RDS. I've got my work cut out for me which I'll write about.

However, I have a potentially interesting business precisely because I evolved a language which solved a niche use-case. The number of problems that I have had to solve up to this point is not for the faint of heart. Life and reality are harsh mistress.

So, maybe, yes, you can save yourself some heartache by not writing a language. Perhaps, a better way to think or phase this is "Writing a programming language is a lonely affair that will most likely end in tragedy after a long death march".

I want to make games, but even some years ago I realized it was not a great path for a multitude of reason (many of which are in this article).

My path, and what I recommend, is do something hard and important which pays the bills at a premium. I did infrastructure work, and I was lucky to have a great decade long career allowing me to "retire early".

Now, I can work on a game at my pace building the tools that I see fit. I'm focused on board games because they have a timeless quality about them. I'm developing an entire SaaS platform and programming language to make the network goo beyond easy. http://www.adama-lang.org/

As I'm getting close to some kind of launch for the SaaS, my next thing is to build up my own web based IDE with a release-often ideology such that I can build a Roblox for online 2D board games. Honestly, I'm having a blast because I'm not suffering tools which are going to fade.

I like to call such a list a "wall of shame", and I have my own over here: http://jeffrey.io/wall-of-shame.html

What has helped me over the years is that I view tech as art, and so these projects are only market failures. Your growth as a technologist is manifest, and it's important to see side projects as a form of practice.

If you can move past the failure label and see everything as successful at something, then you feel a whole lot better. A discipline that I have had for decades now is to write a postmortem on why I believe a project is a failure because that crystalizes my learning.

My history helped me excel as a principal engineer which put me towards an early retirement where I can now focus on my ultimate side project: Adama ( http://www.adama-lang.org/ ) which I am turning into a new kind of PaaS thing.

Here is the crazy thing: I already know that I'm making several mistakes as I have no customers and no one asking me for anything. This is an exceptionally lonely way to start a project as the OP and others have noted, but I'm enjoying it well enough.

I'm working on http://www.adama-lang.org/ which started as a programming language for board games, and it is turning into a reactive privacy-focused data store for Jamstack.

I hope to launch in coming month an "Early Access" edition.

While I do intend to turn this into a business, I'm primarily focusing on small projects to amuse myself. I'm going to break every rule in the business with my LLC. The #1 company value is sleep.

I actually think this is onto something that I'm finding in a different way. Instead of a massive database, what if we had a key-value store mapping keys to tiny databases.

This is, to some degree, what I'm building over at http://www.adama-lang.org/ without a full SQL engine. Each document has tables, and the tables can be indexed. I have yet to find a usecase (in my domain) which requires joins. HOWEVER, I've had a ton of fun building it and I'm getting ready to start making games.

I do believe it would be amazing to have a key-logger service where a reducer like sqlite/adama could come into collapse the log into a single file.

The closest I see is from the Boki paper ( https://www.cs.utexas.edu/~zjia/boki-sosp21.pdf ) which was presented at SOSP21.

CRDT solve a part of your problem, and an important consideration is whether or not you want off-line editing. If you don't need off-line editing, then a WebSocket can do it.

I'm actually using my project to build a collaborative IDE (designer like Figma): http://www.adama-lang.org/

I'm going to be launching it as a SaaS soon so people can spin up a new back-end without managing an infrastructure.

I have not, but I keep meaning to collate everything I've learned into a set of useful defaults just to remind myself what settings I should be enabling and why.

Regarding Litestream, I learned pretty much all I know from their documentation: https://litestream.io/

This presentation is focused on the use-case of vertically scaling a single server and driving everything through that app server, which is running SQLite embedded within your application process.

This is the sweet-spot for SQLite applications, but there have been explorations and advances to running SQLite across a network of app servers. LiteFS (https://fly.io/docs/litefs/), the sibling to Litestream for backups (https://litestream.io), is aimed at precisely this use-case. Similarly, Turso (https://turso.tech) is a new-ish managed database company for running SQLite in a more traditional client-server distribution.

This post intends to help you setup replication for SQLite using Litestream.

I've been messing around with litestream. It is so cool. And, I either found a bug in the -timestamp switch or don't understand it correctly.

What I want to do is time travel into my sqlite database. I'm trying to do some forensics on why my web service returned the wrong data during a production event. Unfortunately, after the event, someone deleted records from the database and I'm unsure what the data looked like and am having trouble recreating the production issue.

Litestream has this great switch: -timestamp. If you use it (AFAICT) you can time travel into your database and go back to the database state at that moment. However, it does not seem to work as I expect it to:

https://github.com/benbjohnson/litestream/issues/564

I have the entirety of the sqlite database from the production event as well. Is there a way I could cycle through the WAL files and restore the database to the point in time before the records I need were deleted?

Will someone take sqlite and compile it into the browser using WASM so I can drag a sqlite database and WAL files into it and then using a timeline slider see all the states of the database over time? :)

We're using SQLite in production very heavily with millions of databases and fairly high operations throughput.

But we did run into some scariness around trying to use Litestream that put me off it for the time being. Litestream is really cool but it is also very much a cool hack and the risk of database corruption issues feels very real.

The scariness I ran into was related to this issue https://github.com/benbjohnson/litestream/issues/510

Litestream is a library that allows you to easily create backups. You can probably just do analytic queries on the backup data and reduce load on your server.

https://litestream.io/

One possible strategy is to have one directory/file per customer which is one SQLite file. But then as the user logs in, you have to look up first what database they should be connected to.

OR somehow derive it from the user ID/username. Keeping all the customer databases in a single directory/disk and then constantly "lite streaming" to S3.

Because each user is isolated, they'll be writing to their own database. But migrations would be a pain. They will have to be rolled out to each database separately.

One upside is, you can give users the ability to take their data with them, any time. It is just a single file.

[0]. https://litestream.io/

Upstream Kuma uses a local SQLite database to store account data, configuration for services to monitor, notification settings, and more. To make sure that our data is available across redeploys, we will bundle Uptime Kuma with Litestream, a project that implements streaming replication for SQLite databases to a remote object storage provider. Effectively, this allows us to treat the local SQLite database as if it were securely stored in a remote database.