overture VS rupy

Fastmail shows lists of messages using a progressively-loaded list, where each item is of a consistent height (88px for me, but it can be a few other values too, depending on your configuration—I think 51px is the default). This means that the scrollbar is real and accurate, and you can seek to any point in your mailbox easily (provided your platform allows interacting with the scrollbar, which largely means “on desktop platforms”). But this does cause problems for very large mailboxes, because browsers only support finite lengths.

A few years back, while I worked at Fastmail, we had a ticket come in from an IE user that they could suddenly only access the first few messages in their mailbox. Trouble was they’d gone over IE’s limit, and IE just ignored the entire height declaration in that case, and so you ended up with only the initially-rendered list items available.

The limits I found:

• Firefox: ignores declarations that resolve to a value higher than 17,895,697 pixels (which is a bit more than 2²⁴).

• IE: ignores declarations that resolve to a value equal to or higher than 10,737,418.23 pixels (2³⁰ − 1 hundredth pixels).

• WebKit: clamps values somewhere around 2²⁵ (~33,554,432) pixels; clamping means you don’t need to worry about it so much, since that was the best workaround in other browsers anyway.

And so we ended up with the workaround code at https://github.com/fastmail/overture/blob/41cdf36f3e7c8f0dd1... (the Firefox check was of much older vintage, I just added the IE case). (Nowadays, the IE part is gone again because IE is gone, hooray!)

So yeah, it actually only took about 200,000 messages in the list to hit this limit and fall over, or subsequently just make the bottom of the mailbox inaccessible. 200,000 messages in one mailbox is uncommon, but not at all unrealistic, especially in an “All mail” sort of mailbox.

One uncommon place where clipping is justified at the design level: lazy-loading but finite lists. I’ll use Fastmail’s webmail (on which I worked a few years back) as an example. I could load a list of a hundred thousand emails, and each message in the list is 88px tall (containing four lines of text—approximately, sender and date, subject, and two lines of preview, with truncation on each), so the list container is made to be 8,800,000 pixels high¹, and I can use its scrollbar to immediately jump to any place, and it will figure out which messages to fetch and render based on the scroll position. If the subject line were wrapped, which would be nice at times, you’d lose this ability: you’d have to guess the approximate height of each element, and your scroll positions will be imprecise and you’ll have to make messy adjustments from time to time. Overall it generally won’t be too bad so long as there’s not too much variation in them, but it’s definitely still inferior.

¹ Browser do have limits on how large you can make containers, and handle excess in different ways. IE had the lowest threshold of failure at around ten million pixels, beyond which point it would ignore values; the workaround I implemented in https://github.com/fastmail/overture/commit/8d01c74d8c5d4ae0... came as a direct result of a customer reporting that scrolling was broken in IE in their mailbox with a couple of hundred thousand emails. Firefox breaks a little after 2²⁴ pixels, also ignoring values, so it’s still covered in https://github.com/fastmail/overture/blob/0c9828a5b77ad14383... (note the IE stuff is gone because IE is dead! :-) ). Chrome accepts larger values, but clamps them to about 2²⁵ pixels.

It is, however, interesting to note that Fastmail’s webmail doesn’t use EventSource, but instead implements it atop fetch or XMLHttpRequest. An implementation atop XMLHttpRequest was required in the past because of IE, but it still deliberately doesn’t use EventSource; my foggy recollection from a few years ago is that it had to do with control over timeout/disconnect/reconnect, and handling Last-Event-ID, plus maybe skipping browser bugs in some older (now positively ancient and definitely unsupported) browsers. The source for that stuff is the three *EventSource.js files in https://github.com/fastmail/overture/tree/master/source/io.

I have been running a Raspberry 2 cluster for 10 years: http://host.rupy.se

A few weeks back the first SD card to fail got so corrupted it failed to reboot!

My key learning is use oversized cards, because then the bitcycle will wear slower!

I'm going from 32GB to 256/512/1024!

How this article does not mention SSE, comet or chunking escapes me.

What does their definition of event-driven really look like in practice.

Nobody has a clue.

Here is the ideal event driven system, it's async-to-async: https://github.com/tinspin/rupy/wiki/Fuse

The example is not working because I had to shut down the services for multiple reasons, but the high level of it is that you use 4 (potentially different) threads to do one request/response middle man transaction.

That way you have _zero_ io-wait or idling. I'm surprised nobody has copied this approach since I invented it 10 years ago. I understand why though you need your entire chain to be async and that means rewriting everything and that is a big risk when it's hard to debug.

But if you succeed you can build something that is 10x perf/watt than all other implementations. Which is going to be important when interest rates go higher and crash our entire industry.

The hardware is peaking.

So software is where you can make the difference: http://host.rupy.se

I think we're all confused over the definition. Also one might understand what all the proponents are talking about better if they think about this more as a process and not some technological solution:

https://github.com/tinspin/rupy/wiki/Process

All input I have is you want your code to run on many machines, in fact you want it to run the same on all machines you need to deliver and preferably more. Vertically and horizontally at the same time, so your services only call localhost but in many separate places.

This in turn mandates a distributed database. And later you discover it has to be capable of async-to-async = no blocking ever anywhere in the whole solution.

The way I do this is I hot-deploy my applications async. to all servers in the cluster, this is what a cluster node looks like in practice (the name next to Host: is the node): http://host.rupy.se if you click "api & metrics" you'll see the services.

With this not only do you get scalability, but also redundancy and development is maintained at live coding levels.

I have hosted my own web server both physically and codevise since 2014.

It's on a Raspberry 2 cluster:

http://host.rupy.se

Since 2016 i have my own database also coded from scratch:

http://root.rupy.se

We need to implement HTTP/1.1 with less bloat, a C non-blocking web server that can share memory between threads is probably the most interesting project for humans right now, is anyone working on that?

I have one in Java: https://github.com/tinspin/rupy

Here is the 2000 lines of code of the entire database: http://root.rupy.se/code?path=/Root.java

And here you can try it out: http://root.rupy.se

The smallest PaaS you have ever seen is one order of magnitude larger than mine: https://github.com/tinspin/rupy

And I bet you the same goes for performance, if not two!

The data is here: http://fuse.rupy.se/about.html

Under Performance. Per watt the fuse/rupy platform completely crushes all competition because of 2 reasons:

- Event driven protocol design, averages at about 4 messages/player/second (means you cannot do spraying or headshots f.ex. which is another feature in my game design opinion).

- Java's memory model with atomic concurrency which needs a VM and GC (C++ copied that memory model in C++11, but it failed completely because they lack both VM and GC, but that model is still to this day the one C++ uses), you can read more about this here: https://github.com/tinspin/rupy/wiki

You can argue those points are bad arguments, but if you look at performance per watt with some consideration for developer friendlyness, I'm pretty sure in 100 years we will still be coding minimalist JavaSE on the server and vanilla C (compiled with C++ compiler) on the client.