rupy VS yugabyte-db

I have been running a Raspberry 2 cluster for 10 years: http://host.rupy.se

A few weeks back the first SD card to fail got so corrupted it failed to reboot!

My key learning is use oversized cards, because then the bitcycle will wear slower!

I'm going from 32GB to 256/512/1024!

How this article does not mention SSE, comet or chunking escapes me.

What does their definition of event-driven really look like in practice.

Nobody has a clue.

Here is the ideal event driven system, it's async-to-async: https://github.com/tinspin/rupy/wiki/Fuse

The example is not working because I had to shut down the services for multiple reasons, but the high level of it is that you use 4 (potentially different) threads to do one request/response middle man transaction.

That way you have _zero_ io-wait or idling. I'm surprised nobody has copied this approach since I invented it 10 years ago. I understand why though you need your entire chain to be async and that means rewriting everything and that is a big risk when it's hard to debug.

But if you succeed you can build something that is 10x perf/watt than all other implementations. Which is going to be important when interest rates go higher and crash our entire industry.

The hardware is peaking.

So software is where you can make the difference: http://host.rupy.se

I think we're all confused over the definition. Also one might understand what all the proponents are talking about better if they think about this more as a process and not some technological solution:

https://github.com/tinspin/rupy/wiki/Process

All input I have is you want your code to run on many machines, in fact you want it to run the same on all machines you need to deliver and preferably more. Vertically and horizontally at the same time, so your services only call localhost but in many separate places.

This in turn mandates a distributed database. And later you discover it has to be capable of async-to-async = no blocking ever anywhere in the whole solution.

The way I do this is I hot-deploy my applications async. to all servers in the cluster, this is what a cluster node looks like in practice (the name next to Host: is the node): http://host.rupy.se if you click "api & metrics" you'll see the services.

With this not only do you get scalability, but also redundancy and development is maintained at live coding levels.

I have hosted my own web server both physically and codevise since 2014.

It's on a Raspberry 2 cluster:

http://host.rupy.se

Since 2016 i have my own database also coded from scratch:

http://root.rupy.se

We need to implement HTTP/1.1 with less bloat, a C non-blocking web server that can share memory between threads is probably the most interesting project for humans right now, is anyone working on that?

I have one in Java: https://github.com/tinspin/rupy

Here is the 2000 lines of code of the entire database: http://root.rupy.se/code?path=/Root.java

And here you can try it out: http://root.rupy.se

The smallest PaaS you have ever seen is one order of magnitude larger than mine: https://github.com/tinspin/rupy

And I bet you the same goes for performance, if not two!

The data is here: http://fuse.rupy.se/about.html

Under Performance. Per watt the fuse/rupy platform completely crushes all competition because of 2 reasons:

- Event driven protocol design, averages at about 4 messages/player/second (means you cannot do spraying or headshots f.ex. which is another feature in my game design opinion).

- Java's memory model with atomic concurrency which needs a VM and GC (C++ copied that memory model in C++11, but it failed completely because they lack both VM and GC, but that model is still to this day the one C++ uses), you can read more about this here: https://github.com/tinspin/rupy/wiki

You can argue those points are bad arguments, but if you look at performance per watt with some consideration for developer friendlyness, I'm pretty sure in 100 years we will still be coding minimalist JavaSE on the server and vanilla C (compiled with C++ compiler) on the client.

It is possible to apply Batched Nested Loop but with additional code that checks the range of the outer bigint and compare it only if it matches the range of integer. This has been added in YugabyteDB 2.21 with #20715 YSQL: Allow BNL on joins over different integer types to help migrations from PostgreSQL with such datatype inconsistencies.

It can be done like https://github.com/yugabyte/yugabyte-db/ has.

Therefore, interleaving or co-partitioning is probably not necessary, and would reduce agility and scalability more than improving the performance. Unless you have a good reason for it that you can share on Issue #79. But, first, test and tune the queries to see if you need something else.

This is a workaround, not supported. I've opened the following issue to get it solve in the YugabyteDB deployment: https://github.com/yugabyte/yugabyte-db/issues/19389

Note that there may still be a need for bitmaps, especially with disjunctions (OR) as the following is about conjunction (AND), and it can still be implemented, differently than PostgreSQL. This is tracked by #4634.

FROM almalinux:8 as build RUN dnf -y update &&\ dnf groupinstall -y 'Development Tools' # get YugabyteDB sources ARG YB_TAG=2.18 RUN git clone --branch ${YB_TAG} https://github.com/yugabyte/yugabyte-db.git WORKDIR yugabyte-db # install dependencies and compilation tools RUN dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm RUN dnf -y install epel-release libatomic rsync python3-devel cmake3 java-1.8.0-openjdk maven npm golang gcc-toolset-12 gcc-toolset-12-libatomic-devel patchelf glibc-langpack-en ccache vim wget python3.11-devel python3.11-pip clang ncurses-devel readline-devel libsqlite3x-devel RUN mkdir /opt/yb-build RUN chown "$USER" /opt/yb-build # Install Python 3 RUN alternatives --remove-all python3 RUN alternatives --remove-all python RUN alternatives --install /usr/bin/python python /usr/bin/python3.11 3 RUN alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 3 # add #include "pg_yb_utils.h" to src/postgres/src/pl/plpython/plpy_procedure.c RUN sed -e '/#include "postgres.h"/a#include "pg_yb_utils.h"' -i src/postgres/src/pl/plpython/plpy_procedure.c # if using python > 3.9 remove #include and #include from src/postgres/src/pl/plpython/plpython.h RUN sed -e '/#include /d' -e '/#include /d' -i src/postgres/src/pl/plpython/plpython.h # add '--with-python', to python/yugabyte/build_postgres.py under the configure_postgres method RUN sed -e "/'\.\/configure',/a\ '--with-python'," -i python/yugabyte/build_postgres.py # Build and package the release RUN YB_CCACHE_DIR="$HOME/.cache/yb_ccache" ./yb_build.sh -j$(nproc) --clean-all --build-yugabyted-ui --no-linuxbrew --clang15 -f release RUN chmod +x bin/get_clients.sh bin/parse_contention.py bin/yb-check-consistency.py RUN YB_USE_LINUXBREW=0 ./yb_release --force WORKDIR / RUN mv /yugabyte-db/build/yugabyte*.tar.gz /yugabyte.tgz

You have seen me using the official YugabyteDB Docker image extensively. This image is suitable for various purposes, including labs, development, testing, and even production. In the past, we used to create it internally due to its seamless integration with our build process. However, some companies prefer to construct the image on their own, which is indeed a commendable practice. After all, it's not advisable to run random images with root privileges on your servers. As a result, we have made a significant alteration by introducing a refined Dockerfile to our Github repository.

Of course, I can guess from the function names, but YugabyteDB is Open Source and I can search for them. What happens here is that I didn't declare a Primary Key for my table and then an internal one (ybctid) is generated, because secondary indexes need a key to address the table row. This ID generation calls /dev/urandom. I made this simple example to show that low-level traces can give a clue about high level data model problems.

To show that it is not difficut to understand what you run, when in a PostgreSQL-compatible database, I'll look at the HammerDB benchmark connected to YugabyteDB. HammerDB has no specific code for it but YugabyteDB is PostgreSQL-compatible (it uses PostgreSQL code on top of distributed storage and transaction).