foundationdb VS GmsCore

Actually, Apple does this for iCloud! They use FoundationDB[1] to store billions of databases, one for each user (plus shared or global databases).

See: https://read.engineerscodex.com/p/how-apple-built-icloud-to-...

Discussed on HN at the time: https://news.ycombinator.com/item?id=39028672

[1]: https://github.com/apple/foundationdb https://en.wikipedia.org/wiki/FoundationDB

Is this the language in question? https://github.com/apple/foundationdb/blob/main/flow/README....

> I wondered if perhaps this WASM use case for a cross-language library was already just as possible and ergonomic using language bindings and maybe thats why this use case doesnt seem like a big deal to people.

Yeah that’s the reason. You don’t notice it a lot of the time, but FFIs are everywhere already. The most common foreign function interface is basically the ability to call C code, or have functions made available to C code. C is used because everyone knows it and it’s simple. And most languages either compile to native code (eg rust) - which makes linking to C code easy. Or the runtime is implemented in C or C++ (eg V8, Ruby). In languages like that, the standard library is already basically implemented via a FFI to C/C++ code.

I’ve got an iOS app I’m working on that’s half rust and half swift, with a touch of C in the middle. The bindings work great - the whole thing links together into one binary, even with link time optimizations. But the glue code is gross, and when I want to fiddle with the rust to Swift API I need to change my code in about 4 different places.

Most FFIs are a one to many relationship in that if you write a clean C API, you can probably write bindings in every language. But you don’t actually want to call naked C code from Ruby or Javascript. Good bindings will make you forget everything is done via ffi. Eg numpy. I haven’t looked at the wasm component model proposal - I assume it’s trying to make this process cleaner, which sounds lovely.

I maintain the nodejs bindings for foundationdb. Foundationdb bindings are all done via ffi linking to their C code. And the API is complex, using promises and things. I find it really interesting browsing their official bindings to go, Java, Python and Ruby. Same bindings. Same wrapped api. Same team of authors. Just different languages. And that’s enough to make the wrapper wildly different in every language. From memory the Java ffi wrapper is 4x as much code as it is in Ruby.

https://github.com/apple/foundationdb/tree/main/bindings

Isn’t apple server on C/C++?

They have their own db in that

https://github.com/apple/foundationdb

> Any meaningful technology is open source.

Clearly untrue, however FoundationDB is open source, with a permissive license.

https://github.com/apple/foundationdb

So is much of the operational tooling for it:

https://github.com/FoundationDB/fdb-kubernetes-operator

I don't recall any of those details but the test involved injecting a bogus block device that always returns garbage, and noting that this results in garbage records returned from client queries. And I don't think those kinds of issues have been eradicated, browsing through their github issues there are people trying to recover corrupted clusters. https://github.com/apple/foundationdb/issues/2480

For my current job, I have to install FoundationDB (they have a github repo) as a dependency for the project's code.

To clarify, FoundationDB has some documented multi-region features, but it's not at all clear that anybody who runs it at scale relies on those multi-region features [1]. Even if they do, it's not obvious they run this Fearless DR mode at any appreciable latency.

Geo-replication is an area where Anna really shines, and foundation pays severe costs. All of the concurrency control problems have contention footprints on the order of round-trips. Fully optimistic concurrency control needs to expose backoffs quite a few round-trips to be live. Even pessimistic concurrency control requires some number of round trips, probably at least 1.5 optimally, but in practice in most tuned systems, probably 3. A heck of a lot of use cases make sense at 0 global RTTs and don't at 1+. The ability to tell the database how to manage concurrency, and then providing causal is ultimately the best you can do. That's awesome.

At the end of the day, I have to believe that there's a whole big mess of applications we'll build this, on systems in the portion of the design space Anna is choosing, one day. This is only recently not cutting edge research, but it's definitely still research. We don't know how to model these concepts at a high-level and in a composable way where the masses of software developers can engage with them.

It's interesting to think about how long ago we were graced with BAYOU [2], that thing was ahead of its time. I suspect it's going to take a little while longer before these sorts of techniques make their way into data storage primitives we think of as part of the standard vernacular, but I believe we'll get there eventually.

Databases are fun!

[1] https://github.com/apple/foundationdb/wiki/Multi-Region-Repl...

I, an engineer, am not doing this myself, too. There is a middle ground though: just use a privacy-oriented Android build, like DivestOS. [1]

There are a couple caveats:

1. It is still a bit tricky for a non-technical person to install. Should not be a problem if they know somebody who can help, though. There's been some progress making the process more user friendly recently (e.g. WebUSB-based GrapheneOS installer).

2. There are some papercuts if you don't install Google services on your phone. microG [2] helps with most but some still remain. My main concern with this setup is that I can't use Google Pay this way, but having to bring my card with me every time seems like an acceptable trade off to me.

[1]: https://divestos.org/

[2]: https://microg.org/

I have been running de-googled LineageOS since before it renamed/reformed from CyanogenMod, so since somewhere around 2013/14. That has looked rather different depending on what exactly I need from my phone but I'll share what my current set up looks like.

First, I have don't use any kind of Google/Samsung/Apple Pay wallets so if you do, this may not be helpful; I've never looked into trying to get any of those working. Also, by "de-googled" I mean that I don't have GApps installed on my phone. I do have microG[0] installed as a Magisk[1]/LSPosed[2] module; this allows a few apps to believe I have GApps while most apps do not see/have access. I do not turn on microG for any apps (i.e. no connection to the Google servers/services via microG).

Most of my apps come from F-Droid[3], a few from Aurora Store[4] (a 3rd-part frontend for the Play Store that does not require either an account or GApps installed), a very few from FFupdater[5], and have played with using Obtanium[6] but currently only have one (weather) app updating via it. I have several different repositories configured in F-Droid but I don't generally keep mental track of which repository I am dependent on for which apps; the default, IzzyOnDroid[7], Bitwarden[8], NewPipe[9], microG[10], and Collabora[11] are some of them.

I have two banking apps installed via Aurora Store, one of which requires microG and root-hiding (via Magisk module) while the other doesn't require either. My browsers (Firefox, Firefox Klar, Brave) come from FFupdater and none require microG. My texting (QUIK SMS), email (K-9 Mail), TOTP authenticator (Aegis), password manager (Bitwarden), GPS/Maps (OsmAnd), file syncing (Nextcloud), notes (Nextcloud Notes), HN reader (HN), and Contacts/Calendar sync (DAVx5, ICSx5) apps all come via F-Droid (either the main repo or others). I have many others apps which come from F-Droid or Aurora Store but the above are my most used.

For file, calendar, notes, photo, & contact syncing, I have a Nextcloud server set up and find it to work quite well; the Nextcloud apps are also quite good. Someone who doesn't want to run their own could use a hosted account[12]. Contacts & calendars are synced to Nextcloud via DAVx5 & ICSx5.

The primary challenges I am aware of at this point are hardware (it is increasingly difficult to install LineageOS on most hardware due to bootloader locks), and navigation (OpenStreetMap data usually doesn't include addresses in the USA). For hardware, the solution is essentially just to properly research the phone you want to buy; I always make sure the model is well supported by LineageOS before purchasing and then tend to hang on to it for several years. For navigation, I usually find the address on my desktop or mobile browser (via DuckDuckGo) and then manually input the location into OsmAnd before the trip but I also keep WeGo Here maps installed in case I don't have time for that (it usually takes <2 minutes and rarely more than 5 to manually find & enter the address). Additionally, getting the one banking app to work without GApps was a pain in the butt initially (requiring testing several Magisk & LSPosed modules), but now it just works and I don't really think about it.

Overall, I don't find my version of de-googled to be a detriment; my phone is useful and I have more control over my data and over annoyances (such as unnecessary notifications) than I would otherwise.

[0] https://microg.org

Is anyone here daily-driving microg and can share their experiences? https://github.com/microg/GmsCore/wiki/Implementation-Status does not exactly inspire confidence.

...will need to be rewritten to avoid Google Play Services.

Not true.

All that needs to happen is for open source developers to "re-implement Google’s proprietary Android user space apps and libraries".

https://microg.org/

microG itself connects directly to Google: https://github.com/microg/GmsCore/wiki/Google-Network-Connec...

No shit, of course they do.

>In general, we obviously try to minimize the connections to Google, but some services strictly rely on them and would just not work without.

What exactly do you think they should do instead?

MicroG

In release notes for GmsCore v0.2.29.233013 (https://github.com/microg/GmsCore/releases/tag/v0.2.29.233013), I also see:

Yes, the Xposed module is one way. There are also other ways

Which one should I use? Is this MicroG's official website right? (https://microg.org/)