libsql VS label-studio

> PS: I've got nothing against Turso, or libSQL. In fact I spent the last year perusing their virtual WAL API. The problem is that I found no documentation, nor any useful open source implementations of it. If there any I'd be very interested. So, thus far, I also don't have anything that drives towards libSQL.

Hey, this is v and I am an engineer at Turso. We do have some documentation and an example implementation of Virtual WAL

docs: https://github.com/tursodatabase/libsql/blob/ef44612/libsql-...

example: https://github.com/tursodatabase/libsql/blob/ef44612/libsql-...

for an open source implementation, you may check how Bottomless works. Bottomless is another project which does back up like litestream and it internally implements a Virtual WAL.

Bottomless - https://github.com/tursodatabase/libsql/tree/main/bottomless

I am sure we can improve our docs, make it more discover-able and easy to find. I am open to feedback and suggestions!

Astro DB is powered by LibSQL, an open source fork of SQLite that was created by Turso. You can use Astro DB's drop-in database to build features like blogs, comment functionality, forums, feedback systems, and user authentication.

sqlite already had an active community fork started by Turso called libsql. They are fixing longstanding API gaps the upstream team isn’t interested in supporting. For example, they added a native write-ahead log API, so you can plug directly into the WAL for streaming replication. This is possible-ish with upstream sqlite + LiteFs but litefs has to implement a whole FUSE file system and can’t run on Mac for that reason.

It’s more risky to run libsql because new features mean new bugs, but it seems worth it to me.

Libsql: https://github.com/tursodatabase/libsql

A bit of a tangent but for those who’d like to use SQLite for a backend, running it as a separate daemon could be an interesting choice, which would also remove there need of Cgo for the build and maybe make things like separate background job processes easier to accomplish. See [1], [2].

—-

1: https://github.com/tursodatabase/libsql/tree/main/libsql-ser...

2: https://news.ycombinator.com/item?id=38602175

https://www.sqlite.org/copyright.html

To summarize, instead of using one of the OSS licenses, the copyright holders simply declare the source to be in the public domain. In order to preserve that status they don't accept patches unless you submit some signed document that you agree with that.

To make things more complicated, they also use their a relatively niche version management system instead of git. Which would complicate making contributions (if they accepted them).

There's a popular fork that fixes all of these issues: https://github.com/libsql/libsql It is MIT licensed, on Github, and open for contributions.

Kind of a weird legal situation for a popular project like this that so many people depend on to have. Not judging; but it is odd. Seems like a lot of wasted efforts between users, would be contributors, and the people that forked this thing to address all that.

You could try making feature requests for https://github.com/libsql/libsql , which is a community fork of SQLite that aims to speed-up the development of long-wanted features.

For a comprehensive view, check out the issues list for libSQL core and sqld. But mostly, I want libSQL to be a home for all builders who believe there is room to take a lean, mean, and SQLite-compatible embedded database to new heights. I’d love to see your contribution!

If instead you have a cohort on hand — -i.e., you do not want to send your data to a third party for any reason, or perhaps you have energetic undergrads — -then you could alternatively consider local, open-source annotation such as CVAT and Label Studio. Finally, nowadays, you might instead work with Large Multimodal Models to have them annotate your data; more on this awkward angle later.

14. LabelStudio by Human Signal | Github | tutorial

For instance, the COCO Annotator is a web-based image annotation tool tailored for the COCO dataset format, allowing collaborative labeling with features like attribute tagging and automatic segmentation. Similarly, Label Studio offers an easy-to-use interface for bounding box object labeling in images.

Huh?

I wrote my own system for classifying a stream of texts in Python, I might Open Source it one of these days but I have to get it to the point where it is modular enough that I can customize it to do the particular things I want without subjecting people to my whims... I use it every day and I'm not afraid to demo it because it is rock solid.

My understanding is that my system would not be hard to adapt to work on images for certain kinds of tasks.

Pytorch is open source, Huggingface is open source. CUDA isn't. This is

https://labelstud.io/

and for annotating text spans there are so many open source tools

https://github.com/doccano/doccano

I worked for a company a few years back that built annotation tools for projects we sold to customers but never quite got to a polished general purpose annotator. Today there are an overwhelming number of companies in this space and products I never heard of, many of which are cloud based or paid. Looks like a gold rush to me.

LabelStudio is free for single use. https://labelstud.io/

LabelStudio is pretty nice, and free & open source, but I have yet to try out their ML integration with a YOLO object detection model.

Label Studio is an open source data labeling tool that includes annotation functionality. It provides a simple user interface (UI) that lets you label various data types, including text, audio, time series data, videos, and images, and export the information to various model formats.