learn VS sqltorrent

I don't have a way to find and curate audiobooks. Plus, LibGen books are already available on IPFS so all I need is collect links. I have been running https://learnawesome.org/ so books seemed far more approachable.

Fantastic! I'll have a deeper look and see if there's any opportunities for integrating this into https://learnawesome.org (which is also open-source).

I am building https://learnawesome.org

It's an attempt to organize world's knowledge. Right now, it looks like GoodReads-like social network for learning resources organized by topics, formats, difficulty levels etc. But there's a knowledge-graph that separates ideas and the medium those ideas are expressed in. For eg: "Sapiens - the book" and "TED Talk given by Yuval Harari" are connected to the same node.

This idea isn't anything new. Here is Danny Hillis talking about it at OSCON 2012: https://youtu.be/wKcZ8ozCah0

https://github.com/learn-awesome/learn for example

Perhaps https://learnawesome.org/.

I have been dealing with the same problem for curating resources at https://learnawesome.org. Projects like Openlibrary do collect unique identifiers for _books_, but for everything else, it mostly takes manual effort. For example, I collect talks/podcasts by the author where they discuss ideas from their books. Then there are summaries written by others.

This is a fantastic effort! Kudos :-)

I have been collecting learning resources and their reviews by experts at https://learnawesome.org/ (open-source, built with Ruby on Rails and TailwindCSS). Would you be kind enough to share the raw JSON files for their books?

LearnAwesome has its own topic taxonomy so it will still require tagging topics manually, but it can save me some effort on scraping/parsing LSE/Nature sites.

I have been adding a number of these free books at https://learnawesome.org/

Do you really care about the format being PDF or is it about the books being FREE? I'd like to make common queries like yours easier. LearnAwesome is open-source, so of course you're free to contribute: https://github.com/learn-awesome/learn

Or even better store data as an sqlite file that is full-text-search indexed. Then you can full-text search the torrent on demand: https://github.com/bittorrent/sqltorrent

"There was that project some guy posted a while back that used a combination of sqlite and partial downloads to enable searches on a database before it was downloaded all the way."

https://github.com/bittorrent/sqltorrent

Interesting question. I looked at the source code to understand that.

SQLite knows where to look for when you open a SQLite database and you run a query, right? It just asks the underlying filesystem to provide N bytes starting from an offset using a C function, then it repeats the same operation on different portions of the file, it does its computation and everybody is happy.

The software relies on sqltorrent, which is a custom VFS for SQLite. That means that SQLite function to read data from a file stored in the filesystem is replaced by a custom function. Such custom code computes which Torrent block(s) should have the highest priority, by dividing the offset and the number of bytes that SQLite wants to read by the size of the torrent blocks. It is just a division.

See: https://github.com/bittorrent/sqltorrent/blob/master/sqltorr...