transcript.fish VS share-file-systems

I have been working on this podcast transcription project for a couple months and it's been super rewarding.

I listen to a podcast called No Such Thing As A Fish, where some researchers talk about their favorite facts they learned that week. Then they riff on it and are generally smart and funny. I listened to the series so many times that I decided I wanted to listen to the show on shuffle, not at the episode level, but at the fact level.

Since I have been playing around with whisper.cpp in python this seemed like a perfect way to combine some technologies I've been wanting to play with.

I ran whisper over the entire podcast and transcribed all the episodes. I had to do this multiple times because I kept messing up. It eventually took like 7 straight days of my M1 processing to get through ~490 episodes.

4 million words, and an 800Mb SQLite database later, I got the transcriptions done and have put up a nice site for searching through the data.

https://transcript.fish

Now I just need to figure out the rest. Breaking it up into facts. Getting the audio working. Highlighting and linking to words, phrases, etc.

Some cool info about the process so far:

1. The SQLite database is chunked up and stored as static files, and the frontend queries the static files directly using HTTP range requests, so it only downloads a couple hundreds kbs when querying.

2. I've been proper using ChatGPT 3.5 free version to help me write python and SQL. It's been pretty game changing as I feel basically no pain from not knowing what I'm doing.

The code is here: https://github.com/noman-land/transcript.fish

Please help if you know how to get whisper speaker diarization working!! I would really appreciate the help.

Interesting, thanks for the tip! I've never heard of lockdown mode. Unfortunately I don't own an iPhone. Any chance you could post an issue with steps to repro?

https://github.com/noman-land/transcript.fish/issues

Thank you!

Some things I learned about trusted localhost HTTPS:

* Windows is the easiest... by far. There is only one trust store and its extremely easy to access at different levels of trust. Firefox has its own trust store so you can either add your certs to both the Windows store AND the Firefox trust store or flip a config in Firefox to tell it to use the Windows trust store like everyone else.

* Linux is a challenge because you have to add your certificates to the OS trust store and then each browser has their own trust stores.

* MacOS is pretty close to impossible, at least fully automated. If the cert is not registered with a third party of the OS's choosing the cert will not be trusted in the browser. The way around this is to manually add your localhost cert chain to the MacOS keychain.

If anybody wants an example here is something I wrote a ways back in JS (but please be warned its specific to my application:

* Build the certificate chain - https://github.com/prettydiff/share-file-systems/blob/master...

* Install the cert by OS type - https://github.com/prettydiff/share-file-systems/blob/master...

That second sample also installs pcap so that I can serve on localhost over ports 80/443.

Some developers believe everything is always a framework or any attempt to avoid frameworks creates a new framework. I cannot help these people. Any non-religion is a cult type nonsense of affirming the consequent fallacy.

Otherwise a valid example is this one file that creates a complete OS-like GUI in the browser awaiting content typically populated from WebSocket messaging: https://github.com/prettydiff/share-file-systems/blob/master...

I wrote a similar concept around private internet access to your file system. It’s at https://github.com/prettydiff/share-file-systems

The window and state management can be demoed on my personal site at https://prettydiff.com

File sharing and soon remote execution over the internet cross OS. Private and no servers.

https://github.com/prettydiff/share-file-systems

Done: https://github.com/prettydiff/share-file-systems/blob/master...

You would need a warrant to extract the messages/identity directly from a person's computer as there is nothing otherwise to obtain.

Perhaps this is true in the context of the web. But I got tired of watching the web as a platform continuously repeat the same mistakes so I started working on something different. In the last day or two I was finally able to functionally prove my competing idea in a way that forcefully imposes privacy with complete Zero Trust conformance.

https://github.com/prettydiff/share-file-systems/blob/master...

I am performing a similar file system tree navigation asynchronously in Node.js which is just a shallow API over the C Linux FS APIs.

I can see you are using opendir and closedir functions? What is the benefit from using the opendir function[1] when readdir[2] can be called on a location directly? Is the benefit that opendir returns a file descriptor for use in opening a stream to gather directory object descriptors?

[1] https://man7.org/linux/man-pages/man3/opendir.3.html

[2] https://man7.org/linux/man-pages/man3/readdir.3.html

Your project is probably more mature but if you want an alternate approach to examine here is I have been doing it: https://github.com/prettydiff/share-file-systems/blob/master...

I considering changing my use of readdir to use the withFileTypes option so that it returns a list of directory entries (objects of artifact name and type) instead of a list of conditions to discern types like I am doing on lines 382-432.

Solved.

Solved for both Windows and Linux (Debian, Arch, Fedora). I might have unlikely solved this of OSX as well, but I am not buying Apply hardware just to test it.

What my solution does is check for certificates created by the project during a build step. If the certificates don't exist it creates them, installs them in the OS, and also install them in the browser. Installation in the browsers is required in Linux and only for FireFox in Windows. These are cert chains containing a self-signed root, intermediary CA, and a local domain cert.

I have these certs configured to work with my own domains so that I can connect to a subdomain addressed to a loopback IP and the cert recognizes that domain, but the domain "localhost" works as well. Sometimes its nice to access a real domain to avoid any restrictions imposed upon accessing address "localhost". You just have to change the domains at the bottom of your OpenSSL option files.

Here is how I solved it with vanilla TypeScript in Node.js (also requires locally installed OpenSSL:

* OpenSSL option file 1 - https://github.com/prettydiff/share-file-systems/blob/master...

* OpenSSL option file 2 - https://github.com/prettydiff/share-file-systems/blob/master...

* Certificate library - https://github.com/prettydiff/share-file-systems/blob/master...

* Certificate interface from build tool - https://github.com/prettydiff/share-file-systems/blob/master...

* Certificate installation - https://github.com/prettydiff/share-file-systems/blob/master...

If you have any questions just open a Github issue on the project.

Email: [email protected]

15 years experience with JavaScript, 6 years experience with TypeScript. I am currently writing a Node based OS in TypeScript to solve for decentralization (not Web3): https://github.com/prettydiff/share-file-systems

I understand performance aggressively enough far beyond the comfort of most developers: https://github.com/prettydiff/wisdom/blob/master/performance...

I started a JS based file sharing application a few years back. It started as a thought experiment of just exposing the file system to the browser in a familiar OS kind of user interface. As new features are added over time it has become more like a high level OS.

https://github.com/prettydiff/share-file-systems

Some architectural decisions I made:

* Micro-service based

* I am now using WebSockets for all services and communication. That has proven in the application to be 7x faster than HTTP.

* I have a universal format wrapping all service messaging, kind of like sending a letter in an envelope. This allows me to using a single service end point for all services and a single means of service monitoring.

* I did not like the existing test automation solutions based upon CDP, because they are too slow and fragile. Also, they do not provide support for a peer-to-peer experience. So I wrote my own test automation solution for testing in the browser and its much faster and predictable.

* I am using an identity based authentication mechanism to restrict access to known users/devices.

* I just write to the file system instead of using a database for data storage. This allows for much faster application start up times and lowers complexity. The performance difference is insignificant after accounting for that in most cases opening a file is more costly than arbitrarily writing to the file system.

* I figured out how to install certificates using automation in both Windows and Linux which allows me to run the application using encrypted transmission protocols (https/wss) on localhost.