yt-dlc VS Deep Java Library (DJL)

(Installation instructions are included in the documentation provided by both forks)

I) https://github.com/blackjack4494/yt-dlc

youtube-dlc is a fork of youtube-dl with the intention of getting features tested by the community merged in the tool faster

We love yt-dlp which is a youtube-dl fork based on the now inactive youtube-dlc. The main focus of the project is adding new features and patches while also keeping up to date with the original project.

ERROR: Channel Restricted; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type youtube-dlc -U to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

$ git init # initialize folder as git repo # add remotes to compare $ git remote add youtube-dl https://github.com/ytdl-org/youtube-dl $ git remote add yt-dlc https://github.com/blackjack4494/yt-dlc $ git remote add yt-dlp https://github.com/yt-dlp/yt-dlp $ git remote update # download info from remotes # show number of different commits $ git log youtube-dl/master..yt-dlc/master --oneline | wc -l # in yt-dlc, not in youtube-dl $ git log yt-dlc/master..youtube-dl/master --oneline | wc -l # in youtube-dl, not in yt-dlc $ git log yt-dlc/master..yt-dlp/master --oneline | wc -l $ git log yt-dlp/master..yt-dlc/master --oneline | wc -l $ git log youtube-dl/master..yt-dlp/master --oneline | wc -l $ git log yt-dlp/master..youtube-dl/master --oneline | wc -l # print list of different commits in a file $ git log youtube-dl/master..yt-dlc/master > orig-dlc.txt # in yt-dlc, not in youtube-dl $ git log yt-dlc/master..youtube-dl/master > dlc-orig.txt # in youtube-dl, not in yt-dlc $ git log yt-dlc/master..yt-dlp/master > dlc-dlp.txt $ git log yt-dlp/master..yt-dlc/master > dlp-dlc.txt $ git log youtube-dl/master..yt-dlp/master > orig-dlp.txt $ git log yt-dlp/master..youtube-dl/master > dlp-orig.txt

yt-dlc

Not who you responded to, but I use youtube-dlc. It's a command-line based utility with a ton of options for downloading YouTube videos.

https://github.com/blackjack4494/yt-dlc/commits/master

The previous update was July 25, and the one before that was December 2020.

While the July update dealt with the updater, having 2 days of small changes since December is not very active.

It seems to have been picked up by Eclipse and there is also Oracle Labs' Tribuo and Deep Java Library. All seem active, but I don't know much about any of them. I agree it's probably best to follow the community and use a more popular tool like PyTorch.

Although it may be a bit more work, you can do both machine learning and AI in Java. If you are doing deep learning, you can use DeepJavaLibrary (I do work on this one at Amazon). If you are looking for other ML algorithms, I have seen Smile, Tribuo, or some around Spark.

Image preprocessing I know less about, but tokenization is something I've dealt with a bunch. There are a few options, either push the tokenizer into the ONNX model and use MS's ONNX Runtime extensions (we've used this when working with sentencepiece tokenizers), port the tokenizer entirely to Java (we did this for BERT), or use a sentencepiece or HF tokenizers wrapper directly (e.g. Amazon's DJL did this - HF, sentencepiece).

https://djl.ai/ seems very promising. I've played around with it quite a bit, not in real production though. It's a very well documented (https://d2l.djl.ai/) and active project, with Amazon working on it.

Here is link number 1 - Previous text "DJL"

To give a contrasting perspective, I think the Java ecosystem is much better suited for many data science tasks, and has a growing and well-maintained set of libraries for general purpose machine learning. I won't list them all, but TF-Java, DJL et al. have implementations of many modern architectures and there are a number of excellent libraries (CoreNLP, Lucene et al.) for working with text.

I think DJL also use use it for their tutorials - https://docs.djl.ai/jupyter/tutorial/01_create_your_first_network.html.

AWS Deep Learning more deep learning.