Django-link-archive VS yt-dlp

I am running my of YouTube front-end.

Link: https://github.com/rumca-js/Django-link-archive

Demo: https://renegat0x0.ddns.net/apps/catalog

Allows me to add channels, download individual videos, bookmark videos, etc. Uses iframe to display the video.

I have no problems with viewing videos:

- I use ff with adblock

- videos are embedded using no-referrer-when-downgrade policy

I have literally no ads.

There are some drawbacks:

- YouTube may change their policies, change embedding strategies

- To have control over videos, you still have to manually download them

I wrote bookmark manager in Django.

https://github.com/rumca-js/Django-link-archive

You can self host it.

You can add RSS sources and auto import new links regularly.

It may not be stare of the art, but gets the job done.

Demo below, but may not be working when you look at it. It runs on raspberry pi.

https://renegat0x0.ddns.net/apps/catalog/entry/11503/

I have myself dabbled a little bit in that subject. Some of my notes:

- some RSS feeds are protected by cloudflare. It is true however that it is not necessary for majority of blogs. If you would like to do more then selenium would be a way to solve "cloudflare" protected links

- sometimes even selenium headless is not enough and full blown browser in selenium is necessary to fool it's protection

- sometimes even that is not enough

- then I started to wonder, why some RSS feeds are so well protected by cloudflare, but who am I to judge?

- sometimes it is beneficial to cover user agent. I feel bad for setting my user agent to chrome, but again, why RSS feeds are so well protected?

- you cannot parse, read entire Internet, therefore you always need to think about compromises. For example I have narrowed area of my searches in one of my projects to domains only. Now I can find most of the common domains, and I sort them by their "importance"

- RSS links do change. There need to be automated means to disable some feeds automatically to prevent checking inactive domains

- I do not see any configurable timeout for reading a page, but I am not familiar with aiohttp. Some pages might waste your time

- I hate that some RSS feeds are not configured properly. Some sites do not provide a valid meta "link" with "application/rss+xml". Some RSS feeds have naive titles like "Home", or no title at all. Such a waste of opportunity

My RSS feed parser, link archiver, web crawler: https://github.com/rumca-js/Django-link-archive. Especially interesting could be file rsshistory/webtools.py. It is not advanced programming craft, but it got the job done.

Additionally, in other project I have collected around 2378 of personal sites. I collect domains in https://github.com/rumca-js/Internet-Places-Database/tree/ma... . These files are JSONs. All personal sites have tag "personal".

Most of the things are collected from:

https://nownownow.com/

https://searchmysite.net/

I wanted also to process domains from https://downloads.marginalia.nu/, but haven't got time to read structure of the files

"""

Disclaimer: I have a automated RSS reader enabled in my network: https://github.com/rumca-js/Django-link-archive

On the other hand it is not 1995. Time has moved on. I wrote a Simple RSS feed, that also serves as search engine for bookmarks.

I am able to run it in attick on raspberry pi. We do not have to rely so heavily on google.

https://github.com/rumca-js/Django-link-archive

It is true that it does not serve me as google, or kagi replacement. It is a very nice addition though.

With a little bit off determination I do not have to be so dependent on google.

Here is also a dump of known domains. Some are personal.

https://github.com/rumca-js/Internet-Places-Database

...and my bookmarks

https://github.com/rumca-js/RSS-Link-Database

Some more years, and google can go to hell.

[4] https://github.com/rumca-js/Django-link-archive

These are exported then to github repositories:

[5] https://github.com/rumca-js/RSS-Link-Database - bookmarks

[6] https://github.com/rumca-js/RSS-Link-Database-2023 - 2023 year news headlines

[7] https://github.com/rumca-js/Internet-Places-Database - all known to me domains, and RSS feeds

My own repositories:

- bookmarked entries https://github.com/rumca-js/RSS-Link-Database

- mostly domains https://github.com/rumca-js/Internet-Places-Database

- all 'news' from 2023 https://github.com/rumca-js/RSS-Link-Database-2023

I am using my own Django program to capture and manage links https://github.com/rumca-js/Django-link-archive.

A list od blogs mentioned by hacker news, some were adres manually by me:

https://github.com/rumca-js/Django-link-archive/blob/main/an...

I have a project. I have posted it once herenon HN. I have not received any feedback then, it hasn't received much traction.

It is a link aggregation. Can be used as a RSS client, or YouTube front end for subscriptions.

It is intended for light, personal use, therefore it is not much scalable, but supports user management.

https://github.com/rumca-js/Django-link-archive

Ha, I have also wrote something similar

https://github.com/rumca-js/Django-link-archive

I support not only youtube, but also any RSS source.

It functions as link aggregation software. I can also fetch meta for all videos in channel, and download videos, audios.

I am using standard Django auth module.

It still lacks polish, and it is under development. I am not a webdev, so I am still struggling with overall architecture

You can put these options in a config file and they will become the default: https://github.com/yt-dlp/yt-dlp?tab=readme-ov-file#configur...

Yep. yt-dlp and youtube-dl

https://github.com/yt-dlp/yt-dlp

https://github.com/ytdl-org/youtube-dl

Will also start to feel the impact. My theory is that we will see a bunch of new video hosting sites as youtube itself attempts to lock down its ecosystem. They haven't paid attention in any adversarial way as far as I can tell.

When they do, it wont be great.

The points you make aren't unreasonable.

It is necessary to establish clear boundaries of what can and can be provided by the maintainers. If not done at an earlier stage of the project, the support burden becomes too much to bear at which point the maintainer transfers ownership, and the project suffers from catastrophic consequences such as the xz backdoor we're talking about here, or other cases where the project mostly stalls and serves as an ego-boosting platform for the new maintainer, as was the case with PhantomJS[6].

This can also happen in your life, where a "friend" sees that you possess a certain skill, and then gradually tries to push an inordinate amount of their personal work related to this field onto you.

Personally, I think it's best to use an approach with extremely clear communication as to what the maintainer can and cannot provide. This can be seen, for example, in yt-dlp[1], where the consumer is clearly informed upfront that not providing detailed information as requested will lead them to block said consumer; or sqlite where their position regarding contributed patches[2] and support[3] is similarly made clear.

Having a shouty BDFL like Torvalds can also help improve code quality[4] and questionable contributions[5], though it is better that the shouty BDFL makes statements that are professional and do not show as much aggression; so for example, "Mauro, shut the fuck up"[7] would become "Mauro, your response is completely unbecoming for a Linux kernel maintainer, and is not in line with the promise of not breaking userspace."

[1] https://github.com/yt-dlp/yt-dlp/issues/new?assignees=&label...

[2] https://www.sqlite.org/copyright.html

[3] https://www.sqlite.org/support.html

[4] https://www.theregister.com/2024/01/29/linux_6_8_rc2/

[5] https://cse.umn.edu/cs/linux-incident

[6] https://github.com/ariya/phantomjs/issues/14541

[7] https://lkml.org/lkml/2012/12/23/75

Or just "yt-dlp "

yt-dlp ( https://github.com/yt-dlp/yt-dlp ) still works pretty well at the current state of Twitter.

To download audio from YouTube videos, you'll utilize the widely used yt-dlp library, which can be installed using the pip command as follows:

Use: `yt-dlp with --write-comments --no-download --batch-file FILE`

- FILE is a text file with a list of YouTube id's/URL's

- https://superuser.com/a/1732443/4390

- https://github.com/yt-dlp/yt-dlp

> Then youtube-dl wasn't a thing anymore (maybe it is again?)...

yt-dlp is definitely a thing: <https://github.com/yt-dlp/yt-dlp>