scrapeghost VS the-algorithm-ml

Not my project but an ex-colleague has been having some success in this direction: https://jamesturk.github.io/scrapeghost/

Yes, there is something like that available - ScrapeGhost.

With AI scraping, tools can be far more resilient than soon enough to minor dom changes. See - https://jamesturk.github.io/scrapeghost/.

Their ToS mentions scraping but it pertains to scraping their frontend instead of using their API, which they don't want you to do.

Also - this library requests the HTML by itself [0] and ships it as a prompt but with preset system messages as the instruction [1].

[0] - https://github.com/jamesturk/scrapeghost/blob/main/src/scrap...

[1] - https://github.com/jamesturk/scrapeghost/blob/main/src/scrap...

I said “parts of the recommender system code.”

This is the kind of highly emotional reaction that’s not helpful.

Yes, I am quite familiar with building ML models, both training and building my own for which I’ve been paid large sums of money, and I’m here to tell you that you don’t know what you’re taking about.

There’s so much more information about an ML system than just the trained model that is important for understanding the effects of the system on a society, and its legal, ethical, and social ramifications.

Just seeing the type of RS being used, the ranking approach, and the information on SimClusters is enough for RAI folks to start to understand the ecosystem effects and how that can show up downstream in social effects.

https://blog.twitter.com/engineering/en_us/topics/open-sourc...

Blog summarising the change: https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm

People who don't share your interests (or at least what Twitter thinks your interests are). This blog post explains it in detail.

Twitter's announcement | Main GitHub Repo | ML GitHub Repo | Engineering Blog Post

where? I searched through the repo and couldn't find it.