github-explorer VS map-of-reddit

clickhouse has pretty good github_events dataset on their playground that folks can use to do some research - some info on the dataset https://ghe.clickhouse.tech/

Example of what user JiaT75 did so far:

https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPT...

pull requests mentioning xz, 5.6 without downgrade, cve being mentioned in the last 60 days:

https://play.clickhouse.com/play?user=play#U0VMRUNUIGNyZWF0Z...

It shouldn’t be hard to implement: https://ghe.clickhouse.tech/#how-to-download-the-data

The article leaves a bitter taste of unnecessary complexity. Data engineering should not be hard.

For example, you can load the GitHub Archive to ClickHouse, and it will be accessible with interactive real-time queries: https://ghe.clickhouse.tech/

See also https://til.simonwillison.net/clickhouse/github-explorer

I recommend checking https://ghe.clickhouse.tech/

It explains the full pipeline - how to download, collect, and analyze this sort of data.

Yes, it's continuously updated.

The source code is here: https://github.com/ClickHouse/github-explorer

This shell scripts updates it: https://github.com/ClickHouse/github-explorer/blob/main/upda...

https://www.reddit.com/r/dataisbeautiful/comments/i8saks/ive...

https://www.reddit.com/r/dataisbeautiful/comments/ic2i0k/oc_...

Stock chart as a landscape image

And I also like these "races" basically a timelapse of a chart

https://www.reddit.com/r/dataisbeautiful/comments/f68fzr/oc_...

And whatever anvaka does, all can be tried out in the browser

https://github.com/anvaka?tab=repositories&q=&type=&language...

e.g. map of reddit:

https://github.com/anvaka/map-of-reddit

Video Demo: https://www.reddit.com/r/dataisbeautiful/comments/12pem68/oc...

Similarly map of github

https://github.com/anvaka/map-of-github

Or package manager visualizations:

https://github.com/anvaka/pm

Google search completion visualizations

https://github.com/anvaka/vs

Which has heavy dependencies on this:https://github.com/anvaka/map-of-reddit/tree/main

Direct link to map: https://anvaka.github.io/map-of-reddit/

https://anvaka.github.io/map-of-reddit gives a sense of the sheer number of nsfw subreddits, shown on the southern continent.

Map is generated by clustering subreddits based oncommenters who comment on multiple subreddits, so I don't think you can infer much from the colored area, that's just background for each group of the graph.

Don't get demoralized by the comments below, while the corporate actions done by Reddit often causes division as it attempts more and more to become a sterile inclusive media mainstream, the very fact it exists as a forum for communities means that there will always be areas for discussion, even if it means having to focus on the smaller subs. I want to emphasise that the most important thing in any scenario is civility. You can pretty much go to any subreddit and say a controversial thing, and while there will obviously be a wave of comments berating it, as long as mods don't remove it there will nearly always be someone to have the intellectual discussion you're looking for. Even just trying to find a non-partisan subreddit like centrists will increase your chances exponentially. A.K.A, if you come approaching here describing the place as white genocide and porn, it's exactly what you'll find. Second of all, while communities that are dedicated to hot topic issues are banned, ones that are around political commentators are less so. And because of the subject matter, most of these attract posters who are wanting to voice their opinion on the topic rather than any dedication to the streamer. Hell, there have even been communities that have branched off completely off there original author to be their own separate entity entirely ( themotte.org and r/theschism both formed as a forum for sensitive issues in the slatestarcodex subreddit). A bunch of the time the users of each one overlap, which makes them easy to find if using websites like https://anvaka.github.io/map-of-reddit/ or https://subredditstats.com/subreddit-user-overlaps

Would be cool to have a https://anvaka.github.io/map-of-reddit/ of that

I love these things - to see everything sort of classified, and all at once, helps me find things I would never think to look for. Forgive me if this has been posted somewhere else, but if you want to see a reddit map, here it is: https://anvaka.github.io/map-of-reddit/?x=18083.096950551575...