anarki VS hnrss

It still is a single single-core server, dang references it frequently when there's unusually high traffic [0]. And the language you're referring to is Arc [1]. They do have caching for not-logged-in users, historically done through nginx [2]. From other comments in this thread, it sounds like they just temporarily put Cloudflare in front of that single server to block a DDoS.

[0] https://news.ycombinator.com/item?id=38310213

[1] https://arclanguage.github.io/

[2] https://news.ycombinator.com/item?id=26473226

Slashdot, reddit, and HN are similar in that the source code was available. For HN, as part of arc under the Artistic license. All 3 abandoned public source code releases.

https://sourceforge.net/projects/slashcode/

https://github.com/arclanguage/anarki

https://github.com/reddit-archive/reddit

>Where is the HN source code right now? Free and Open?

Yes and no.

HN itself is running a proprietary fork of Arc Lisp, which you can find here[0]. The Arc maintainters don't take public PRs or feature requests, and HN itself has numerous changes to the codebase which aren't public for business reasons.

There is a public fork of Arc called Anarki[1] which has no direct connection to HN or Arc Lisp, and for which the community and development is... well... anarchic.

And given the general culture here around minimalism and stasis (not wanting to introduce new features for fear of entropy that would negatively affect the signal to noise ratio and push the site towards Eternal September) chances are it's not likely to happen.

But dang's email is at the bottom of the page if you want to ask him.

[0]http://arclanguage.org/

[1]https://github.com/arclanguage/anarki

The original version was open sourced (Perl artistic License) http://arclanguage.org/ There is an active fork in https://github.com/arclanguage/anarki but it's totally independent and the current conde in HN can be (very) different.

My guess is that it's very difficult to keep all the details of the secret sauce hidden. They change the details very often. For example the front page is ordered by points/time^1.6, but the 1.6 changes from time to time without notice (I think it was 1.8 for some time, perhaps it's 1.8 or something else now. Some people have analyzed the front page and got compatible results, but I don't remember the exponent they found and I'm too lazy to try).

You might find something useful here:

https://github.com/arclanguage/anarki/tree/master/apps/news

It was published in the public version of 2009 https://github.com/arclanguage/anarki/blob/9f2e1dd53b5b66bb4... Look for "contro-factor" (i.e. controversial-factor).

This is old code, and the mods make tweaks here and there without warning, so the details may have changed. (I'm not sure if gravity changed from 1.8 to 1.6 (???).)

I didn't test it personally, but it feels like HN is using something very similar. And there are a few black box analysis of the sorting of the front page that got similar results. Also, minimaxir is the kind of person that is probably running an script to use the HN API to verify the claims.

Absolutely. It's basically the "community version" of Paul Graham's Arc.

Not to mention arc/anarki (hn is/was written in arc - anarki comes with a "news" example app/forum):

https://github.com/arclanguage/anarki/tree/HEAD#readme

arc [1] / racket implimentation of demystifycpp might be able to provide something 'usable' at the command line / straight up web browser html file.

[1] : https://arclanguage.github.io/

That’s interesting.

I have predictive models that can predict if a headline (w/o the rest of the article and not considering the URL) will (a) get more than 10 votes and (b) if it does get more than 10 votes will the votes/comments ratio be more than 2 (which is roughly average)

The first model gets a ROC-AUC (see https://scikit-learn.org/stable/modules/generated/sklearn.me...) in the low 60’s (not good, the second model gets in the low 70’s (actually pretty good though it is a heat seeking missile for clickbait headlines) and my latest content-based recommender for RSS items gets almost 80. (I saw a paper that one system at TikTok gets about 85)

To do all that you need about 10,000 headlines and don’t get a lot of benefit from having more than 100,000. The ceilings on performance have more to do with the nature of the problem rather than my models: the same article can get submitted twice and get 0 votes one time and 200 the other time so it can never be as accurate as “is this an article about galactic astronomy?”

I had it ingest the HN comments firehose and found the amount of articles was overwhelming, my YOShInOn RSS reader now ingests the “best comments” from

https://hnrss.github.io/

together with 110 other feeds and actually I like the comments it picks out a lot. Now that the system is adding about 3000 items per day it might be able to handle a big feed like the comments firehose since now those comments are diluted with so many quality articles. For a problem like that you might want a two-score system with: (i) is it relevant? (something I like) and (ii) is it popular? (like Google’s PageRank)

I think you could make a model that compares comments in the best comments feed with other comments. I have tried formulating the problems above as regression problems where I try to predict the actual score and it does not work well because of the uncertainty problem but formulated as a classification problem for a score over a threshold it is easy to make a well-calibrated model that tells you “this article has a 20% chance of frontpaging” which is about the best anyone can do.

Subscribe via rss, so you can scratch the curiosity itch and each the FOMO, without coming to the site all the time and looking over the same things 20 times?

https://hnrss.github.io/

https://news.ycombinator.com/item?id=9491978

and this https://hnrss.github.io/

ps i’m ok with some % of false positives, but hopefully a sprinkle of OpenAI could keep that magically low?

thanks

It sounds interesting but I use https://hnrss.github.io/

Unless it had most of the features of hnrss.org I would not be able to use it.

Perhaps you could pivot your approach and submit a PR to hnrss for the feature?

Looks like there's an unmerged PR on the third-party hnrss project that would add this: https://github.com/hnrss/hnrss/pull/84

Check out below link to get a more customized, topic wise rss feeds.

https://hnrss.github.io/