https-bot VS https-everywhere

It's inspired by this comment I made: https://news.ycombinator.com/item?id=26599746.

I actually saw several comments with HTTP URL posted, and that was the only one I bothered to comment on. So I thought that this is something better suited for bots than human.

I hacked this together over yesterday and today: https://github.com/fishy/https-bot.

Basically it uses the Firebase API (https://github.com/HackerNews/API) to find comments with HTTP URLs in them, try the HTTPS version, compare the contents, post back a comment if the contents are more than 95% similar.

The "95% similar" part was actually the first part I wrote in the code. At first I tried a few existing go packages implementing diff/lcs, but most of them was quite slow and does a lot of allocations when I'm comparing two randomly generated 10KiB blobs, so I wrote my own (https://pkg.go.dev/github.com/fishy/https-bot/similarity), which is optimized for space (it does almost no allocations), and it's also faster because allocations are slow. (I know this is an unfair comparison that most of the existing implementations need to give you an output that can be used to reconstruct the two blobs back, so at least some of their allocations are required and unavoidable)

I also wrote a bug that it would find the same HTTP url in every run and post the same comment over and over again. My apologize to dang or whoever dealt with it (or maybe the system is good enough that it blocked those repetitive comments automatically).

In the end it successfully made 6 comments across ~4 hours (not including the repetitive ones). All of those comments are flagged (likely due to hn policy), https://news.ycombinator.com/item?id=26692588 is the only one that's still visible to other users at the time of writing, if you are curious.

I just killed it completely from the request of dang. Although it only lived for a few hours, it's still a fun exercise. Maybe I'll convert it into a reddit bot next? Who knows.

Thank you for the feedback. I added FAQ to the README and answered this question: https://github.com/fishy/https-bot/blob/main/README.md#this-...

What's the issue with Firefox's implementation? And this doesn't change the fact that HTTPSe is literally fully sunset.

Here's a small historical curiosity: In HTTPS Everywhere, the functionality used to be call "Block all HTTP requests" but when that functionality was changed (back in 2016 [0]) to not block connections to non-HTTPS hidden services, it was renamed to "Block all unencrypted requests".

I just checked, and indeed Tor Browser 11.5 does not block non-HTTPS connections to hidden services, even when in "HTTPS-Only" mode. Thankfully.

[0]: https://github.com/EFForg/https-everywhere/pull/4370

HTTPS Everywhere is discontinued by its creator and will be removed completely next year, because every major browser includes the functionality already anyways.

HTTPS Everywhere Chrome Web Store Github

[1] https://github.com/EFForg/https-everywhere/tree/master/src/chrome/content/rules

You should ask those writers or see if you find anything useful at their github page here

It's a ruleset for the HTTP Everywhere browser extension.

Maybe try upstream at https://github.com/EFForg/https-everywhere