Hacker News API VS distroless

I built a plugin that allows Coze bots to make requests to the Hacker News API.

I recognise the huge amounts of effort involved in this and I applaud the moderators for keeping HN an interesting place to be.

That said, I think it's reasonable for us to have visibility on their manual interventions, and this could be easily surfaced via the Hacker News API (https://github.com/HackerNews/API), if the Story JSON included the values of "contro," "bury," and "gag" fields, which are currently opaque to users of the API

See https://medium.com/hacking-and-gonzo/how-hacker-news-ranking... for more discussion on terminology

Google probably stopped updating BigQuery when they started hosting live Hacker News data in Firebase: https://github.com/HackerNews/API

The live nature of the Firebase data is awesome, but the lack of ability to query is a loss.

Nice work! I did something similar with a personal project a few months ago using an open source llm. Also, not sure if you know, there is an api you can use. https://github.com/HackerNews/API

Hi HN, I made a live feed for viewing all of the new items on Hacker News in almost-realtime. It is a single, modern HTML file that doesn't use polling.

It works by establishing a websocket connection to the Hacker News Firebase database, to receive updates every time the HN server updates Firebase, which is about once every 30 seconds. This is very efficient, putting no load on HN's server and using minimal bandwidth.

To make the feed continuous despite the delay, it waits to display each item until exactly 30 seconds before displaying it. I think this is a fair tradeoff, it gives you a sense of how active HN is. For comparision, there are something like 6-7 thousand tweets every second.

Official HN Firebase API: https://github.com/HackerNews/API

Source code: https://github.com/jerbear2008/hn-live/blob/main/index.html

You could use the API: https://github.com/HackerNews/API

You can now get Hacker News data in real time from the Hacker News API powered by Firebase: https://github.com/HackerNews/API

This is great (real time!), but also kind of a pain (38+ millions individual http requests to get the whole thing).

Thankfully there's no authentication or apparent rate limiting. I fumbled my way through downloading the whole thing with curl. I screwed up a few times so made over 70 million requests in total.

Toy analysis of the data I downloaded here: https://public.tableau.com/app/profile/isna/viz/HackerNewsDa...

Get your submissions from this API

https://hacker-news.firebaseio.com/v0/user/ohjeez.json?print...

and then scan the "submitted" articles as described here

https://github.com/HackerNews/API

I have a crawler that sucks down all the posts from HN and then I read it into Pandas and write all sorts of queries. The boggle I have now is that I want to use the same system to (1) make sure YOShInOn never submits duplicate articles, and (2) have accurate vote and comment scores. (1) requires picking up articles as soon as possible, (2) requires waiting two weeks or so until the scores have settled down to what they are going to be. I guess I gotta come back and rescan things in 2 weeks so I have the right scores.

From looking at the Hacker News API, you might be able to use that instead of scraping from the DOM: https://github.com/HackerNews/API#users

lots of questions here regarding what this product is. I guess i can provide some information for the context, from a perspective of an outside contributor.

Chainguard Images is a set of hardened container images.

They were built by the original team that brought you Google's Distroless (https://github.com/GoogleContainerTools/distroless)

However, there were few problems with Distroless:

1. distroless were based on Debian - which in turn, limited to Debian's release cadence for fixing CVE.

2. distroless is using bazelbuild, which is not exactly easy to contrib, customize, etc...

3. distroless images are hard to extend.

Chainguard built a new "undistro" OS for container workload, named Wolfi, using their OSS projects like melange (for packaging pkgs) and apko (for building images).

The idea is (from my understanding) is that

1. You don't have to rely on upstream to cut a release. Chainguard will be doing that, with lots of automation & guardrails in placed. This allow them to fix vulnerabilties extremely fast.

> If you have one image based on Ubuntu in your stack, you may as well base them all on Ubuntu, because you only need to download (and store!) the common base image once

This is only true if your infrastructure is static. If your infrastructure is highly elastic, image size has an impact on your time to scale up.

Of course, there are better choices than Alpine to optimize image size. Distroless (https://github.com/GoogleContainerTools/distroless) is a good example.

The same as our code dependencies, container updates can include security patches and bug fixes and improvements. However, they can also include breaking changes and it is crucial you test them thoroughly before putting them into production. Wherever possible, I recommend using the distroless base image which will drastically reduce both your image size, your risk vector, and therefore your maintenance version going forward.

# Use a large Node.js base image to build the application and name it "build" FROM node:18-alpine as build WORKDIR /app # Copy the package.json and package-lock.json files into the working directory before copying the rest of the files # This will cache the dependencies and speed up subsequent builds if the dependencies don't change COPY package*.json /app # You might want to use yarn or pnpm instead RUN npm install COPY . /app RUN npm run build # Instead of using a node:18-alpine image, we are using a distroless image. These are provided by google: https://github.com/GoogleContainerTools/distroless FROM gcr.io/distroless/nodejs:18 as prod WORKDIR /app # Copy the built application from the "build" image into the "prod" image COPY --from=build /app/.output /app/.output # Since this image only contains node.js, we do not need to specify the node command and simply pass the path to the index.mjs file! CMD ["/app/.output/server/index.mjs"]

Lots of examples without the entire OS as other comments mention, an example would be Googles distroless[0]

[0]: https://github.com/GoogleContainerTools/distroless

Docker doesn't do this all the time. Distroless Docker containers are relatively common. https://github.com/GoogleContainerTools/distroless

Deployment: https://github.com/GoogleContainerTools/distroless

Or use distroless image as it includes one, among others. https://github.com/GoogleContainerTools/distroless/blob/main/base/README.md