alertmanager-status VS thegreatsuspender

I dunno, I don't really mind self-hosting monitoring infrastructure. I basically pay for a website uptime checker to check that Alertmanager is working. If Alertmanager is down, obviously you have to manually check to see what else is down, but it doesn't fail open.

I wrote a little glue to make this straightforward for anyone else who uses Prometheus/Alertmanager: https://github.com/jrockway/alertmanager-status This ensures that the website check checks the health of the whole alerting pipeline; Prometheus has an always firing alert, Alertmanager is set to send that alert to alertmanager-status, and alertmanager-status starts failing its external health check if it isn't seeing that alert firing at the configured interval. If one of [Prometheus, Alertmanager, alertmanager-status] fails, then your website health check fails.

It is quite awkward that the output of "working" and "completely broken" alerting systems have the same visible effect -- no alerts.

For Prometheus users, I wrote alertmanager-status to let a third-party "website up?" monitoring server check your alertmanager: https://github.com/jrockway/alertmanager-status

(I also wrote one of the main Google Fiber monitoring systems back when I was at Google. We spent quite a bit of time on monitoring monitoring, because whenever there was an actual incident people would ask us "is this real, or just the monitoring system being down?" Previous monitoring systems were flaky so people were kind of conditioned to ignore the improved system -- so we had to have a lot of dashboards to show them that there was really an ongoing issue.)

Many things!

jsso2: Identity provider and authenticating proxy for your non-enterprise use cases. WebAuthn only, no passwords! I was tired of typing a password for things like Grafana and PGAdmin, and IP whitelisting my home Internet for things that didn't have built-in authentication. https://github.com/jrockway/jsso2

If I were starting from 0 today, I'd just use Dex and Envoy's built-in OAuth support. OAuth is overly complicated, requiring a bunch of configuration for each app, and a ton of code in each app... but it won. So use that.

jlog: I read a lot of log files in my day-to-day work and really like the idea of structured logs, but found them hard to read. jlog translates timestamps to my local time zone, lets me query them with jq, etc.: https://github.com/jrockway/json-logs Can't live without it, I use it many times every day, and have even convinced other people to use it without writing any documentation. (There are binary releases and a --help though!)

"kubectl jq": I wanted to play with writing Kubernetes plugins, so I made one that is just "kubectl get x -o json | jq". I use it pretty regularly, but the Kubernetes client machinery doesn't give you autocompletion for free, so it's pretty painful to use. When they fix that, I plan to write more kubernetes extensions (including one that invokes jlog on the logs, saving a pipe ;) https://github.com/jrockway/kubectl-jq

alertmanager-status: How do you know if your Prometheus/Alertmanager is working? If it breaks, it won't be sending you an alert, after all. https://github.com/jrockway/alertmanager-status

ekglue: The good parts of Istio, written by someone who read the xDS spec :P https://github.comjrockway/ekglue

For my day job, I work on Pachyderm Hub, which you should totally use if you want to run production-quality data science workloads (data provenance, reproducibility, etc.): https://hub.pachyderm.com/ I could write a lot about it, but basically... we have customers that want to use Pachyderm, but the complexity of Kubernetes stands in their way. How do you store logs? How do you monitor things? How do you give your coworkers access? We solve those problems by letting you click a button in a web UI. (As for why you'd want to use Pachyderm: https://www.pachyderm.com/use-cases/)

Happened in (2021)[https://github.com/greatsuspender/thegreatsuspender/issues/1...], and then a few others have forked the extension and tried to revive it, only to eventually sell to nefarious owners or sell user data themselves

I went to github and downloaded the last known "good version, installed it manually."

You want someone to die for disabling a potentially malicious extension that is unmantained since 2020?

Also if you want to read up on the removal of the app and the malware issues this post goes over it as well as other recovery options

Similar code projects have had issues like this before, like the open source Great Suspender.

What can happen after a project changes hands - https://github.com/greatsuspender/thegreatsuspender/issues/1263

Better link https://github.com/greatsuspender/thegreatsuspender/issues/1...

> TLDR: The old maintainer appears to have sold the extension to parties unknown, who have malicious intent to exploit the users of this extension in advertising fraud, tracking, and more. In v7.1.8 of the extension (published to the web store but NOT to GitHub), arbitrary code was executed from a remote server, which appeared to be used to commit a variety of tracking and fraud actions. After Microsoft removed it from Edge for malware, v7.1.9 was created without this code: that has been the code distributed by the web store since November, and it does not appear to load the compromised script. However, the malicious maintainer remains in control, however, and can introduce an update at any time. It further appears that, while v7.1.9 was what was listed on the store, those who had the hostile v7.1.8 installed did NOT automatically receive the malware-removing update, and continued running the hostile code until Google force-disabled the extension.

For what it is worth, you may have heard of the Great Suspender incident (https://github.com/greatsuspender/thegreatsuspender/issues/1263). It was used by millions, and was also open source on GitHub, but it could still end up becoming malicious.

Long ass comment: That is not true for the most part. While the increased amount of individuals working of an OSS project may lead to better vulnerability detection according to both parties of the closed-source/proprietary debate, it doesn't lead to a massively more secure software overall. Not all reviewers have the similar experience or expertise and, because of it, not everyone will be able to review, identify or patch any flaws or vulnerability of a specific software since it may require other skills beyond just basic programming skills such as network or cryptographic skills. [1] Some even suggested that the large number of users contributing to the project can lead people "into a false sense of security." [2] Overall, some papers conclude that being an open source software or a proprietary software isn't an important factor for security and suggest considering other factors, such as the particular vendor/maintainer that controls the entire process. [3] After all, what if the maintainer decides to sabotage their own code? What if the project was sold to another maintainer for its own shaddy needs?

If you're referring to The Great Suspender, that extension was bought by an advertising company earlier this year. I'm using the last good version (github) though.