alibi-detect VS logseq

Numerous tools exist for detecting anomalies in time series data, but Alibi Detect stood out to me, particularly for its capabilities and its compatibility with both TensorFlow and PyTorch backends.

That is, to see whether a test image belongs in the distribution of the training images and to provide a routine for special cases. After a bit of reading Ive found that this is related to the field of drift detection in which I tried out alibi-detect . Whereby the training images are trained by an autoencoder and any subsequent drift will be flagged by the AE.

Wasserstein distance is not very suitable for drift detection on most problems given that the sample complexity (and estimation error) scales with O(n^(-1/d)) with n the number of instances (100k-10m in your case) and d the feature dimension (192 in your case). More interesting will be to use for instance a detector based on the maximum mean discrepancy (MMD) with estimation error of O(n^(-1/2)). Notice the absence of the feature dimension here. You can find scalable implementations in Alibi Detect (disclosure: I am a contributor): MMD docs, image example. We just added the KeOps backend for the MMD detector to scale and speed up the drift detector further, so if you install from master, you can leverage this backend and easily scale the detector to 1mn instances on e.g. 1 RTX2080Ti GPU. Check this example for more info.

Seldon | Multiple positions | London/Cambridge UK | Onsite/Remote | Full time | seldon.io

At Seldon we are building industry leading solutions for deploying, monitoring, and explaining machine learning models. We are an open-core company with several successful open source projects like:

* https://github.com/SeldonIO/seldon-core

* https://github.com/SeldonIO/mlserver

* https://github.com/SeldonIO/alibi

* https://github.com/SeldonIO/alibi-detect

* https://github.com/SeldonIO/tempo

We are hiring for a range of positions, including software engineers(go, k8s), ml engineers (python, go), frontend engineers (js), UX designer, and product managers. All open positions can be found at https://www.seldon.io/careers/

I work in this area and also contribute to outlier/drift detection library https://github.com/SeldonIO/alibi-detect. To tackle this type of problem, I would strongly encourage following a more principled, fundamentally (statistically) sound approach. So for instance measuring metrics such as the KL-divergence (or many other f-divergences) will not be that informative since it has a lot of undesirable properties for the problem at hand (in order to be informative requires already overlapping distributions P and Q, it is asymmetric, not a real distance metric, will not scale well with data dimensionality etc). So you should probably look at Integral Probability Metrics (IPMs) such as the Maximum Mean Discrepancy (MMD) instead which have much nicer behaviour to monitor drift. I highly recommend the Interpretable Comparison of Distributions and Models NeurIPS workshop talks for more in-depth background.

All of the above functionality and more can be easily used under a simple API in https://github.com/SeldonIO/alibi-detect.

Logseq support via our Logseq Plugin

Sorry, but _what exactly_ «it seems to do» from your point of view?

My «second brain» now is almost 300Mb of text, pictures, sound files, PDF and other stuff. As I already mentioned, it contains tables, mathematical formulae, sheet music, cross-references, code samples, UML diagrams and graphs in Graphviz format. It is versioned, indexed by local search engine, analyzed by AI assistant and shared between many computers and mobile devices. And (last but not least) it works: it allows me to solve my tasks way more faster than with the assistant of external, non-personalized tools (like ChatGPT, StackExchange or Google).

I know no tools for all this tasks except org-mode. Well, maybe Evernote in the 2010-s was something similar — but with less features, with more bugs and with worse interface.

Personal note-taking _is_ a complex task per se (well, at least for someone like typical HN visitor). I've seen many note-taking tools, that were ridiculously featureless, stupid and inconvenient because they were _not_ complex enough.

> Sure if one wants to do emacs-gardening it is fine.

1)You can use org-mode outside Emacs. See for example Logseq (https://logseq.com/), organice (https://organice.200ok.ch/) or EasyOrg.

2)Org-mode works in Emacs out of the box, you don't need any «emacs-gardening» to use org-mode.

3)The term «Emacs-gardening» itself sound a bit like hate-speech for me. The complexity of Emacs customization is overrated, mostly due to opinions of people who never used Emacs or used it in the previous millennium.

Obsidian is great.

For those looking for an open source alternative (or don't want to pay the Obsidian fees for professional usage) check out Logseq: https://logseq.com/

For an opensource alternative to Obsidian checkout Logseq (1). I spent a while thinking obsidian was opensource out of my own ignorance and was disappointed when I learned it was not.

1: https://logseq.com/

I use logseq to keep journal of my daily work.

While Emacs and Org mode can certainly be used for this (and, when they can't, you can always inject little python/js scripts in your emacs config to take care of specific things), I'd also recommend you take a look at Logseq.