-
BIG-bench
Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
It is voluntary. The convention is used in BIG-bench [1] and mentioned in their README here:
> All task files (including README.md and task.json) contain a "canary" string, which should not be edited. This is to prevent benchmark tasks from leaking into web-scraped training data.
It's not universally adopted and of course anyone is free to ignore it.
[1] https://github.com/google/BIG-bench#creating-a-json-task
https://github.com/danluu/post-mortems is a fun read and has been around for a while.