opendata.cern.ch VS awesome-public-datasets

I think the idea of Framework is really good, but static data limits the applications, excluding monitoring and other cases in which the data is constantly changing, but the dashboard can stay as it is. For example, I'd love to see a revamped Framework version of the LHC beam monitor and related pages (see https://op-webtools.web.cern.ch/vistar/, but check again in 2 months or so, when the accelerator will be running).

In high-energy physics, ROOT is /the/ toolkit for data analysis, and I guess jsROOT (https://root.cern.ch/js/) could also be used to load data to be shown in Framework dashboards. I thought the idea of Framework as a blogging engine with powerful data visualization built-in could be very interesting. Think, for example, about physicists pulling open data (https://opendata.cern.ch) and writing about their analysis or someone pulling data from https://ourworldindata.org/ in their own visualizations to support their case while writing about a particular subject, etc.

> XetHub has the world’s first natively cross-platform, user-mode filesystem implementation, allowing you to mount arbitrarily large datasets on your machine.

Not really world's first. CERN has developed EOS (https://eos-web.web.cern.ch/) for many years, and even though it's not available natively on Windows, it is available on Linux and macOS. EOS uses FUSE, though, not NFS.

> This enables you to, in just a few seconds, locally mount ~660 GB of Llama 2 models or write DuckDB queries to analyze large parquet files and scan just the data you need.

If you mount all instances of EOS at CERN on your machine with the FUSE client, that in principle mounts hundreds of PB of data from LHC experiments, although much of this data requires special permissions to be accessed. However, there's also a lot of open data. See https://opendata.cern.ch/.

CERN Open Data has lots of examples from various collaborations: https://opendata.cern.ch/

https://opendata.cern.ch/ - datasets from CERN if you're interested in particle physics. Lots of image data.

data from CERN

See: https://opendata.cern.ch/

Huh? CERN publishes their data: https://opendata.cern.ch/ CERN is also pretty big on open source in general: https://home.cern/science/computing/open-source-open-science Again, the attitude seems to be, "There are times where we may not want to be 100% open, so let's assume there are good reasons it won't work for EA." I'm not saying everyone needs to publish their bank account numbers, passwords, and a video stream of the office bathroom. You can use sense and still be open.

CERN actually releases their data publicly and you are free to analyze them for yourself.

Merry Christmas buddy.

You'll find a ton of public datasets on GitHub [1].

Maven Analytics offers a monthly data analytics challenge [2] that you can enter for free. See their past competitions for some interesting datasets.

As I'm based in Ireland I'll also recommend the Irish Data Portal [3].

[1] https://github.com/awesomedata/awesome-public-datasets

The “data.csv” file contains historical weather data from Tel Aviv, sourced from another article I wrote. If you wish, you can swap it with a public dataset, for example, from here.

If you want something personal and fun, you will need lots of different data. As such, you can take a look at these publicly available datasets. Maybe you can find out some interesting relationships.

Some others I use: https://data.world/search This github project with links: https://github.com/awesomedata/awesome-public-datasets Data.fivethirtyeight.com r/datasets and similar subreddits can also be of help.

There's a lot in here; it's a good start.

Github: Awesome Public Datasets I didn't find much of use here for me, as much of this was either very specialized or very large datasets. But maybe you'll find something of use, here.