opendata.cern.ch
nfsserve
| opendata.cern.ch | nfsserve | |
|---|---|---|
| 13 | 2 | |
| 635 | 527 | |
| 0.6% | 3.6% | |
| 9.2 | 6.1 | |
| 11 days ago | about 1 month ago | |
| Python | Rust | |
| GNU General Public License v3.0 only | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
opendata.cern.ch
-
Observable 2.0, a static site generator for data apps
I think the idea of Framework is really good, but static data limits the applications, excluding monitoring and other cases in which the data is constantly changing, but the dashboard can stay as it is. For example, I'd love to see a revamped Framework version of the LHC beam monitor and related pages (see https://op-webtools.web.cern.ch/vistar/, but check again in 2 months or so, when the accelerator will be running).
In high-energy physics, ROOT is /the/ toolkit for data analysis, and I guess jsROOT (https://root.cern.ch/js/) could also be used to load data to be shown in Framework dashboards. I thought the idea of Framework as a blogging engine with powerful data visualization built-in could be very interesting. Think, for example, about physicists pulling open data (https://opendata.cern.ch) and writing about their analysis or someone pulling data from https://ourworldindata.org/ in their own visualizations to support their case while writing about a particular subject, etc.
-
NFS > FUSE: Why We Built Our Own NFS Server in Rust
> XetHub has the world’s first natively cross-platform, user-mode filesystem implementation, allowing you to mount arbitrarily large datasets on your machine.
Not really world's first. CERN has developed EOS (https://eos-web.web.cern.ch/) for many years, and even though it's not available natively on Windows, it is available on Linux and macOS. EOS uses FUSE, though, not NFS.
> This enables you to, in just a few seconds, locally mount ~660 GB of Llama 2 models or write DuckDB queries to analyze large parquet files and scan just the data you need.
If you mount all instances of EOS at CERN on your machine with the FUSE client, that in principle mounts hundreds of PB of data from LHC experiments, although much of this data requires special permissions to be accessed. However, there's also a lot of open data. See https://opendata.cern.ch/.
- Are modern physicists dancing with the devil?
-
Good Series, Tutorial, or Book on Particle Physics Analysis using Python or Root for Undergraduates
CERN Open Data has lots of examples from various collaborations: https://opendata.cern.ch/
-
If you are in the process of building your data analytics project/portfolio, here's a useful video where you can find all the datasets you need
https://opendata.cern.ch/ - datasets from CERN if you're interested in particle physics. Lots of image data.
-
Why atheists behave so unscientific?
data from CERN
-
Es ce que les données récoltées sont disponible au public ?
See: https://opendata.cern.ch/
-
Before There Was Effective Altruism, There Was Effective Philanthropy
Huh? CERN publishes their data: https://opendata.cern.ch/ CERN is also pretty big on open source in general: https://home.cern/science/computing/open-source-open-science Again, the attitude seems to be, "There are times where we may not want to be 100% open, so let's assume there are good reasons it won't work for EA." I'm not saying everyone needs to publish their bank account numbers, passwords, and a video stream of the office bathroom. You can use sense and still be open.
-
[P] Official Imagen Website by Google Brain
CERN actually releases their data publicly and you are free to analyze them for yourself.
- What is the largest free data set that you know of?
nfsserve
-
Mounting Git commits as folders with NFS
Very nice! I really like the design choices here. Though I would personally have used Rust and https://github.com/xetdata/nfsserve.
- NFS > FUSE: Why We Built Our Own NFS Server in Rust
What are some alternatives?
awesome-public-datasets - A topic-centric list of HQ open datasets.
file-system-stress-testing - A tool that can be used to stress test POSIX filesystems.
Mediawiki - 🌻 The collaborative editing software that runs Wikipedia. Mirror from https://gerrit.wikimedia.org/g/mediawiki/core. See https://mediawiki.org/wiki/Developer_access for contributing.
fuse-t
data - Data and code behind the articles and graphics at FiveThirtyEight
bb-adrs - Architecture Decision Records for Buildbarn
wikdict-web - Web front end for WikDict dictionaries
Herbie - Download numerical weather prediction datasets (HRRR, RAP, GFS, IFS, etc.) from NOMADS, NODD partners (Amazon, Google, Microsoft), ECMWF open data, and the University of Utah Pando Archive System.
fatcat-scholar - search interface for scholarly works
nflscrapR-data - Data files (.csv) accessed with nflscrapR and summarized at the player-level
Data-Science-For-Beginners - 10 Weeks, 20 Lessons, Data Science for All!