framework
opendata.cern.ch
framework | opendata.cern.ch | |
---|---|---|
9 | 13 | |
1,857 | 636 | |
7.5% | 0.8% | |
9.9 | 9.2 | |
about 12 hours ago | 3 days ago | |
TypeScript | Python | |
ISC License | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
framework
- Observable Framework – The best dashboards are built with code
- Observable Framework 1.1
-
Interesting Ideas in Observable Framework
Thanks for the feedback. We have a PR open to make it easier to register new interpreters (without needing to fallback to .sh or .exe); it’ll let you specify the interpreter associated with a given file extension (e.g., .kts for Kotlin). https://github.com/observablehq/framework/pull/935
As for inputs-driving-data-loaders, that does go against the grain a bit since Framework favors static data snapshots so that the built site is self-contained and performant. But a technique that works well is to generate Parquet files in data loaders representing the superset of data that you want to interact with, and then using DuckDB/SQL in the client to extract the subset you want to visualize. This tends to perform well, though obviously it’s dependent on the size of the superset you want to interact with.
- Observable Framework: A static site generator for data apps, dashboards, reports
-
Observable 2.0, a static site generator for data apps
From the Observable Framework point of view, you’re very welcome to use Apache ECharts or any other library instead of Observable Plot, since you can import whatever you like and it’s all just JavaScript.
Since there was a lot of interest in this thread, Mike added a page to the docs with an ECharts example: https://observablehq.com/framework/lib/echarts
There are two pieces of that example code specific to Framework: the html`` tagged template literal creates a DOM element (see https://github.com/observablehq/htl, also usable outside Framework), and the display function inserts it into the document above the code block (see https://observablehq.com/framework/javascript/display). Note that, whereas Observable Plot takes an options object and returns a DOM element, ECharts instead takes a DOM element and mutates it — but in general they should be equally easy to use in Framework.
Like Plot (and Vega-Lite, another great option), ECharts is also now one of Framework’s built-in “recommended libraries” (see https://observablehq.com/framework/javascript/imports#implic...), meaning that if you reference `echarts` Framework will lazy-load it for you. Adding that was a two-line diff: https://github.com/observablehq/framework/pull/811/files#dif.... But I wanna emphasize that Framework doesn’t have to explicitly “support” a given library for you to use it. “Supporting” in this case just means the convenience of saving you a one-line import statement. But don’t wait for our blessing!! Use whatever.
opendata.cern.ch
-
Observable 2.0, a static site generator for data apps
I think the idea of Framework is really good, but static data limits the applications, excluding monitoring and other cases in which the data is constantly changing, but the dashboard can stay as it is. For example, I'd love to see a revamped Framework version of the LHC beam monitor and related pages (see https://op-webtools.web.cern.ch/vistar/, but check again in 2 months or so, when the accelerator will be running).
In high-energy physics, ROOT is /the/ toolkit for data analysis, and I guess jsROOT (https://root.cern.ch/js/) could also be used to load data to be shown in Framework dashboards. I thought the idea of Framework as a blogging engine with powerful data visualization built-in could be very interesting. Think, for example, about physicists pulling open data (https://opendata.cern.ch) and writing about their analysis or someone pulling data from https://ourworldindata.org/ in their own visualizations to support their case while writing about a particular subject, etc.
-
NFS > FUSE: Why We Built Our Own NFS Server in Rust
> XetHub has the world’s first natively cross-platform, user-mode filesystem implementation, allowing you to mount arbitrarily large datasets on your machine.
Not really world's first. CERN has developed EOS (https://eos-web.web.cern.ch/) for many years, and even though it's not available natively on Windows, it is available on Linux and macOS. EOS uses FUSE, though, not NFS.
> This enables you to, in just a few seconds, locally mount ~660 GB of Llama 2 models or write DuckDB queries to analyze large parquet files and scan just the data you need.
If you mount all instances of EOS at CERN on your machine with the FUSE client, that in principle mounts hundreds of PB of data from LHC experiments, although much of this data requires special permissions to be accessed. However, there's also a lot of open data. See https://opendata.cern.ch/.
- Are modern physicists dancing with the devil?
-
Good Series, Tutorial, or Book on Particle Physics Analysis using Python or Root for Undergraduates
CERN Open Data has lots of examples from various collaborations: https://opendata.cern.ch/
-
If you are in the process of building your data analytics project/portfolio, here's a useful video where you can find all the datasets you need
https://opendata.cern.ch/ - datasets from CERN if you're interested in particle physics. Lots of image data.
-
Why atheists behave so unscientific?
data from CERN
-
Es ce que les données récoltées sont disponible au public ?
See: https://opendata.cern.ch/
-
Before There Was Effective Altruism, There Was Effective Philanthropy
Huh? CERN publishes their data: https://opendata.cern.ch/ CERN is also pretty big on open source in general: https://home.cern/science/computing/open-source-open-science Again, the attitude seems to be, "There are times where we may not want to be 100% open, so let's assume there are good reasons it won't work for EA." I'm not saying everyone needs to publish their bank account numbers, passwords, and a video stream of the office bathroom. You can use sense and still be open.
-
[P] Official Imagen Website by Google Brain
CERN actually releases their data publicly and you are free to analyze them for yourself.
- What is the largest free data set that you know of?
What are some alternatives?
evidence - Business intelligence as code: build fast, interactive data visualizations in pure SQL and markdown
nfsserve - A Rust NFS Server implementation
owid-grapher - A platform for creating interactive data visualizations
awesome-public-datasets - A topic-centric list of HQ open datasets.
dataflow - An experimental self-hosted Observable notebook editor, with support for FileAttachments, Secrets, custom standard libraries, and more!
Mediawiki - 🌻 The collaborative editing software that runs Wikipedia. Mirror from https://gerrit.wikimedia.org/g/mediawiki/core. See https://mediawiki.org/wiki/Developer_access for contributing.
obsplot - Observable Plot bindings for R
data - Data and code behind the articles and graphics at FiveThirtyEight
datasette-dashboards - Datasette plugin providing data dashboards from metadata
wikdict-web - Web front end for WikDict dictionaries
pyobsplot - Observable Plot in Jupyter notebooks and Quarto documents
Herbie - Download numerical weather prediction datasets (HRRR, RAP, GFS, IFS, etc.) from NOMADS, NODD partners (Amazon, Google, Microsoft), ECMWF open data, and the University of Utah Pando Archive System.