Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
Top 23 parallel-computing Open-Source Projects
-
For parallel programming nowadays, personally I reach for C++ Taskflow when I really care about performance, or a mix of core.async and running multiple load balanced instances when I’m doing more traditional web backend stuff in Clojure.
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
Project mention: Distil-Whisper: distilled version of Whisper that is 6 times faster, 49% smaller | news.ycombinator.com | 2023-10-31
Just a point of clarification - faster-whisper references it but ctranslate2[0] is what's really doing the magic here.
Ctranslate2 is a sleeper powerhouse project that enables a lot. They should be up front and center and get the credit they deserve.
-
swifter
A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner (by jmcarpenter2)
-
kokkos
Kokkos C++ Performance Portability Programming Ecosystem: The Programming Model - Parallel Execution and Memory Abstraction
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
awesome-machine-learning-in-compilers
Must read research papers and links to tools and datasets that are related to using machine learning for compilers and systems optimisation
-
Arraymancer
A fast, ergonomic and portable tensor library in Nim with a deep learning focus for CPU, GPU and embedded devices via OpenMP, Cuda and OpenCL backends
It is a small DSL written using macros at https://github.com/mratsim/Arraymancer/blob/master/src/array....
Nim has pretty great meta-programming capabilities and arraymancer employs some cool features like emitting cuda-kernels on the fly using standard templates depending on backend !
-
-
Project mention: I made a Python package to do adaptive sampling of functions in parallel [OC] | /r/compsci | 2023-04-29
Yes! Check it out at https://github.com/python-adaptive/adaptive/
-
-
-
Kratos
Kratos Multiphysics (A.K.A Kratos) is a framework for building parallel multi-disciplinary simulation software. Modularity, extensibility and HPC are the main objectives. Kratos has BSD license and is written in C++ with extensive Python interface. (by KratosMultiphysics)
-
-
-
GRASS GIS offers powerful raster, vector, and geospatial processing engines in a single integrated software suite. It includes tools for terrain and ecosystem modeling, hydrology, visualization of raster and vector data, management and analysis of geospatial data, and the processing of satellite and aerial imagery. It comes with a temporal framework for advanced time series processing and a Python API for rapid geospatial programming. GRASS GIS has been optimized for performance and large geospatial data analysis.
-
https://github.com/topics/datalog?l=rust ... Cozo, Crepe
Crepe: https://github.com/ekzhang/crepe :
> Crepe is a library that allows you to write declarative logic programs in Rust, with a Datalog-like syntax. It provides a procedural macro that generates efficient, safe code and interoperates seamlessly with Rust programs.
Looks like there's not yet a Python grammar for the treeedb tree-sitter: https://github.com/langston-barrett/treeedb :
> Generate Soufflé Datalog types, relations, and facts that represent ASTs from a variety of programming languages.
Looks like roxi supports n3, which adds `=>` "implies" to the Turtle lightweight RDF representation: https://github.com/pbonte/roxi
FWIW rdflib/owl-rl: https://owl-rl.readthedocs.io/en/latest/owlrl.html :
> simple forward chaining rules are used to extend (recursively) the incoming graph with all triples that the rule sets permit (ie, the “deductive closure” of the graph is computed).
ForwardChainingStore and BackwardChainingStore implementations w/ rdflib in Python: https://github.com/RDFLib/FuXi/issues/15
Fast CUDA hashmaps
Gdlog is built on CuCollections.
GPU HashMap libs to benchmark: Warpcore, CuCollections,
https://github.com/NVIDIA/cuCollections
https://github.com/NVIDIA/cccl
https://github.com/sleeepyjack/warpcore
/? Rocm HashMap
DeMoriarty/DOKsparse:
-
-
-
-
Hyperactive
An optimization and data collection toolbox for convenient and fast prototyping of computationally expensive models.
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
parallel-computing related posts
- Forkrun: Runs multiple inputs through a command in parallel using bash coprocs
- Forkrun – A pure-bash function for parallelizing loops
- Symbolics.jl
- Minimal implementation of Mamba, the new LLM architecture, in 1 file of PyTorch
- Coarrays
- Awesome research papers on ML in Compilers
- I made a Python package to do adaptive learning of functions in parallel [P]
-
A note from our sponsor - InfluxDB
www.influxdata.com | 17 Apr 2024
Index
What are some of the best open-source parallel-computing projects? This list will help you:
Project | Stars | |
---|---|---|
1 | Taskflow | 9,496 |
2 | Joblib | 3,638 |
3 | CTranslate2 | 2,750 |
4 | swifter | 2,456 |
5 | kokkos | 1,712 |
6 | mfem | 1,530 |
7 | Vc | 1,413 |
8 | awesome-machine-learning-in-compilers | 1,319 |
9 | Arraymancer | 1,298 |
10 | Symbolics.jl | 1,285 |
11 | adaptive | 1,107 |
12 | elmerfem | 1,085 |
13 | pyopencl | 1,030 |
14 | Kratos | 953 |
15 | future | 930 |
16 | accelerate | 886 |
17 | grass | 763 |
18 | cccl | 737 |
19 | dolfinx | 648 |
20 | oneMKL | 562 |
21 | OpenTimer | 508 |
22 | Hyperactive | 487 |
23 | post-me | 480 |