Concurrent

Open-source projects categorized as Concurrent
Edit details
Language: + Rust + Go + C# + Scala + C++ + Ruby + Elixir + Python + TypeScript + PHP + Assembly + JavaScript
Topics: Concurrency Async Golang Parallel Go
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
Sponsored

Top 23 Concurrent Open-Source Projects

  • garnet

    Garnet is a remote cache-store from Microsoft Research that offers strong performance (throughput and latency), scalability, storage, recovery, cluster sharding, key migration, and replication features. Garnet can work with existing Redis clients.

    Project mention: A MySQL compatible database engine written in pure Go | news.ycombinator.com | 2024-04-09

    You would be surprised by performance of modern .NET :)

    Writing no-alloc is oftentimes done by reducing complexity and not doing "stupid" tricks that actually work against JIT and CoreLib features.

    For databases specifically, .NET is actually positioned very well with its low-level features (intrisics incl. SIMD, FFI, struct generics though not entirely low-level) and high-throughput GC.

    Interesting example of this applied in practice is Garnet[0]/FASTER[1]. Keep in mind that its codebase still consist of un-idiomatic C# and you can do way better by further simplification, but it already does the job well enough.

    [0] https://github.com/microsoft/garnet

    [1] https://github.com/microsoft/FASTER

  • sled

    the champagne of beta embedded databases

    Project mention: SableDb – a key/value store that uses RocksDB and Redis API (written in Rust) | news.ycombinator.com | 2024-04-04

    a few times, seems interesting. The author's also built a lot of other cool concurrency primitives for Rust as well.

    [0] https://github.com/spacejam/sled

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • FASTER

    Fast persistent recoverable log and key-value store + cache, in C# and C++.

    Project mention: A MySQL compatible database engine written in pure Go | news.ycombinator.com | 2024-04-09

    You would be surprised by performance of modern .NET :)

    Writing no-alloc is oftentimes done by reducing complexity and not doing "stupid" tricks that actually work against JIT and CoreLib features.

    For databases specifically, .NET is actually positioned very well with its low-level features (intrisics incl. SIMD, FFI, struct generics though not entirely low-level) and high-throughput GC.

    Interesting example of this applied in practice is Garnet[0]/FASTER[1]. Keep in mind that its codebase still consist of un-idiomatic C# and you can do way better by further simplification, but it already does the job well enough.

    [0] https://github.com/microsoft/garnet

    [1] https://github.com/microsoft/FASTER

  • ZIO

    ZIO — A type-safe, composable library for async and concurrent programming in Scala

    Project mention: The golden age of Kotlin and its uncertain future | news.ycombinator.com | 2024-01-11

  • dashmap

    Blazing fast concurrent HashMap for Rust.

    Project mention: StupidAlloc: what if memory allocation was bad actually | /r/rust | 2023-07-07

  • falcon

    A high-performance web server for Ruby, supporting HTTP/1, HTTP/2 and TLS. (by socketry)

  • broadway

    Concurrent and multi-stage data ingestion and data processing with Elixir

    Project mention: Switching to Elixir | news.ycombinator.com | 2023-11-09

    You can actually have "background jobs" in very different ways in Elixir.

    > I want background work to live on different compute capacity than http requests, both because they have very different resources usage

    In Elixir, because of the way the BEAM works (the unit of parallelism is much cheaper and consume a low amount of memory), "incoming http requests" and related "workers" are not as expensive (a lot less actually) compared to other stacks (for instance Ruby and Python), where it is quite critical to release "http workers" and not hold the connection (which is what lead to the creation of background job tools like Resque, DelayedJob, Sidekiq, Celery...).

    This means that you can actually hold incoming HTTP connections a lot longer without troubles.

    A consequence of this is that implementing "reverse proxies", or anything calling third party servers _right in the middle_ of your own HTTP call, is usually perfectly acceptable (something I've done more than a couple of times, the latest one powering the reverse proxy behind https://transport.data.gouv.fr - code available at https://github.com/etalab/transport-site/tree/master/apps/un...).

    As a consequence, what would be a bad pattern in Python or Ruby (holding the incoming HTTP connection) is not a problem with Elixir.

    > because I want to have state or queues in front of background work so there's a well-defined process for retry, error handling, and back-pressure.

    Unless you deal with immediate stuff like reverse proxying or cheap "one off async tasks" (like recording a metric), there also are solutions to have more "stateful" background works in Elixir, too.

    A popular background job queue is https://github.com/sorentwo/oban (roughly similar to Sidekiq at al), which uses Postgres.

    It handles retries, errors etc.

    But it's not the only solution, as you have other tools dedicated to processing, such as Broadway (https://github.com/dashbitco/broadway), which handles back-pressure, fault-tolerance, batching etc natively.

    You also have more simple options, such as flow (https://github.com/dashbitco/flow), gen_stage (https://github.com/elixir-lang/gen_stage), Task.async_stream (https://hexdocs.pm/elixir/1.12/Task.html#async_stream/5) etc.

    It allows to use the "right tool for the job" quite easily.

    It is also interesting to note there is no need to "go evented" if you need to fetch data from multiple HTTP servers: it can happen in the exact same process (even: in a background task attached to your HTTP server), as done here https://transport.data.gouv.fr/explore (if you zoom you will see vehicle moving in realtime, and ~80 data sources are being polled every 10 seconds & broadcasted to the visitors via pubsub & websockets).

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • arq

    Fast job queuing and RPC in python with asyncio and redis.

    Project mention: Future Plan for Arq | news.ycombinator.com | 2024-03-18

  • ElixirBooks

    List of Elixir books

    Project mention: How long did it take you to learn Elixir before advancing to Phoenix? | /r/elixir | 2023-06-07

    Go through getting started, maybe pick one of the starter books from this list but honestly the getting started guide probably covers all the same stuff.

  • oh

    A new Unix shell.

  • ultra-runner

    🏃⛰ Ultra fast monorepo script runner and build tool

  • haxmap

    Fastest and most memory efficient golang concurrent hashmap

  • fork

    A lightweight solution for running code concurrently in PHP

  • tofu

    Functional programming toolbox

    Project mention: Companies with interesting Scala projects | /r/scala | 2023-06-02

    Tinkoff has a strong Scala FP presense: - https://github.com/tofu-tf/tofu - https://github.com/tofu-tf/derevo - https://github.com/tofu-tf/typed-schema

  • libfork

    A bleeding-edge, lock-free, wait-free, continuation-stealing tasking library built on C++20's coroutines

  • stretto

    Stretto is a Rust implementation for Dgraph's ristretto (https://github.com/dgraph-io/ristretto). A high performance memory-bound Rust cache. (by al8n)

  • goconcurrentqueue

    Go concurrent-safe, goroutine-safe, thread-safe queue

  • libconcurrent

    :copyright: Concurrent Programming Library (Coroutine) for C11

  • util

    A collection of useful utility functions (by shomali11)

  • rubico

    [a]synchronous functional programming

  • r3bl-open-core

    TUI framework and developer productivity apps in Rust 🦀

    Project mention: [Media] TUI framework fans - I've just updated the r3bl_tui crate to v0.3.3. It now supports smart lists, color support (truecolor, ansi 256, grayscale) based on terminal capabilities, custom Markdown parser & syntax highlighter. It is inspired by React. https://crates.io/crates/r3bl_tui | /r/rust | 2023-04-22

  • ZoneTree

    ZoneTree is a persistent, high-performance, transactional, ACID-compliant ordered key-value database for NET. It can operate in memory or on local/cloud storage.

  • YACLib

    Yet Another Concurrency Library

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2024-04-09.

Concurrent related posts

Index

What are some of the best open-source Concurrent projects? This list will help you:

Project Stars
1 garnet 8,887
2 sled 7,723
3 FASTER 6,175
4 ZIO 3,981
5 dashmap 2,709
6 falcon 2,465
7 broadway 2,280
8 arq 1,902
9 ElixirBooks 1,345
10 oh 1,342
11 ultra-runner 1,186
12 haxmap 831
13 fork 788
14 tofu 515
15 libfork 450
16 stretto 391
17 goconcurrentqueue 358
18 libconcurrent 354
19 util 286
20 rubico 267
21 r3bl-open-core 266
22 ZoneTree 265
23 YACLib 245

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com