System Design: Twitter

• elasticsearch-mapper-attachments

  102 503 0.0 Java

  Discontinued Mapper Attachments Type plugin for Elasticsearch

  

Sometimes traditional DBMS are not performant enough, we need something which allows us to store, search, and analyze huge volumes of data quickly and in near real-time and give results within milliseconds. Elasticsearch can help us with this use case.

• murder

  1,344 11 10.0 Ruby

  Large scale server deploys using BitTorrent and the BitTornado library (by ervinb)

Let's design a Twitter like social media service, similar to services like Facebook, Instagram, etc.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• Apache Spark

  101 38,320 10.0 Scala

  Apache Spark - A unified analytics engine for large-scale data processing

  

Recording analytics and metrics is one of our extended requirements. As we will be using Apache Kafka to publish all sorts of events, we can process these events and run analytics on the data using Apache Spark which is an open-source unified analytics engine for large-scale data processing.

• Redis

  318 64,821 9.7 C

  Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps.

  

We can use solutions like Redis or Memcached and cache 20% of the daily traffic but what kind of cache eviction policy would best fit our needs?

• Memcached

  55 13,199 8.4 C

  memcached development tree

  

We can use solutions like Redis or Memcached and cache 20% of the daily traffic but what kind of cache eviction policy would best fit our needs?

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project