if you had to restart at 0 knowledge what would you do?

This page summarizes the projects mentioned and recommended in the original post on /r/PinoyProgrammer

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  • Aeron

    Efficient reliable UDP unicast, UDP multicast, and IPC message transport

  • Java: In the past years C++ in finance has been rapidly supplanted by Java thanks to breakthrough technologies in the past decade like LMAX Disruptor, Chronicle Queue, Azul JVM, and Aeron (not the ergonomic chair, but this one, the transport protocol that breaks kafka performance records out of the park - not really a full kafka replacement, as Kafka enforces subscriber GD and aeron is more of an OSI layer 4 better than TCP; google "Best-effort delivery vs reliable delivery"). There's plenty more but thanks to these technologies, they allowed a Java based stack to perform the latency and throughput requirements needed for high frequency trading/HFT. From top trading firms like Two Sigma to the New York Stock Exchange, they're in Java. For banks, large modern western banks worth their salt and have modernized their systems are dominated by Java, especially thanks to Azul. To list a few banks, ING, Wells Fargo, Credit Suisse, and Barclays are all in Azul. Even at work Java still dominates.

  • Event Store

    EventStoreDB, the event-native database. Designed for Event Sourcing, Event-Driven, and Microservices architectures

  • C#: In Europe, Java is still strong but many trading firms use C# because of the strong Microsoft culture in Europe, as well as because of strongly supported C# libraries like say EventStore, which tends to be used for the matchmaking engines for stock exchanges (especially that exchange matchmaking problem is basically SMR). And skimming over the code, it has Paxos implemented too, making it good for dealing with partial failures (failover), essential for any HFT/trading firm. C#'s also the biggest ecosystem that many of the breakthrough java tech mentioned earlier was first ported to.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • JCTools

  • Install some tool that would help you see the performance of your system, like a graph of the CPU usage, the top processes being used, disk activity/read/write, etc. Every time you run your program, glance at those numbers, eventually you'll develop an intuition. Basically write code and profile. A good exercise would be practicing with data structures, this site has an exhaustive list of them, find some stuff that's interesting then google the implementation, then build it yourself, test it, debug, profile, optimize, and understand the performance constraints. Eventually you'll develop better understanding and can compare between other people's works, optimizing them. If you want to go beyond, read some papers on lock-free algorithms https://github.com/JCTools/JCTools/tree/master/resources then read Brendan Gregg's blog and books. Read about how profiling tools work https://github.com/andikleen/pmu-tools/wiki/toplev-manual

  • pmu-tools

    Intel PMU profiling tools

  • Install some tool that would help you see the performance of your system, like a graph of the CPU usage, the top processes being used, disk activity/read/write, etc. Every time you run your program, glance at those numbers, eventually you'll develop an intuition. Basically write code and profile. A good exercise would be practicing with data structures, this site has an exhaustive list of them, find some stuff that's interesting then google the implementation, then build it yourself, test it, debug, profile, optimize, and understand the performance constraints. Eventually you'll develop better understanding and can compare between other people's works, optimizing them. If you want to go beyond, read some papers on lock-free algorithms https://github.com/JCTools/JCTools/tree/master/resources then read Brendan Gregg's blog and books. Read about how profiling tools work https://github.com/andikleen/pmu-tools/wiki/toplev-manual

  • bips

    Bitcoin Improvement Proposals

  • Transactions: the most common construct in distributed systems are transactions, especially when you want to create atomic operations with commit/rollback. Learn ACID, DB recovery, (write ahead) logging/ARIES. Then learn to do distributed transactions, like learn about 2-phase commit to learn how to coordinate transactions. Maybe build a project that demands 2PC and similar approach such as digital wallets (or else you'll get problems like double spending), or study systems using it (ex. Apache Flink internals). With 2PC, this ties us nicely into the next one being Consensus...

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • JEP Draft: Deprecate Memory-Access Methods in Sun.misc.Unsafe for Removal

    3 projects | news.ycombinator.com | 16 Jan 2024
  • LMAX Disruptor – High Performance Inter-Thread Messaging Library

    1 project | /r/hackernews | 20 Nov 2023
  • LMAX Disruptor – High Performance Inter-Thread Messaging Library

    4 projects | news.ycombinator.com | 17 Nov 2023
  • Disruptor 4.0.0 Released

    1 project | news.ycombinator.com | 29 Sep 2023
  • About Pool Sizing

    1 project | news.ycombinator.com | 5 Sep 2023