Redis VS cockroach

The AWS Lambda function had been written in Java and therefore the team looked at different Java-based Redis drivers and eventually decided to use Jedis, a Java client for Redis designed for performance and ease of use. One of the reasons they chose to use Jedis was that it proved to be a lightweight library. Instantiating the client connection takes somewhere between 200-600ms (depending on the lambda memory settings) which impacts the cold start of the lambda function.

Caching - while it's not possible to cache everything, there's always a large percentage of your website / app that can be cached for an hour or ten minutes or 1 day etc... - all depends on the type of content but the longer you can cache for without negatively effecting content quality - the better. A good caching server example would be redis : https://redis.io/ or https://memcached.org/

The code uses the codehooks-js library to handle routing and middleware for the API, and the node-fetch library to make HTTP requests to the OpenAI API. The code also uses the built-in codehooks.io Key-Value store to cache the response from the OpenAI API for 60 seconds, in order to reduce the number of requests made to the OpenAI API (If you want to create your own Node.js Express backend, you just need to replace this cache with Redis for example).

Redis is an in-memory database and an open-source streaming engine with an IMDB data structure server that supports multiple data sets and data streams. This open-source platform has high throughputs and lower bandwidth due to its in-memory features that lead to faster processing of big data in data science applications.

Quirrel is a popular open-source webhook framework that makes it easy to add webhook support to your app. It makes uses of redis under the hood.

Redis

Redis is an in-memory data store that you can use as a database – or a cache in front of your regular database. The speed of Redis makes it ideal to cache database queries, complex computations, API calls, and session state.

I didn't finish college and I didn't see anything about Databases when I was there, so I don't have any theoretical background and on these 4 years coding at enterprise level my focus was totally in things like: Clean Code, SOLID, Design Patterns and other stuff code related and my only Database knowledge was about SQL/MySQL and Cache/Redis.

For the course I started to collect Open Source projects developed by corporations. There are a few where the product is open source such as Redis and Elastic. There are others where the company shares some of its code as open source such as Netflix, Facebook, or Booking.com.

The answer is yes. Introducing Redis, an open source database, in-memory, key-value data store and it is most-loved database by developers for five years running. Redis is commonly used as a primary database, cache, queue, and message broker. Redis is an open source (BSD licensed) that is used primarily as an application cache and quick response database and because the data is stored in memory rather than on a disk, Redis delivered unparalleled speed, reliability, and performance.

Spencer Kimball (now CEO at CockroachDB) wrote an interesting article on this topic in 2021 where they created spencerkimball/stargazers based on a Python script. So I started thinking: could I create a data pipeline using Nifi and Kafka (two OSS tools often used with Druid) to get the API data into Druid - and then use SQL to do the analytics? The answer was yes! And I have documented the outcome below. Here’s my analytical pipeline for Github stars data using Nifi, Kafka and Druid.

One of the most popular distributed DB is built in Go: https://www.cockroachlabs.com/

Recently, I worked on such an integration with Splunk. The Splunk dashboard files that emulate the DB Console are now available in our repo for everyone's benefit.

Technically not postgres, but could be worth checking out cockroach db, it can use any postgres driver from programming languages and is build for distribution. I recently moved from postgres to cockroach (with golang postgres driver) and didnt need to change a single query

CockroachDB

Cockroachdb is written in go if you need some inspiration: https://github.com/cockroachdb/cockroach