rudderstack-docs VS nodejs-bigquery

2/ Featured tools this week - Transform and RudderStack

RudderStack is an open-source Customer Data Pipeline that allows you to track and send real-time events from your web, mobile, and server-side sources to your entire customer data stack. Our primary repository - rudder-server - is open-sourced on GitHub.

This post will explain how your customer data pipeline can help improve your data privacy and how to ensure your data privacy with RudderStack.

However, limitations to traditional CDPs, especially around connecting to best-of-breed customer tooling and exposing data for use across an organization have driven a new generation of non-CDPs. Solutions like Snowplow's (website, GitHub) data delivery platform and RudderStack's (website, GitHub) customer data platform for developers ingest data from a multitude of sources, apply in-stream transformations, and route data to your data warehouse, like Snowplow, or your warehouse plus your preferred customer tooling destinations for activation, like RudderStack.

I learned many lessons from this journey - lessons that deserve a post of their own - but there's one lesson that I learned early on that stands out. In this blog, I talk about why we merged Blendo with RudderStack, building the team and working together to build a great product.

In fact, developers have already started contributing to our documentation. Recently, Benedikt from the Userlist team created the docs for the Userlist destination for RudderStack (see the pull request here). They also built the Userlist integration, submitted a pull request, and it is now live on our platform! This is the beauty of open source!

TLDR: general steps/starting point for setting up an app with Rudderstack ( or Segment) to track customer events

As a part of our SDK roadmap at RudderStack, we wanted to develop a Flutter SDK. Our existing SDKs include features such as storing event details and persisting user details on the database, and much more. However, these features are already implemented in our Android and iOS SDKs.

To load Stripe data into Postgres, you can use platforms such as Stitch Data and Rudderstack. In this guide, we will use Stitch Data because it is a cheap and a fast solution.

With your Tracking Plans in place, you can use the existing Data Governance API's to evaluate your inbound events, payload samples and metadata to compare them against your plans. You can also use the RudderTyper tool we're releasing alongside Tracking Plans. RudderTyper is a tool for generating strongly-typed RudderStack analytics library wrappers based on your published tracking plan specs, meaning your data will conform to your defined schema upon capture.

Amazon Redshift still remains a bit of a mystery to me, even after a whole session on it unpacking loads of its features, possibilities and use cases. Trying to draw some parallels in my head with BigQuery - Google Cloud Platform's own cloud data wharehouse service, which I know well - also didn't help much. So it remains one of those things that now I know a little bit more about than yesterday, but still feels like I haven't even begun to scratch its surface. So one to watch for me, and learn more about. The use case presented by National Rail was great and full of detail too.

With data warehouse solutions (BigQuery, Snowflake, Redshift) going mainstream, modern data stacks are becoming increasingly boring - great news if you're starting from scratch!

If you wanted to upgrade that idea, track down articles relating to that swing for discussion and post those. There is definite value in that data, and it is a pretty simple thing to do. You are just using a Cloud Composer to ingest the data and storing it in a data warehouse like BigQuery or Snowflake, creating a Twitter bot to post outputs to Twitter using something like Airflow.

look into BigQuery. that's the best way to handle it. Especially if you're familiar with sql. It's super smooth to integrate with sheets using the bigQuery API which makes it great.

Data Warehouse - BigQuery

(Our data stack: Fivetran, BigQuery, dbt, dbt Cloud, Looker, Datafold)

The main DWH offerings that meet the above expectations are Snowflake, Google Bigquery, and Amazon Redshift. Featurewise, these three have similar functionalities but there are differences, e.g. how long it takes to spin up new compute resources or how much maintenance work they need. Costwise, it seems they end up with similar numbers on your bill, depending on which blog post you read.

BigQuery

Although we’ve focused on Snowflake in this article, the same features of Datafold can be used for other cloud products like Redshift or BigQuery.

Working on a new project using DBT, Google BigQuery & MongoDB