meltano VS supabase

- https://github.com/meltano/meltano

We'd love to hear what you think of Meltano (Cloud). If you join the Beta, you get 100 free credits (200 hourly or 100 daily runs) and a 20% discount on the pricing at GA (June 27). The first 100 to sign up get 1,000 credits -- that's 83 days of hourly runs or 3 years of dailies!

The team and I will be checking in here throughout the day, so don't hesitate to ask questions! If we don't get to you, feel free to join 3,500+ Meltano fans on https://meltano.com/slack and we'll chat there!

Meltano[0] might be of interest to you. Easy way to move data that should be very familiar for software engineers. If a connector doesn't exist our SDK makes it easy to build it.

[0] https://github.com/meltano/meltano

(disclaimer - I work at Meltano)

We currently don't do any process optimization on a per-stream basis when doing an extract. We have seen folks in the community running each tap separately for each stream which can speed it up. We've got an issue around this (Melturbo).

Meltano

I built my data pipeline to Extract some data from websites and CSV files, Load it into my database, and Transform it into a reporting-ready schema. I used Python and Pandas to extract and load some of the data and Meltano to load some additional supporting data. All of that data went into a PostgreSQL database hosted in the cloud on Azure where I then used dbt to create data models in the database optimized for reporting. Finally, I use Metabase to visualize the data. (whew! that's a lot of moving parts!)

I'm learning https://meltano.com/ right now, so am building custom Taps, mostly for fun. I'm enjoying it. I'm pulling in a variety of data from https://www.geonames.org/ and Canadian weather/climate data into BigQuery

https://meltano.com/ is ELT but I like it

It was a great experience using Supabase’s rock-solid PostgreSQL database for this app. The DX around that product is phenomenal: viewing and managing the DB data was a lifesaver when you don’t want to craft your own admin panel from scratch.

I didn't really give much thought as to which backend I would use. I already had 2 projects in Supabase (BOXCUT & MineWork), but also a few projects in Firebase too. I was more concerned at the time at actually building the product.

Sign up for SupaBase: Head over to SupaBase and sign up. Create a new workspace and project with your preferred names.

Setting up Supabase Create a new Supabase project, and get the connection string for the database from settings > database.

Supabase (start for free)

Built with Supabase, Astro, Unreal Speech, Stable Diffusion, Replicate, Metropolitan Museum of Art

Supabase will be used for storing article data in the database and the cover image of the article in storage. Chakra UI will be used to provide style to the elements. By using both, we can build the blog with ease.

For AutoRepurpose, I opted for Supabase as the backbone of the backend. It has reliably supported Penelope AI, which garnered over 15k users in 2022 without any issues.

Semantic search demo

1. Create a new Supabase project: Visit Supabase and create a new project.