frostdb
parquet-go
| frostdb | parquet-go | |
|---|---|---|
| 5 | 3 | |
| 1,210 | 1,170 | |
| 1.1% | - | |
| 9.5 | 4.2 | |
| 3 days ago | about 2 months ago | |
| Go | Go | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
frostdb
-
Polar Signals Cloud Is Generally Available
> In addition to that we built a custom columnar database
I did some digging in your blog history and it seems that is referencing https://www.polarsignals.com/blog/posts/2022/07/22/frostdb-i... and digging into the "but why?" section <https://github.com/polarsignals/frostdb#why-you-should-use-f...> seems to imply you favored the embedded feature over having something standalone, but I would enjoy hearing (or reading a blog post!) about why you felt it was a better use of your engineering to make your own columar DB versus using one of the existing columanr dbs that I have seen referenced a ton in other Show HN announcements around both logging and metrics services
-
anyone have experience writing data to parquet files? Is there a better alternative for storing large amounts of financial tick data?
We use clickhouse, but i would take a look at https://github.com/polarsignals/frostdb
- Open Source Databases in Go
-
ArcticDB: A Database for Observability
Hey all, one of the creators of ArcticDB here. We're going to be around for a while and answer any questions you might have about it!
It's open source so if you just want to check out the repo: https://github.com/polarsignals/arcticdb
-
arcticDB: embedded columnar database written in Go
Direct link to the DB project -> https://github.com/polarsignals/arcticdb
parquet-go
-
anyone have experience writing data to parquet files? Is there a better alternative for storing large amounts of financial tick data?
xitongsys/parquet-go
-
Consider Using CSV
> It's so complex to work with, that unless you're specifically in data science, it's both unheard of and unusable.
FWIW, in my experience at a "data analytics platform" company, it's reasonably popular for data-heavy workflows since Because Parquet is well-defined, and file sizes are a fraction of their CSV equivalents.
> Is it a limitation of the format itself?
I don't think so. In other languages, you can generally read/write Parquet files without a ton of dependencies (e.g. https://github.com/xitongsys/parquet-go).
-
parquet-tools
(Throwaway account) I built https://github.com/hangxie/parquet-tools, the original idea was from parquet-go, it is still the underlying lib for my implementation.
What are some alternatives?
column - High-performance, columnar, in-memory store with bitmap indexing in Go
parquet-go - Go library to read/write Parquet files
tad - A desktop application for viewing and analyzing tabular data
clover - A lightweight document-oriented NoSQL database written in pure Golang.
parquet-tools - Utility to deal with Parquet data
marketstore - DataFrame Server for Financial Timeseries Data
Apache Parquet - Apache Parquet
levigo - levigo is a Go wrapper for LevelDB
xsv - A fast CSV command line toolkit written in Rust.
InfluxDB - Scalable datastore for metrics, events, and real-time analytics