marketstore
parquet-go
| marketstore | parquet-go | |
|---|---|---|
| 31 | 3 | |
| 1,819 | 1,171 | |
| 0.2% | - | |
| 0.0 | 4.2 | |
| 11 months ago | 2 months ago | |
| Go | Go | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
marketstore
-
anyone have experience writing data to parquet files? Is there a better alternative for storing large amounts of financial tick data?
I'm also open to the idea of a better tick-storage solution, but after trying many out like Alpaca Marketstore, TimescaleDB, etc. it seems like parquet files would fit my needs better and give me more flexibility for replaying data into my algorithms.
- marketstore: NEW Data - star count:1654.0
- marketstore: NEW Data - star count:1583.0
parquet-go
-
anyone have experience writing data to parquet files? Is there a better alternative for storing large amounts of financial tick data?
xitongsys/parquet-go
-
Consider Using CSV
> It's so complex to work with, that unless you're specifically in data science, it's both unheard of and unusable.
FWIW, in my experience at a "data analytics platform" company, it's reasonably popular for data-heavy workflows since Because Parquet is well-defined, and file sizes are a fraction of their CSV equivalents.
> Is it a limitation of the format itself?
I don't think so. In other languages, you can generally read/write Parquet files without a ton of dependencies (e.g. https://github.com/xitongsys/parquet-go).
-
parquet-tools
(Throwaway account) I built https://github.com/hangxie/parquet-tools, the original idea was from parquet-go, it is still the underlying lib for my implementation.
What are some alternatives?
alpaca-trade-api-python - Python client for Alpaca's trade API
parquet-go - Go library to read/write Parquet files
kaspad - Kaspad is the reference full node Kaspa implementation written in Go (golang).
tad - A desktop application for viewing and analyzing tabular data
tstorage - An embedded time-series database
parquet-tools - Utility to deal with Parquet data
cnosdb - A cloud-native open source distributed time series database with high performance, high compression ratio and high availability. http://www.cnosdb.cloud
Apache Parquet - Apache Parquet
alpaca-trade-api-js - Node.js library for Alpaca Trade API.
xsv - A fast CSV command line toolkit written in Rust.
bbgo - The modern cryptocurrency trading bot framework written in Go.
frostdb - ❄️ Coolest database around 🧊 Embeddable column database written in Go.