peaks-framework
peaks-consolidation
peaks-framework | peaks-consolidation | |
---|---|---|
8 | 37 | |
46 | 102 | |
- | - | |
10.0 | 9.6 | |
about 1 year ago | 5 months ago | |
Go | Go | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
peaks-framework
-
HTML5 Web Pivot/Drilldown Viewer for Polars.exe
If there are a strong demand on this toy, I will request Polars team to support my development on this matter after I completing my important task to publish Peaks.exe runtime to the github github.com/hkpeaks/peaks-framework
-
Leverage A-SQL Statement to Accelerate ETL Processing
Github: github.com/hkpeaks/peaks-framework
-
Databricks Clusters
CSV is splittable. This project is playing splittable csv github.com/hkpeaks/peaks-framework
-
Test On 4 Concurrent Jobs Using Python-Polars 0.17.11 to GroupBy Billion Rows
This project has only 3-month history, first trial vesion to be released in Jun, provide most fundamental commands. For further info, you can visit github.com/hkpeaks/peaks-framework
-
Converting my new code (Bytearray2Float64) into 19 Programming Language
In my previous dotnet pivot table web, I have implemented a in-memory data object serialized in disk. You can find the source code https://github.com/hkpeaks/peaks-framework/tree/main/PeaksDataFrameViewer
-
Need Golang Community to Support a Hyper-performance of DataFrame Library
My Peaks DataFrame project is aiming at billion-row level data processing (extract-transform-load) for csv/parquet/json/excel/html files with little memory (recommend 16GB or above, for billion rows processing, it is recommended using NVMe SSD harddisk).
-
Compare Golang Speed with C#, Golang, Pandas and Polars
Source code of basic programming can be download from github.com/hkpeaks/peaks-framework
-
When I wanted to leave C#, I had two choices.
Source Code: github.com/hkpeaks/peaks-framework/tree/main/CompareProgrammingLanguage
peaks-consolidation
-
Filter a 7 billion-row dataset using 32GB Memory
Script and Data
- Success in Lighting Fast Billion-Row Sorting Using a 32GB Desktop PC
-
Stochastic Sublinear Streaming Algorithms
This streaming algorithms will support billion-row sorting
-
Billion-row Sorting Scripts for Peaks, Polars, Pandas and DuckDB
Sample data for 100,000 rows
-
Understand The SQL Execution Order
SplitFile2Folder: allows to filter a big CSV file or a folder which contains many CSV file to a folder/sub-folder which results many table partitions
-
OrderBy{Ledger(A) Account(D) Quantity(FloatA) Part_No(A)}
Now I am upgrading the Peaks, not only support orderby, but also support billion-row sorting use cases using only 32GB memory.
-
Success to Build 2 functions: SplitFile2Folder and FilterFromFolder
BillionRowsTestingLog is a set of processing logs included in the 1st pre-release delivery. These are foucs on billion-row databending exercises.
-
Implement Go Streaming to Process Over Memory Size Dataset
I have decided to share my written algorithms for a period of time by spending additional time to maintain the repository.
- Solved to Filter/Summarize Data from Huge CSV Files
- Solved Huge CSV File
What are some alternatives?
PeaksDataFrameViewer - Peaks DataFrame Viewer (previously known as youFast Desktop) is an HTML5 pivot table that supports fast and responsive viewing of transactions in summary view and pivot view. Both views allow you to drill down from summary figures to transaction level.
Pandas - Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
omniparser - omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
Fieldnotes - Public repository of my field notes from 25+ years as computer guy
polars - Dataframes powered by a multithreaded, vectorized query engine, written in Rust
gobyexample - Go by Example
glibc - GNU Libc
rustle - An HTTP-based in-memory broker inspired by Redis.
chi - lightweight, idiomatic and composable router for building Go HTTP services