klogg
q
Our great sponsors
klogg | q | |
---|---|---|
11 | 46 | |
1,954 | 10,098 | |
- | - | |
8.3 | 3.6 | |
about 1 month ago | 2 months ago | |
C++ | Python | |
GNU General Public License v3.0 only | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
klogg
-
Toolong: Terminal application to view, tail, merge, and search log files
I'd love to see a tool that lets you modify large files efficiently.
I had to replace line 4 of a 200 GB SQL dump, it took a substantial amount of compute time to perform a find / replace with sed and it also required having over double the disk space since sed creates a temp file before it writes out the new file.
Using a hex editor could have worked but it seemed too risky because data integrity was really important.
There's also some other scenarios where maybe you have a massive file and the tool that's using it (such as a SQL import) throws an error on line 1,025,421. Trying to find what the contents of that line is on the command line could be time consuming if you need to read in the whole file. For read operations I know there's a few graphical tools like https://github.com/variar/klogg that efficiently let you scan, search and jump to points in a file quickly but I haven't found a good one on the command line.
-
Textanalysistool.net
reminds me a bit of klogg https://github.com/variar/klogg which is more for log files and based off glogg which went dead. it has nice filtering and highlighting type stuff. It's great for live views of log files.
-
Not giving developers root access to their dev machines
For anybody who goes through this hell, I highly recommend [Klogg](https://github.com/variar/klogg).
- Share your greatest free tools
- Contrarian here: What legacy software will they have to pry from your cold, dead fingers before you give it up?
-
Superintendent.app: A desktop app for working with large CSV files using SQL. v2.0 now can load 1GB file in 20s and support regex/date parsing!
For example, the file star2002-full.csv from https://sdm.lbl.gov/fastbit/data/samples.html is a 1.99GB file and it takes less than 10 seconds to load in Notepad+. It's almost instant in https://github.com/variar/klogg.
q
-
I wrote this iCalendar (.ics) command-line utility to turn common calendar exports into more broadly compatible CSV files.
CSV utilities (still haven't pick a favorite one...): https://github.com/harelba/q https://github.com/BurntSushi/xsv https://github.com/wireservice/csvkit https://github.com/johnkerl/miller
- Segítség kérés Excel automatizáláshoz
-
Show HN: ClickHouse-local – a small tool for serverless data analytics
I think they're talking about https://github.com/harelba/q, which is not very fast.
-
sqly - execute SQL against CSV / JSON with shell
Apparently, there were many who thought the same thing; Tools to execute SQL against CSV were trdsql, q, csvq, TextQL. They were highly functional, hoewver, had many options and no input completion. I found it just a little difficult to use.
-
Q – Run SQL Directly on CSV or TSV Files
http://harelba.github.io/q/#requirements
"q is packaged as a compiled standalone-executable that has no dependencies, not even python itself."
This is not quite true, on MacOS:
"q: A full installation of Xcode.app 12.4 is required to compile
Hi, author of q here.
Regarding the error you got, q currently does not autodetect headers, so you'd need to add -H as a flag in order to use the "country" column name. You're absolutely correct on failing-fast here - It's a bug which i'll fix.
In general regarding speed - q supports automatic caching of the CSV files (through the "-C readwrite" flag). Once it's activated, it will write the data into another file (with a .qsql extension), and will use it automatically in further queries in order to speed things considerably.
Effectively, the .qsql files are regular sqlite3 files (with some metadata), and q can be used to query them directly (or any regular sqlite3 file), including the ability to seamlessly join between multiple sqlite3 files.
- PostgreSQL alternative for Large amounts of data
-
q VS trdsql - a user suggested alternative
2 projects | 25 Jun 2022
- One-liner for running queries against CSV files with SQLite
What are some alternatives?
textql - Execute SQL against structured text like CSV or TSV
RIP - Free,Open-Source,Cross-platform agent and Post-exploiton tool written in Golang and C++.
csvq - SQL-like query language for csv
octosql - OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.
xsv - A fast CSV command line toolkit written in Rust.
InquirerPy - :snake: Python port of Inquirer.js (A collection of common interactive command-line user interfaces)
ledger - Double-entry accounting system with a command-line reporting interface
simdjson - Parsing gigabytes of JSON per second : used by Facebook/Meta Velox, the Node.js runtime, ClickHouse, WatermelonDB, Apache Doris, Milvus, StarRocks
sqlitebrowser - Official home of the DB Browser for SQLite (DB4S) project. Previously known as "SQLite Database Browser" and "Database Browser for SQLite". Website at:
siuba - Python library for using dplyr like syntax with pandas and SQL
emacs-edbi - Database Interface for Emacs Lisp
sqlite-utils - Python CLI utility and library for manipulating SQLite databases