klogg VS lnav

I'd love to see a tool that lets you modify large files efficiently.

I had to replace line 4 of a 200 GB SQL dump, it took a substantial amount of compute time to perform a find / replace with sed and it also required having over double the disk space since sed creates a temp file before it writes out the new file.

Using a hex editor could have worked but it seemed too risky because data integrity was really important.

There's also some other scenarios where maybe you have a massive file and the tool that's using it (such as a SQL import) throws an error on line 1,025,421. Trying to find what the contents of that line is on the command line could be time consuming if you need to read in the whole file. For read operations I know there's a few graphical tools like https://github.com/variar/klogg that efficiently let you scan, search and jump to points in a file quickly but I haven't found a good one on the command line.

reminds me a bit of klogg https://github.com/variar/klogg which is more for log files and based off glogg which went dead. it has nice filtering and highlighting type stuff. It's great for live views of log files.

glogg / klogg

For anybody who goes through this hell, I highly recommend [Klogg](https://github.com/variar/klogg).

For example, the file star2002-full.csv from https://sdm.lbl.gov/fastbit/data/samples.html is a 1.99GB file and it takes less than 10 seconds to load in Notepad+. It's almost instant in https://github.com/variar/klogg.

See https://lnav.org for a powerful mini-ETL CLI power tool; it embeds SQLite, supports ~every format, has great UX and easily handles a few million rows at a time.

The code base seems like a good reference as a small Python project.

My fav option in this class of apps: https://lnav.org/ It lets you use journalctl with pipes as requested here: https://github.com/Textualize/toolong/issues/4

For local development, I cannot recommend lnav[1] enough. Discovering this tool was a game changer in my day to day life. Adding comments, filtering in/out, prettify and analyse distribution is hard to live without now.

I don't think a browser tool would fit in my workflow. I need to pipe the output to the tool.

[1] https://lnav.org/

It looks like they run an SSH server inside a Docker container defined by this Dockerfile [1]. This uses the ForceCommand directive in the sshd_config file to ensure that a specific command is run when a user connects (rather than the user connecting directly to a shell).

Depending on whether the user connects as the `playground` or `tutorial1` user they interact with a bash script that is either [2] or [3].

[1]: https://github.com/tstack/lnav/blob/master/demo/Dockerfile

[2]: https://github.com/tstack/lnav/blob/master/docs/tutorials/pl...

[3]: https://github.com/tstack/lnav/blob/master/docs/tutorials/tu...