pgbadger VS bat

It might also help to use pgbadger or something similar to process your postgres logs and see whether some event is aligned with your outages.

Interestingly, when a place does get to the point where the single instance has capacity issues (after upgrading to EPYC and lots of flash drives) then other non-obvious stuff shows up too.

For example, at one place just over a year ago they were well into this territory. One of weird problems for them was with pgBadger's memory usage (https://github.com/darold/pgbadger). That's written in perl, which doesn't seem to go garbage collection well. So even on a reporting node with a few hundred GB's of ram, it could take more than 24 hours to do a "monthly" reporting run.

There wasn't a solution in place at the time I left, so they're probably still having the issue... ;)

First of all I used a profiler (pgbadger and netdata) to figure out where the lags were coming from. I then tried the usual stuff (increasing shared_buffers, max_wal_size, min_wal_size from their ultra low defaults), but the biggest performance gain came from moving the database from eMMC to a mechanical hard drive :-D

Look into PgBadger (a log parser/analyser): https://github.com/darold/pgbadger

That’s the same as bat:[1] one of the features is syntax highlighting. Kind of unexpected to find a concatenation program… which also does that.

[1] https://github.com/sharkdp/bat

View on GitHub

4. bat

Good find, thanks! I'll check if I prefer it to moar.

As for bat, according to https://github.com/sharkdp/bat#using-bat-on-windows, the Chocolatey package simply installs `less` alongside `bat`. Seems like a good idea, but I haven't tried it.

Try bat (it’s like cat but better) https://github.com/sharkdp/bat

bat

For this reason I have a zsh function in my .zshrc with bat (which pages by default, if it's longer than your console height):

https://github.com/sharkdp/bat#highlighting---help-messages

  # in your .bashrc/.zshrc/*rc