-
ioztat
ioztat is a storage load analysis tool for OpenZFS. It provides iostat-like statistics at an individual dataset/zvol level.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
In the meantime I was diagnosing some more (zpool iostat -ylq 10, zpool iostat -yw 10, Running zfsslow from https://github.com/iovisor/bcc) and it looks likes a number of READS are taking a long time.
Have you considered using ioztat to try to isolate the problematic bits of the workload? If you go this route, it would help to carve your home directory (or wherever you're doing the most work that hits disk) into discrete datasets--eg make ~/Pictures, ~/Videos, ~/Documents their own datasets.