Our great sponsors
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
websocketd
Turn any program that uses STDIN/STDOUT into a WebSocket server. Like inetd, but for WebSockets.
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
It's nice to have this in the standard library, but it doesn't solve any existing pain points around structured log metadata and contexts. We use zap [0] and store a zap logger on the request context which allows different parts of the request pipeline to log with things like tenantid, traceId, and correlationId automatically appended. But getting a logger off the context is annoying, leads to inconsistent logging practices, and creates a logger dependency throughout most of our Go code.
[0] https://github.com/uber-go/zap
> I also don't see something else I might want: a way to have a different "view" for certain log messages; maybe to switch between filtering/viewing particular ones, maybe to just have line-format be conditional based on the detected format.
Have a look at the following comment on an issue that might be similar to what you're thinking of:
https://github.com/tstack/lnav/issues/1065#issuecomment-1602...
> I guess I can sort of do this based on `module-field`? but I might want it lighter-weight/finer-grained than that.
Unfortunately, the "module-field" does not work for JSON logs at the moment. It's something I should really fix.
Ultimately, lnav has existed for almost two decades now and I use it every day. So, it's always seeing improvements. If you're having a problem with it, file an issue on github. I don't always get around quickly to fixing other folks feature requests / issues, but it tends to happen eventually.
Thanks.
I hadn't even considered collecting traces/spans in this way yet, and have taken the approach of "stuff outputting logs in JSON format to stderr/local file". I usually end up writing a (temporary, structured) log message with the relevant span tags, but wouldn't it would be much better to run the actual trace/span code and be able to verify it locally without the ad-hoc log message?
The prototype I built is a web application that creates websocket connections, and if those connections receive messages that are JSON, log lines are added. Columns are built dynamically as log messages arrive, and then you can pick which columns to render in the table. If you're curious here's the code, including a screenshot: https://github.com/corytheboyd-smartsheet/json-log-explorer
With websockets, it's very easy to use websocketd (http://websocketd.com), which will watch input files for new lines, and write them verbatim as websocket messages to listeners (the web app).
To make the idea real, would want to figure out how to not require the user to run websocketd out of band, but watching good ol' files is dead simple, and very easy to add to most code (add a new log sink, use existing log file, etc.)
I hadn't even considered collecting traces/spans in this way yet, and have taken the approach of "stuff outputting logs in JSON format to stderr/local file". I usually end up writing a (temporary, structured) log message with the relevant span tags, but wouldn't it would be much better to run the actual trace/span code and be able to verify it locally without the ad-hoc log message?
The prototype I built is a web application that creates websocket connections, and if those connections receive messages that are JSON, log lines are added. Columns are built dynamically as log messages arrive, and then you can pick which columns to render in the table. If you're curious here's the code, including a screenshot: https://github.com/corytheboyd-smartsheet/json-log-explorer
With websockets, it's very easy to use websocketd (http://websocketd.com), which will watch input files for new lines, and write them verbatim as websocket messages to listeners (the web app).
To make the idea real, would want to figure out how to not require the user to run websocketd out of band, but watching good ol' files is dead simple, and very easy to add to most code (add a new log sink, use existing log file, etc.)
This is such an infuriating problem. I'm convinced I'm using Go wrong, because I simply can't understand how this doesn't make it a toy language. Why the $expletive am I wasting 20-30 and more minutes per week of my life looking for the source of an error!?
Have you seen https://github.com/tomarrell/wrapcheck? It's a linter than does a fairly good job of warning when an error originates from an external package but hasn't been wrapped in your codebase to make it unique or stacktraced. It comes with https://github.com/golangci/golangci-lint and can even be made part of your in-editor LSP diagnostics.
But still, it's not perfect. And so I remain convinced that I'm misunderstanding something fundamental about the language because not being able to consistently find the source of an error is such an egregious failing for a programming language.
This is such an infuriating problem. I'm convinced I'm using Go wrong, because I simply can't understand how this doesn't make it a toy language. Why the $expletive am I wasting 20-30 and more minutes per week of my life looking for the source of an error!?
Have you seen https://github.com/tomarrell/wrapcheck? It's a linter than does a fairly good job of warning when an error originates from an external package but hasn't been wrapped in your codebase to make it unique or stacktraced. It comes with https://github.com/golangci/golangci-lint and can even be made part of your in-editor LSP diagnostics.
But still, it's not perfect. And so I remain convinced that I'm misunderstanding something fundamental about the language because not being able to consistently find the source of an error is such an egregious failing for a programming language.
Agreed. In fact I wrote a (very) small library to help deal with it.
https://github.com/kitd/chock