-
jvns highlights some of the easier to forgot or overlook mistakes, but their source article https://100go.co is a great refresher and introduction as well.
-
InfluxDB
Purpose built for real-time analytics at any scale. InfluxDB Platform is powered by columnar analytics, optimized for cost-efficient storage, and built with open data standards.
-
A (shameless) plug: I've been building a collection of Go bits like this. Hopefully it can be useful to someone other than me, too:
https://github.com/geokat/easygo
-
There is no free lunch in software :)
> using the syntax of C pointers (but without pointer arithmetic)
Go structs, when they have their address taken, act the same way object references do in C# or Java. Taking an address of a struct in Go and assigning it to a location likely turns it into a heap allocation. Even when not, it is important to understand what a stack in Go is and how it is implemented:
Go makes a different set of tradeoffs by using a virtual/managed stack. When more memory on stack is required - the request goes through an allocator and the new memory range is attached to a previous one in a linked-list fashion (if this has changed in recent versions - please correct me). This is rather effective but notably can suffer from locality issues where, for example, two adjacently accessed structs in a hot loop are placed in different memory locations. Modern CPUs can deal with such non-locality of data well, but it is a concern still.
It also, in a way, makes it somewhat similar to the role Gen0 / Nursery GC heaps play in .NET and OpenJDK GC implementations - heap allocations just bump a pointer within thread-local allocation context, and if a specific heap has ran out of free memory - more memory is requested from GC (or a collection is triggered, or both). By nature of both having generational heaps and moving garbage collector, the data in the heap has high locality and consecutively allocated objects are placed in a linear fashion. One of the main drawbacks of such approach is much higher implementation complexity.
Additionally, the Go choice comes at a tradeoff of making FFI (comparatively) expensive - you pay about 0.5-2ns for a call across interop into C in .NET (when statically linked, it's just a direct call + branch), where-as the price of FFI in GoGC is around 50ns (and also interacts worse with the executor if you block goroutines).
Overall, the design behind memory management in Go is interesting and makes a lot of sense for the scenarios Go was intended for (lightweight networked microservices, CLI tooling, background daemons, etc.).
However, it trades off smaller memory footprint for a significantly lower allocation throughput. Because GC in Go is, at least partially, write-barrier driven, it makes WBs much more expensive - something you pay for when you assign a Go struct pointer to a heap or not provably local location (I don't know if Go performs WB elision/cheap WB selection the way OpenJDK and .NET do).
To explore this further, I put together a small demonstration[0] based on BenchmarksGame's BinaryTrees suite which stresses this exact scenario. Both Go and C# there can be additionally inspected with e.g. dtrace on macOS or most other native profilers like Samply[1].
> Now, in working with real C# code and real C# programmers, all this is false...
> but if you want to store it in a field, you need to use some class type as a box,
This does not correspond to language specification or the kind of code that is being written outside of enterprise.
First of all, `ref T` syntax in C# is a much more powerful concept than a simple reference to a local variable. `ref T` in .NET is a 'byref' aka managed pointer. Byrefs can point to arbitrary memory and are GC-aware. This means that they can point to stack, unmanged memory (you can even mmap it directly) GC on NonGC heap object interiors, etc. If they point to GC heap object interiors, they are appropriately updated by GC when it moves the objects, and are ignored when they are not. They cannot be stored on heap, which does cause some tension, but if you have a pointer-rich data structure, then classes are a better choice almost every time.
Byrefs can be held by ref structs and the most common example used everywhere today is Span - internally it is (ref T _reference, int _length) and is used for interacting with and slicing of arbitrary contiguous memory. You can find additional details on low-level memory management techniques here: https://news.ycombinator.com/item?id=40963672
Byrefs, alongside regular C pointers in C#, support pointer arithmetics as well. Of course it is just as unsafe, but for targeted hot paths this is indispensable. You can use it for fancy things like vectorized byte pair count within a sequence: https://github.com/U8String/U8String/blob/split-refactor/Sou...
Last but not least, the average line-of-business code indeed rarely uses structs - in the past it was mostly classes, today it is a mix of classes, records, and sometimes record structs for single-field wrappers (still rarely). However, C# is a multi-paradigm language with strong low-level capabilities and there is a sea of projects beyond enterprise that make use of all the new and old low-level features. It's something that both C# and .NET were designed in mind from the very beginning. In this regard, they brings you much closer to the metal than Go unless significant changes are introduced to it.
If you're interested, feel free to explore Ryujinx[2] and Garnet[3] which are more recent examples of projects demonstrating suitability of C# in domains historically reserved for C, C++ and Rust.
[0]: https://gist.github.com/neon-sunset/72e6aa57c6a4c5eb0e2711e1...
[1]: https://github.com/mstange/samply
[2]: https://github.com/search?q=repo%3ARyujinx%2FRyujinx+ref+str...
[3]: https://github.com/search?q=repo%3Amicrosoft%2Fgarnet+struct...
-
There is no free lunch in software :)
> using the syntax of C pointers (but without pointer arithmetic)
Go structs, when they have their address taken, act the same way object references do in C# or Java. Taking an address of a struct in Go and assigning it to a location likely turns it into a heap allocation. Even when not, it is important to understand what a stack in Go is and how it is implemented:
Go makes a different set of tradeoffs by using a virtual/managed stack. When more memory on stack is required - the request goes through an allocator and the new memory range is attached to a previous one in a linked-list fashion (if this has changed in recent versions - please correct me). This is rather effective but notably can suffer from locality issues where, for example, two adjacently accessed structs in a hot loop are placed in different memory locations. Modern CPUs can deal with such non-locality of data well, but it is a concern still.
It also, in a way, makes it somewhat similar to the role Gen0 / Nursery GC heaps play in .NET and OpenJDK GC implementations - heap allocations just bump a pointer within thread-local allocation context, and if a specific heap has ran out of free memory - more memory is requested from GC (or a collection is triggered, or both). By nature of both having generational heaps and moving garbage collector, the data in the heap has high locality and consecutively allocated objects are placed in a linear fashion. One of the main drawbacks of such approach is much higher implementation complexity.
Additionally, the Go choice comes at a tradeoff of making FFI (comparatively) expensive - you pay about 0.5-2ns for a call across interop into C in .NET (when statically linked, it's just a direct call + branch), where-as the price of FFI in GoGC is around 50ns (and also interacts worse with the executor if you block goroutines).
Overall, the design behind memory management in Go is interesting and makes a lot of sense for the scenarios Go was intended for (lightweight networked microservices, CLI tooling, background daemons, etc.).
However, it trades off smaller memory footprint for a significantly lower allocation throughput. Because GC in Go is, at least partially, write-barrier driven, it makes WBs much more expensive - something you pay for when you assign a Go struct pointer to a heap or not provably local location (I don't know if Go performs WB elision/cheap WB selection the way OpenJDK and .NET do).
To explore this further, I put together a small demonstration[0] based on BenchmarksGame's BinaryTrees suite which stresses this exact scenario. Both Go and C# there can be additionally inspected with e.g. dtrace on macOS or most other native profilers like Samply[1].
> Now, in working with real C# code and real C# programmers, all this is false...
> but if you want to store it in a field, you need to use some class type as a box,
This does not correspond to language specification or the kind of code that is being written outside of enterprise.
First of all, `ref T` syntax in C# is a much more powerful concept than a simple reference to a local variable. `ref T` in .NET is a 'byref' aka managed pointer. Byrefs can point to arbitrary memory and are GC-aware. This means that they can point to stack, unmanged memory (you can even mmap it directly) GC on NonGC heap object interiors, etc. If they point to GC heap object interiors, they are appropriately updated by GC when it moves the objects, and are ignored when they are not. They cannot be stored on heap, which does cause some tension, but if you have a pointer-rich data structure, then classes are a better choice almost every time.
Byrefs can be held by ref structs and the most common example used everywhere today is Span - internally it is (ref T _reference, int _length) and is used for interacting with and slicing of arbitrary contiguous memory. You can find additional details on low-level memory management techniques here: https://news.ycombinator.com/item?id=40963672
Byrefs, alongside regular C pointers in C#, support pointer arithmetics as well. Of course it is just as unsafe, but for targeted hot paths this is indispensable. You can use it for fancy things like vectorized byte pair count within a sequence: https://github.com/U8String/U8String/blob/split-refactor/Sou...
Last but not least, the average line-of-business code indeed rarely uses structs - in the past it was mostly classes, today it is a mix of classes, records, and sometimes record structs for single-field wrappers (still rarely). However, C# is a multi-paradigm language with strong low-level capabilities and there is a sea of projects beyond enterprise that make use of all the new and old low-level features. It's something that both C# and .NET were designed in mind from the very beginning. In this regard, they brings you much closer to the metal than Go unless significant changes are introduced to it.
If you're interested, feel free to explore Ryujinx[2] and Garnet[3] which are more recent examples of projects demonstrating suitability of C# in domains historically reserved for C, C++ and Rust.
[0]: https://gist.github.com/neon-sunset/72e6aa57c6a4c5eb0e2711e1...
[1]: https://github.com/mstange/samply
[2]: https://github.com/search?q=repo%3ARyujinx%2FRyujinx+ref+str...
[3]: https://github.com/search?q=repo%3Amicrosoft%2Fgarnet+struct...
