ffi-overhead
go
ffi-overhead | go | |
---|---|---|
19 | 2,093 | |
645 | 120,346 | |
- | 0.6% | |
0.0 | 10.0 | |
11 months ago | 4 days ago | |
C | Go | |
Apache License 2.0 | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ffi-overhead
-
3 years of fulltime Rust game development, and why we're leaving Rust behind
The overhead for Go in benchmarks is insane in contrast to other languages - https://github.com/dyu/ffi-overhead Are there reasons why Go does not copy what Julia does?
-
Can Fortran survive another 15 years?
What about the other benchmarks on the same site? https://docs.sciml.ai/SciMLBenchmarksOutput/stable/Bio/BCR/ BCR takes about a hundred seconds and is pretty indicative of systems biological models, coming from 1122 ODEs with 24388 terms that describe a stiff chemical reaction network modeling the BCR signaling network from Barua et al. Or the discrete diffusion models https://docs.sciml.ai/SciMLBenchmarksOutput/stable/Jumps/Dif... which are the justification behind the claims in https://www.biorxiv.org/content/10.1101/2022.07.30.502135v1 that the O(1) scaling methods scale better than O(log n) scaling for large enough models? I mean.
> If you use special routines (BLAS/LAPACK, ...), use them everywhere as the respective community does.
It tests with and with BLAS/LAPACK (which isn't always helpful, which of course you'd see from the benchmarks if you read them). One of the key differences of course though is that there are some pure Julia tools like https://github.com/JuliaLinearAlgebra/RecursiveFactorization... which outperform the respective OpenBLAS/MKL equivalent in many scenarios, and that's one noted factor for the performance boost (and is not trivial to wrap into the interface of the other solvers, so it's not done). There are other benchmarks showing that it's not apples to apples and is instead conservative in many cases, for example https://github.com/SciML/SciPyDiffEq.jl#measuring-overhead showing the SciPyDiffEq handling with the Julia JIT optimizations gives a lower overhead than direct SciPy+Numba, so we use the lower overhead numbers in https://docs.sciml.ai/SciMLBenchmarksOutput/stable/MultiLang....
> you must compile/write whole programs in each of the respective languages to enable full compiler/interpreter optimizations
You do realize that a .so has lower overhead to call from a JIT compiled language than from a static compiled language like C because you can optimize away some of the bindings at the runtime right? https://github.com/dyu/ffi-overhead is a measurement of that, and you see LuaJIT and Julia as faster than C and Fortran here. This shouldn't be surprising because it's pretty clear how that works?
I mean yes, someone can always ask for more benchmarks, but now we have a site that's auto updating tons and tons of ODE benchmarks with ODE systems ranging from size 2 to the thousands, with as many things as we can wrap in as many scenarios as we can wrap. And we don't even "win" all of our benchmarks because unlike for you, these benchmarks aren't for winning but for tracking development (somehow for Hacker News folks they ignore the utility part and go straight to language wars...).
If you have a concrete change you think can improve the benchmarks, then please share it at https://github.com/SciML/SciMLBenchmarks.jl. We'll be happy to make and maintain another.
-
When dealing with C, when is Go slow?
If you're calling back and forth between C and Go in a performance critical way. It's one of the slowest languages for wrapping C that there is. I've personally hit this bottleneck in numerous projects, wrapping things like libutp and sqlite. See also https://github.com/dyu/ffi-overhead
-
Understanding N and 1 queries problem
Piling on about overhead (and SQLite), many high-level languages take some hit for using an FFI. So you're still incentivized to avoid tons of SQLite calls.
https://github.com/dyu/ffi-overhead
-
Are there plans to improve concurrency in Rust?
Go doesn't even have native thread stacks. When call any FFI function Go has to switch over to an on-demand stack and coordinate the goroutine and the runtime to avoid preemption and starvation. This is part of why Go's calling overhead is over 30x slower than C/C++/Rust (source). It's understandbly become Go community culture to act like FFI is just not even an option and reinvent everything in Go, but that reinvented Go suffers from these other problems plus many more (such as optimizing far worse than GCC or LLVM).
-
Comparing the C FFI overhead on various languages
Some of the results look outdated. The Dart results look bad (25x slower than C), but looking at the code (https://github.com/dyu/ffi-overhead/tree/master/dart) it appears to be five years old. Dart has a new FFI as of Dart 2.5 (2019): https://medium.com/dartlang/announcing-dart-2-5-super-charge... I'm curious how the new FFI would fare in these benchmarks.
-
Would docker be faster if it were written in rust?
In that case, the libcontainer library would be faster if written in most other languages seeing as Go has unfortunate C-calling performance. In this FFI benchmark Rust is on par with C with 1193ms (total benchmarking time), while Go took 37975ms doing the same.
-
Using Windows API in Julia?
Hi there folks! I'm going to call the Windows API as rapidly as possible and will be doing some calculations with the results, and I thought Julia might be perfect for this task as its FFI is impressively fast, and of course, Julia is fast regarding numbers as well :).
go
-
Arena-Based Parsers
The description indicates it is not production ready, and is archived at the same time.
If you pull all stops in each respective language, C# will always end up winning at parsing text as it offers C structs, pointers, zero-cost interop, Rust-style struct generics, cross-platform SIMD API and simply has better compiler. You can win back some performance in Go by writing hot parts in Go's ASM dialect at much greater effort for a specific platform.
For example, Go has to resort to this https://github.com/golang/go/blob/4ed358b57efdad9ed710be7f4f... in order to efficiently scan memory, while in C# you write the following once and it compiles to all supported ISAs with their respective SIMD instructions for a given vector width: https://github.com/dotnet/runtime/blob/56e67a7aacb8a644cc6b8... (there is a lot of code because C# covers much wider range of scenarios and does not accept sacrificing performance in odd lengths and edge cases, which Go does).
Another example is computing CRC32: you have to write ASM for Go https://github.com/golang/go/blob/4ed358b57efdad9ed710be7f4f..., in C# you simply write standard vectorized routine once https://github.com/dotnet/runtime/blob/56e67a7aacb8a644cc6b8... (its codegen is competitive with hand-intrinsified C++ code).
There is a lot more of this. Performance and low-level primitives to achieve it have been an area of focus of .NET for a long time, so it is disheartening to see one tenth of effort in Go to receive so much spotlight.
-
Go: the future encoding/json/v2 module
A Discussion about including this package in Go as encoding/json/v2 has been started on the Go Github project on 2023-10-05. Please provide your feedback there.
-
Evolving the Go Standard Library with math/rand/v2
I like the Principles section. Very measured and practical approach to releasing new stdlib packages. https://go.dev/blog/randv2#principles
The end of the post they mention that an encoding/json/v2 package is in the works: https://github.com/golang/go/discussions/63397
-
Microsoft Maintains Go Fork for FIPS 140-2 Support
There used to be the GO FIPS branch :
https://github.com/golang/go/tree/dev.boringcrypto/misc/bori...
But it looks dead.
And it looks like https://github.com/golang-fips/go as well.
-
Borgo is a statically typed language that compiles to Go
I'm not sure what exactly you mean by acknowledgement, but here are some counterexamples:
- A proposal for sum types by a Go team member: https://github.com/golang/go/issues/57644
- The community proposal with some comments from the Go team: https://github.com/golang/go/issues/19412
Here are some excerpts from the latest Go survey [1]:
- "The top responses in the closed-form were learning how to write Go effectively (15%) and the verbosity of error handling (13%)."
- "The most common response mentioned Go’s type system, and often asked specifically for enums, option types, or sum types in Go."
I think the problem is not the lack of will on the part of the Go team, but rather that these issues are not easy to fix in a way that fits the language and doesn't cause too many issues with backwards compatibility.
[1]: https://go.dev/blog/survey2024-h1-results
-
AWS Serverless Diversity: Multi-Language Strategies for Optimal Solutions
Now, I’m not going to use C++ again; I left that chapter years ago, and it’s not going to happen. C++ isn’t memory safe and easy to use and would require extended time for developers to adapt. Rust is the new kid on the block, but I’ve heard mixed opinions about its developer experience, and there aren’t many libraries around it yet. LLRD is too new for my taste, but **Go** caught my attention.
-
How to use Retrieval Augmented Generation (RAG) for Go applications
Generative AI development has been democratised, thanks to powerful Machine Learning models (specifically Large Language Models such as Claude, Meta's LLama 2, etc.) being exposed by managed platforms/services as API calls. This frees developers from the infrastructure concerns and lets them focus on the core business problems. This also means that developers are free to use the programming language best suited for their solution. Python has typically been the go-to language when it comes to AI/ML solutions, but there is more flexibility in this area. In this post you will see how to leverage the Go programming language to use Vector Databases and techniques such as Retrieval Augmented Generation (RAG) with langchaingo. If you are a Go developer who wants to how to build learn generative AI applications, you are in the right place!
-
From Homemade HTTP Router to New ServeMux
net/http: add methods and path variables to ServeMux patterns Discussion about ServeMux enhancements
-
Building a Playful File Locker with GoFr
Make sure you have Go installed https://go.dev/.
- Fastest way to get IPv4 address from string
What are some alternatives?
sqlite
v - Simple, fast, safe, compiled language for developing maintainable software. Compiles itself in <1s with zero library dependencies. Supports automatic C => V translation. https://vlang.io
krustlet - Kubernetes Rust Kubelet
TinyGo - Go compiler for small places. Microcontrollers, WebAssembly (WASM/WASI), and command-line tools. Based on LLVM.
glmark2 - glmark2 is an OpenGL 2.0 and ES 2.0 benchmark
zig - General-purpose programming language and toolchain for maintaining robust, optimal, and reusable software.
kutil - Go Utilities
Nim - Nim is a statically typed compiled systems programming language. It combines successful concepts from mature languages like Python, Ada and Modula. Its design focuses on efficiency, expressiveness, and elegance (in that order of priority).
lzbench - lzbench is an in-memory benchmark of open-source LZ77/LZSS/LZMA compressors
Angular - Deliver web apps with confidence 🚀
CheeseShop - Examples of using PyO3 Rust bindings for Python with little to no silliness.
golang-developer-roadmap - Roadmap to becoming a Go developer in 2020