tcmalloc | rfcs | |
---|---|---|
15 | 666 | |
4,081 | 5,711 | |
1.4% | 0.9% | |
9.8 | 9.8 | |
3 days ago | 2 days ago | |
C++ | Markdown | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
tcmalloc
-
Configuring HugePages on Google's TCMalloc
https://github.com/google/tcmalloc/issues/190
-
Configuring HugePages on TCMalloc
I had earlier raised a query on the github.com/google/tcmalloc regarding how I can force tcmalloc to back memory with hugetlbfs instead of using Transparent Huge Pages. I have attached the link to my query below. Please let me know if there is an possible way to do this.
-
New memory related fields in Yugabyte 2.17.3 pg_stat_activity
The allocated_mem_bytes field shows the memory allocated by the memory allocator. PostgreSQL is setup in an extensible way, which includes the ability to choose a memory allocator, which for PostgreSQL is ptmalloc, and for YSQL is tcmalloc. PostgreSQL has the ability to change the memory allocator, but by default uses the operating system memory allocator.
-
Spotting and Avoiding Heap Fragmentation in Rust Applications
> * Switching from libc malloc to tcmalloc (dating myself a little bit)
If you think of tcmalloc as an old crusty allocator, you've probably only seen the gperftools version of it.
This is the version Google now uses internally: https://github.com/google/tcmalloc
It's worth a fresh look. In particular, it supports per-CPU caches as an alternative to per-thread caches. Those are fantastic if you have a lot more threads than CPUs.
-
I've had bad luck with transparent hugepages on my Linux machines
The default setting of max_ptes_none is also problematic.
On a stock kernel, it's 511. TCMalloc's docs recommend using max_ptes_none set to 0 for this reason: https://github.com/google/tcmalloc/blob/master/docs/tuning.m...
(Disclosure: I work on TCMalloc and authored the above doc.)
-
Pages Are a Good Idea
The easiest way to exploit THP, by far, is to link your program against TCMalloc and forget about it. Literally free money. Highly recommended.
https://github.com/google/tcmalloc
- Why tcmalloc using aggresive decommit == false is a litte better than jemalloc
- System memory allocator free operation zeroes out deallocated blocks in iOS 16
-
malloc() and free() are a bad API
This means that efficient malloc implementation is typically overly complicated. mimalloc for example is almost 8K lines of C afaik, which is one of the smaller but still efficient malloc implementation I'm aware of. (Try looking into tcmalloc for comparison).
-
malloc global mutex?
Yes, it is synchronized, you can also swap out the implementation typically. There are different allocators out there depending on what you are trying to optimize for (memory, single thread performance, multithread performance, locality, etc). A lot of multithreading optimized ones use per thread pools, so each individual allocation doesn't need to globally lock, but changing the pools themself does, or large allocations that aren't part of the pools. For example https://github.com/google/tcmalloc
rfcs
-
Ask HN: What April Fools jokes have you noticed this year?
RFC: Add large language models to Rust
https://github.com/rust-lang/rfcs/pull/3603
- Rust to add large language models to the standard library
-
Why does Rust choose not to provide `for` comprehensions?
Man, SO and family has really gone downhill. That top answer is absolutely terrible. In fact, if you care, you can literally look at the RFC discussion here to see the actual debate: https://github.com/rust-lang/rfcs/pull/582
Basically, `for x in y` is kind of redundant, already sorta-kinda supported by itertools, and there's also a ton of macros that sorta-kinda do it already. It would just be language bloat at this point.
Literally has nothing to do with memory management.
- Coroutines in C
-
Uv: Python Packaging in Rust
Congrats!
> Similarly, uv does not yet generate a platform-agnostic lockfile. This matches pip-tools, but differs from Poetry and PDM, making uv a better fit for projects built around the pip and pip-tools workflows.
Do you expect to make the higher level workflow independent of requirements.txt / support a platform-agnostic lockfile? Being attached to Rye makes me think "no".
Without being platform agnostic, to me this is dead-on-arrival and unable to meet the "Cargo for Python" aim.
> uv supports alternate resolution strategies. By default, uv follows the standard Python dependency resolution strategy of preferring the latest compatible version of each package. But by passing --resolution=lowest, library authors can test their packages against the lowest-compatible version of their dependencies. (This is similar to Go's Minimal version selection.)
> uv allows for resolutions against arbitrary target Python versions. While pip and pip-tools always resolve against the currently-installed Python version (generating, e.g., a Python 3.12-compatible resolution when running under Python 3.12), uv accepts a --python-version parameter, enabling you to generate, e.g., Python 3.7-compatible resolutions even when running under newer versions.
This is great to see though!
I can understand it being a flag on these lower level, directly invoked dependency resolution operations.
While you aren't onto the higher level operations yet, I think it'd be useful to see if there is any cross-ecosystem learning we can do for my MSRV RFC: https://github.com/rust-lang/rfcs/pull/3537
How are you handling pre-releases in you resolution? Unsure how much of that is specified in PEPs. Its something that Cargo is weak in today but we're slowly improving.
- RFC: Rust Has Provenance
-
The bane of my existence: Supporting both async and sync code in Rust
In the early days of Rust there was a debate about whether to support "green threads" and in doing that require runtime support. It was actually implemented and included for a time but it creates problems when trying to do library or embedded code. At the time Go for example chose to go that route, and it was both nice (goroutines are nice to write and well supported) and expensive (effectively requires GC etc). I don't remember the details but there is a Rust RFC from when they removed green threads:
https://github.com/rust-lang/rfcs/blob/0806be4f282144cfcd55b...
-
Why stdout is faster than stderr?
I did some more digging. By RFC 899, I believe Alex Crichton meant PR 899 in this repo:
https://github.com/rust-lang/rfcs/pull/899
Still, no real discussion of why unbuffered stderr.
- Go: What We Got Right, What We Got Wrong
-
Ask HN: What's the fastest programming language with a large standard library?
Rust has had a stable SIMD vector API[1] for a long time. But, it's architecture specific. The portable API[2] isn't stable yet, but you probably can't use the portable API for some of the more exotic uses of SIMD anyway. Indeed, that's true in .NET's case too[3].
Rust does all this SIMD too. It just isn't in the standard library. But the regex crate does it. Indeed, this is where .NET got its SIMD approach for multiple substring search from in the first place[4]. ;-)
You're right that Rust's standard library is conservatively vectorized though[5]. The main thing blocking this isn't the lack of SIMD availability. It's more about how the standard library is internally structured, and the fact that things like substring search are not actually defined in `std` directly, but rather, in `core`. There are plans to fix this[6].
[1]: https://doc.rust-lang.org/std/arch/index.html
[2]: https://doc.rust-lang.org/std/simd/index.html
[3]: https://github.com/dotnet/runtime/blob/72fae0073b35a404f03c3...
[4]: https://github.com/dotnet/runtime/pull/88394#issuecomment-16...
[5]: https://github.com/BurntSushi/memchr#why-is-the-standard-lib...
[6]: https://github.com/rust-lang/rfcs/pull/3469
What are some alternatives?
image-spec - OCI Image Format
rust - Empowering everyone to build reliable and efficient software.
jemalloc
bubblewrap - Low-level unprivileged sandboxing tool used by Flatpak and similar projects
tinyrenderer - A brief computer graphics / rendering course
crates.io - The Rust package registry
dlmalloc - Doug Lea's memory allocator
polonius - Defines the Rust borrow checker.
mimalloc - mimalloc is a compact general purpose allocator with excellent performance.
Rust-for-Linux - Adding support for the Rust language to the Linux kernel.
glibc - Unofficial mirror of sourceware glibc repository. Updated daily.
rust-gc - Simple tracing (mark and sweep) garbage collector for Rust