ml-compiler-opt VS bloaty

I did a bit of work on this last summer on (much) smaller models [1] and it was briefly discussed towards the end of last year's MLGO panel [2]. For heuristic replacements specifically, you might be able to glean some things (or just use interpretable models like decision trees), but something like a neural network works fundamentally differently than the existing heuristics, so you probably wouldn't see most of the performance gains. For just tuning heuristics, the usual practice is to make most of the parameters configurable and then use something like bayesian optimization to try and find an optimal set, and this is sometimes done as a baseline in pieces of ML-in-compiler research.

1. https://github.com/google/ml-compiler-opt/pull/109

If you're using Clang/LLVM you can also enable ML inlining[1] (assuming you build from source) which can save up to around 7% if all goes well.

There are also talks of work on just brute forcing the inlining for size problem for embedded releases for smallish applications. It's definitely feasible if the problem is important enough to you to throw some compute at it [2].

1. https://github.com/google/ml-compiler-opt

2. https://doi.org/10.1145/3503222.3507744

LLVM's inlining-for-size and register-allocation-for-performance optimizations are both implemented using machine learning models trained by Google.

Continue reading | Checkout the paper, github, demo and ref article.

Looks like they do have a pretrained model:

https://github.com/google/ml-compiler-opt/releases/download/...

The code will by default auto-download it during the build process. It's about 800 kbytes, which seems very reasonable for something that will reduce the generated code size by gigabytes for a large codebase.

ESP32s aren't really ‘lower level’ in the sense that anyone is likely to write assembly code for them (compared to, say, 8051 or PIC), other than maybe some driver author at Espressif. The big win from using RISC-V, other than name recognition, is mainstream compiler support (which is nothing to sneeze at, especially when it's largely funded by someone else).

When I worked on Matter¹, the Xtensa and RISC-V versions were basically fungible from the software point of view. (And really, so were other vendors' various ARMs.) We did find that Bloaty McBloatface² didn't support Xtensa, so I had to write an alternative.

¹ https://github.com/project-chip/connectedhomeip/

² https://github.com/google/bloaty

I’ve gotten good insight into what takes up space in binaries by profiling with Bloaty (https://github.com/google/bloaty). My last profiling session showed that clang’s ThinLTO was inlining too aggressively in some cases, causing functions that should be tiny to be 75 kB+.

I'm surprised they didn't go for the binary size analysis tools like

https://github.com/google/bloaty

Or goweight.

I was able to shave a few % off our app binary with Bloaty. It’s pretty hard to use but once you figure out how to make regular expressions to properly classify things from your codebase, you can really visually analyze what your binary is composed of.

Bloaty