magika VS meson

As someone that has worked in a space that has to deal with uploaded files for the last few years, and someone who maintains a WASM libmagic Node package ( https://github.com/moshen/wasmagic ) , I have to say I really love seeing new entries into the file type detection space.

Though I have to say when looking at the Node module, I don't understand why they released it.

Their docs say it's slow:

https://github.com/google/magika/blob/120205323e260dad4e5877...

It loads the model an runtime:

https://github.com/google/magika/blob/120205323e260dad4e5877...

They mark it as Experimental in the documentation, but it seems like it was just made for the web demo.

Also as others have mentioned. The model appears to only detect 116 file types:

https://github.com/google/magika/blob/120205323e260dad4e5877...

Where libmagic detects... a lot. Over 1600 last time I checked:

https://github.com/file/file/tree/4cbd5c8f0851201d203755b76c...

I guess I'm confused by this release. Sure it detected most of my list of sample files, but in a sample set of 4 zip files, it misidentified one.

We are very excited to announce the release of Magika our AI powered fast and efficient file type identification lib and tool - https://opensource.googleblog.com/2024/02/magika-ai-powered-fast-and-efficient-file-type-identification.html

Thanks to its optimized Keras model, large scale training dataset, and Onnx Magika massively outperform other file identification tools while be very fast even on CPU.

Magika python code and model is open sourced on Github: https://github.com/google/magika and we also provide an experimental TFJS based npm package https://www.npmjs.com/package/magika

With the team we hope you will find Magika useful for your own projects. Let us know what you think or if you have any question!

[1]: https://github.com/mesonbuild/meson/issues/8153

The Meson build system Version: 1.2.3 Source dir: /home/patrik/workspace/Waybar Build dir: /home/patrik/workspace/Waybar/build Build type: native build Project name: waybar Project version: 0.9.24 C compiler for the host machine: cc (gcc 13.2.0 "cc (Debian 13.2.0-5) 13.2.0") C linker for the host machine: cc ld.bfd 2.41 C++ compiler for the host machine: c++ (gcc 13.2.0 "c++ (Debian 13.2.0-5) 13.2.0") C++ linker for the host machine: c++ ld.bfd 2.41 Host machine cpu family: x86_64 Host machine cpu: x86_64 Compiler for C++ supports link arguments -lc++fs: NO Compiler for C++ supports link arguments -lc++experimental: NO Compiler for C++ supports link arguments -lstdc++fs: YES Program git found: YES (/usr/bin/git) WARNING: You should add the boolean check kwarg to the run_command call. It currently defaults to false, but it will default to true in future releases of meson. See also: https://github.com/mesonbuild/meson/issues/9300 Has header "filesystem" : YES Checking if "nl_langinfo with _NL_TIME_WEEK_1STDAY, _NL_TIME_FIRST_WEEKDAY" : links: YES Run-time dependency threads found: YES Found pkg-config: /usr/bin/pkg-config (1.8.1) Run-time dependency fmt found: YES 9.1.0 Run-time dependency spdlog found: YES 1.12.0 Run-time dependency wayland-client found: YES 1.22.0 Run-time dependency wayland-cursor found: YES 1.22.0 Run-time dependency wayland-protocols found: YES 1.32 Run-time dependency gtkmm-3.0 found: YES 3.24.8 Run-time dependency dbusmenu-gtk3-0.4 found: YES 16.04.0 Run-time dependency gio-unix-2.0 found: YES 2.78.1 Run-time dependency jsoncpp found: YES 1.9.4 Run-time dependency sigc++-2.0 found: YES 2.12.1 Found CMake: /usr/bin/cmake (3.27.7) Run-time dependency libinotify found: NO (tried pkgconfig and cmake) Run-time dependency epoll-shim found: NO (tried pkgconfig and cmake) Run-time dependency libinput found: YES 1.23.0 Run-time dependency libnl-3.0 found: YES 3.7.0 Run-time dependency libnl-genl-3.0 found: YES 3.7.0 Run-time dependency upower-glib found: YES 1.90.2 Run-time dependency libpipewire-0.3 found: YES 0.3.85 Run-time dependency playerctl found: YES 2.4.1 Run-time dependency libpulse found: YES 16.1 Run-time dependency libudev found: YES 252 Run-time dependency libevdev found: YES 1.13.1 Run-time dependency libmpdclient found: YES 2.20 Run-time dependency xkbregistry found: YES 1.6.0 Run-time dependency jack found: YES 0.126.0 Run-time dependency wireplumber-0.4 found: YES 0.4.15 Library sndio found: YES Checking for function "sioctl_open" with dependency -lsndio: YES Run-time dependency gtk-layer-shell-0 found: YES 0.8.1 Run-time dependency systemd found: YES 252 Computing int of "__cpp_lib_chrono" : 201611 Configuring waybar.service using configuration Run-time dependency cava found: NO (tried pkgconfig and cmake) Looking for a fallback subproject for the dependency cava Executing subproject cava cava| Project name: cava cava| Project version: 0.9.1 cava| C compiler for the host machine: cc (gcc 13.2.0 "cc (Debian 13.2.0-5) 13.2.0") cava| C linker for the host machine: cc ld.bfd 2.41 cava| Has header "iniparser.h" : NO cava| Has header "iniparser4/iniparser.h" : NO Message: cava is not found. Building waybar without cava subprojects/cava-0.9.1/meson.build:65:3: ERROR: Problem encountered: iniparser library is required A full log can be found at /home/patrik/workspace/Waybar/build/meson-logs/meson-log.txt WARNING: Running the setup command as `meson [options]` instead of `meson setup [options]` is ambiguous and deprecated.

As it happens, I recently landed a PR in meson to add a clang-like Weverything mode that includes all of that, so you can get a minimal list of more or less all GCC warnings, organized by version, from the meson source here: https://github.com/mesonbuild/meson/blob/710a753c78077220b13a9f7e999dcdb61339efb1/mesonbuild/compilers/mixins/gnu.py

Came here to post the same. The answer for How to build software? is Meson[1] for C and C++ and also other languages. Works well on Windows and Mac, too.

I’ve written a small Makefile to learn the basic and backgrounds. Make is fine. But the next high-level would have been Autotools, which is an intimidating and weird set of tools. Most new stuff written in C/C++ use now Meson and it feels sane.

[1] https://mesonbuild.com

If you are very fortunate, you'll be able to choose something else. I like meson myself: it looks a bit like python, it's popular, small, simple, well-documented, easy to install and update, and it works well everywhere.

I suggest changing the build tool. Meson improved C and C++ a lot:

https://mesonbuild.com/

The dependency declaration and auto-detection is nice. But the hidden extra is WrapDB, built-in package management (if wanted):

    https://mesonbuild.com/Wrap-dependency-system-manual.html

> C's only REAL problem (in my opinion) which is the lack of dependency management. Most everything else can be done with a makefile and a half decent editor.

Care to hear about our lord and saviour Meson?

Both of your quoted problems are mutually incompatible: dependency management isn't the job of the compiler, it's a job for the build or host system. If you want to keep writing makefiles, be prepared to write your own `wget` and `git` invocations to download subprojects.

Meanwhile, Meson solves the dependency management problem in a way that makes both developers and system integrators/distributions happy. It forces you to make a project that doesn't have broken inter-file or header dependency chains and cleans up all the clutter and cruft of a makefile written for any non-trivial project, while making it trivial to integrate other meson projects into your build, let other people integrate your project into theirs, and provides all of the toggles and environment variables distribution developers need to package your library properly. You can really have your cake and eat it too.

https://mesonbuild.com/

Other people going down this route seem to end up writing cmake replacements instead. I'm thinking of something like meson here except that meson never intended to transpile to cmake.