ziglyph VS arocc

> My big problem with Zig is that Andrew Kelley is promising a lot of features, but doesn't really deliver much.

Have you, like, seen the release notes for 0.9.0?

https://ziglang.org/download/0.9.0/release-notes.html

> Zig still can't proper handle UTF-8 strings [1] in 2022

There's plenty of discussion on the subject in basically every HN thread about Zig: the stdlib has some utf8 and wtf validation code, ziglyph implements the full unicode spec.

https://github.com/jecolon/ziglyph

You might not like how it's done, but its factually incorrect to state that Zig can't handle unicode.

> In a `recent` interview[2], he claims that Zig is faster than C and Rust, but he refers to extremely short benchmarking that has almost no value in the real world.

From my reddit reply to this same topic:

This podcast interview might not be the best showcase of the practical implications of Zig's take on safety and performance. If you want something with more meat, I highly recommend Andrew's recent talk from Handmade Seattle, where he shows the work being done on the Zig self-hosted compiler.

https://media.handmade-seattle.com/practical-data-oriented-d...

Lots of bit fiddling that can't be fully proven safe statically, but then you get a compiler capable of compiling Zig code stupidly fast, and that's even without factoring in incremental compilation with in-place binary patching, with which we're aiming for sub-millisecond rebuilds of arbitrarily large projects.

> The ecosystem for zig is insignificant now and a stable release would help the language.

I hope you don't mind if we don't take this advice, given the overall tone of your post.

For Unicode text processing you can take a look at Ziglyph https://github.com/jecolon/ziglyph and for a sample UTF-8 string structure, Zigstr https://github.com/jecolon/zigstr . (bias alert: I'm the author of both. :^D )

Agreed, and Zig also has a lib for that as well:

https://github.com/jecolon/ziglyph/

Yes, sorry about that - I omitted a bit of that information for brevity.

If you want to play with allkeys.txt (which is by far much more sequential, simpler data than UnicodeData.txt) then you only need to remove the non-NFD strings (since the Unicode Collation Algorithm's first step requires you to decompose the string's code points to canonical NFD form), that removes ~2,000 entries.

The full file parser code, which strips those out and other useless information like comments and version information can be found here: https://github.com/jecolon/ziglyph/blob/main/src/collator/Al...

If you want to play around with UnicodeData.txt (which is less sequential, more complex data) then only two fields are used (the code point and decomposition field), and only records where the second field is not empty (the full decomposition type name in angle brackets is not needed, only whether it is or is not there is important.)

The full parser code for that file can be found here: https://github.com/jecolon/ziglyph/blob/main/src/normalizer/...

Hope that helps!

Possible reference as it requires to use the compiler as part of language abi: https://github.com/Vexu/arocc/issues/178 Not sure, where a better thread with explanations of the flaws is.

Zig calls clang to compile C code. This doesn't add a new dependency since Zig already depends on LLVM. In the future when Zig doesn't depend as much on LLVM, there might be a reason to use a C compiler written in Zig (e.g. https://github.com/Vexu/arocc)

Bit field rules are underspecified or plain wrongly implemented, because in their edge cases clang and GCC differ in semantics. See https://github.com/Vexu/arocc/issues/178 This should be further restricted with static asserts as compiler semantics even changed with versions and doing this manually/doing code review is error prone.

You might want to contribute or look into https://github.com/Vexu/arocc, which is planned to be eventually an alternative frontend. Is arocc able to handle your use cases?

> Does this mean that y'all are open to the self-hosted compiler supporting CPU architectures unlikely to ever have LLVM support?

Yes! We won't block 1.0 on the quality of the less mainstream targets, but that's what the tier system is for - to ship a compiler that has varying levels of quality for various targets, while communicating clearly to users what kind of experience they can expect for each one.

SuperH patches are absolutely welcome.

> how is zig cc anticipated to work with a self-hosted Zig? Will there be a dependency on clang [...]?

The main distribution of Zig will be LLVM/Clang-enabled. However it is already possible to build a version of Zig that does not have these features enabled. In such case, compiling C, C++, and Objective-C code will result in an error.

However, the arocc project[1] is emerging, which, depending on a combination of how much funding ZSF gets and how much enthusiasm the unpaid contributors working in their spare time have, is looking like a promising C frontend that would be available even without LLVM/Clang. It is C only, however, with no intention of compiling C++ or Objective-C.

> would zig cc support the planned C backend?

As it is currently implemented: no. Zig invokes clang to turn C source code into object files.

However, with the arocc frontend above, this would be converting the C source code into ZIR (or perhaps AIR), which could then be lowered with any of the backends, including the (partially complete) C backend. In such case, the C output would look drastically different than the input. It would look more like an IR than natural C code that a human would write.

[1]: https://github.com/Vexu/arocc

Implementing only the language part takes like 10k LOC.

> 9. Without a C compiler, we're stuck with, wedded to, and beholden to libclang.

> I wouldn't be surprised that the eventual cost of adapting ourselves to

> libclang will exceed the cost of doing our own C compiler.

This is a really insightful point. I had to learn this the hard way :)

We might follow your lead on this, as we have done with so many other great ideas implemented in D.

Ironically, Vexu started from the other side as you, with the preprocessor mostly done, but the backend left to-do: https://github.com/Vexu/arocc

One thing that might make libclang worth the cost, however, is its ability to compile C++ code as well. On Zig's end of things, all we have to do is provide libcxx, libcxxabi, libunwind, compiler-rt, and linking, and then libclang is really pulling a lot of weight by compiling C++ code into object files. Sadly this ability is just too useful in practice to ignore. For example, LLVM itself is C++ so if Zig wants to be able to bootstrap itself, it needs this capability.

Still, I think your maneuver here is the best long-term approach to tackle this problem, and I imagine as time goes on we'll start to migrate towards D's solution here. Maybe someday the Zig distribution that does not have LLVM extensions enabled will be the more popular one!

I'll be watching the evolution of this new feature in D with great interest!