lleaves VS zod

I built this decision tree (LightGBM) compiler last summer: https://github.com/siboehm/lleaves

It get's you ~10x speedups for batch predictions, more if your model is big. It's not complicated, it ended up being <1K lines of Python code. I heard a couple of stories like yours, where people had multi-node spark clusters running LightGBM, and it always amused me because by if you compiled the trees instead you could get rid of the whole cluster.

In a near-future version I'll expose some of the compilation parameters, I was somewhat afraid of having an API that's too complicated deterring people who just want a no-fuzz drop-in replacement for LightGBM. But as long as I keep sane defaults and have the parameters optional it should be fine. Relevant parameters are definitely block size (needs to adjust to L1i size and tree size) as well as the LLVM codemodel (a smaller adress space increases single-batch prediction speeds but doesn't work for large models). The thread-size specific compilation I'm still looking into, it makes the API more complicated and so might not be worth it.

Zod is an open-source schema declaration and validation library that emphasizes TypeScript. It can refer to any data type, from simple to complex. Zod eliminates duplicative type declarations by inferring static TypeScript types and allows easy composition of complex data structures from simpler ones. It has no dependencies, is compatible with Node.js and modern browsers, and has a concise, chainable interface. Zod is lightweight (8kb when zipped), immutable, with methods returning new instances. It encourages parsing over validation and is not limited to TypeScript but works well with JavaScript as well.

Zod is a TypeScript-first schema declaration and validation library. It helps create schemas for any data type and is very developer-friendly. Zod has the functional approach of "parse, don't validate." It supports coercion in all primitive types.

Link: https://zod.dev/

You can check out their documentation here.

You can learn more about Zod on their website here.

In JS, it's a good idea anyway to use some JSON parsing library instead of JSON.parse.

With Zod, you can use z.bigint() parser. If you take the "parse any JSON" snippet https://zod.dev/?id=json-type and change z.number() to z.bigint(), it should do what you are looking for.

We will validate our input using client-side zod. Zod handles TypeScript-first schema validation with static type inference. This means that it will not only validate your fields, it will also set types on validated fields.

A word of warning: while discriminated unions are very powerful, there's an ongoing discussion on whether discriminated unions should be deprecated and replaced with a different API.

I was first introduced to Zod by Adam Bobrow - a colleague of mine and a dear friend. Adam was sick and tired from JavaScript's brittleness, and about two years ago he started migrating our code base to TypeScript. But that wasn't enough for him. He kept complaining: "What good are my types, if some other service decides to send me bad data and breaks my code?". That's when he discovered Zod.