enforce VS returnn

Every point in this blog post strikes me as either (1) unaware of the tooling around python typing other than mypy, or (2) a criticism of static-typing-bolted-on-to-a-dynamically-typed-language, rather than Python's hints. Regarding (1), my advise to OP is to try out Pyright, Pydantic, and Typeguard. Pyright, especailly, is amazing and makes the process of working with type hints 2 or 3 times smoother IMO. And, I don't think points that fall under (2) are fair criticisms of type *hints*. They are called hints for a reason.

Otherwise, here's a point-by-point response, either recommending OP checks out tooling, or showing that the point being made is not specific to Python.

> type hints are not binding.

There are projects [0][1] that allow you to enforce type hints at runtime if you so choose.

It's worth mentioning that this is very analogous to how Typescript does it, in that type info is erased completely at runtime.

> Type checking is your job after all, ...[and that] requires maintenance.

There are LSPs like Pyright[2] (pyright specifically is the absolute best, IMO) that report type errors as you code. Again, this is very very similar to typescript.

> There is an Any type and it renders everything useless

I have never seen a static-typing tool that was bolted on to a dynamically typed language, without an `Any` type, including typescript.

> Duck type compatibility of int and float

The author admits that they cannot state why this behavior is problematic, except for saying that it's "ambiguous".

> Most projects need third-party type hints

Again, this is a criticism of all cases where static types are bolted on dynamically typed languages, not Python's implementation specifically.

> Sadly, dataclasses ignore type hints as well

Pydantic[3] is an amazing data parsing library that takes advantage of type hints, and it's interface is a superset of that of dataclasses. What's more, it underpins FastAPI[4], an amazing API-backend framework (with 44K Github stars).

> Type inference and lazy programmers

The argument of this section boils down to using `Any` as a generic argument not being an error by default. This is configurable to be an error both in Pyright[5], and mypy[6].

> Exceptions are not covered [like Java]

I can't find the interview/presentation, but Guido Van Rossum specifically calls out Java's implementation of "exception annotations" as a demonstration of why that is a bad idea, and that it would never happen in Python. I'm not saying Guido's opinion is the absolute truth, but just letting you know that this is an explicit decision, not an unwanted shortcoming.

[0] https://github.com/RussBaz/enforce

[1] https://github.com/agronholm/typeguard

[2] https://github.com/microsoft/pyright

[3] https://pydantic-docs.helpmanual.io

[4] https://github.com/tiangolo/fastapi

[5] https://github.com/microsoft/pyright/blob/main/docs/configur...

[6] https://mypy.readthedocs.io/en/stable/config_file.html#confv...

Not by default. But there are libraries to enforce types. https://github.com/RussBaz/enforce or/and https://pydantic-docs.helpmanual.io/

That looks very interesting.

I actually have developed (and am developing) sth very similar, what we call the RETURNN frontend, a new frontend + new backends for our RETURNN framework. The new frontend is supporting very similar Python code to define models as you see in PyTorch or Keras, i.e. a core Tensor class, a base Module class you can derive, a Parameter class, and then a core functional API to perform all the computations. That supports multiple backends, currently mostly TensorFlow (graph-based) and PyTorch, but JAX was something I also planned. Some details here: https://github.com/rwth-i6/returnn/issues/1120

(Note that we went a bit further ahead and made named dimensions a core principle of the framework.)

(Example beam search implementation: https://github.com/rwth-i6/i6_experiments/blob/14b66c4dc74c0...)

One difficulty I found was how design the API in a way that works well both for eager-mode frameworks (PyTorch, TF eager-mode) and graph-based frameworks (TF graph-mode, JAX). That mostly involves everything where there is some state, or sth code which should not just execute in the inner training loop but e.g. for initialization only, or after each epoch, or whatever. So for example:

- Parameter initialization.

- Anything involving buffers, e.g. batch normalization.

- Other custom training loops? Or e.g. an outer loop and an inner loop (e.g. like GAN training)?

- How to implement sth like weight normalization? In PyTorch, the module.param is renamed, and then there is a pre-forward hook, which on-the-fly calculates module.param for each call for forward. So, just following the same logic for both eager-mode and graph-mode?

- How to deal with control flow context, accessing values outside the loop which came from inside, etc. Those things are naturally possible eager-mode, where you would get the most recent value, and where there is no real control flow context.

- Device logic: Have device defined explicitly for each tensor (like PyTorch), or automatically eagerly move tensors to the GPU (like TensorFlow)? Moving from one device to another (or CPU) is automatic or must be explicit?

I see that you have keras_core.callbacks.LambdaCallback which is maybe similar, but can you effectively update the logic of the module in there?

> warnings of IDEs are simple to ignore

This is unusual. In my experience, of codebases I have worked with or have seen, when there are type hints, there are almost all perfectly correct.

Also, you can setup the CI to check also for IDE warnings. For example, we use this script for PyCharm: https://github.com/rwth-i6/returnn/blob/master/tests/pycharm...

The test for PyCharm inspections only passes when there are no warnings.

Although, I have to admit, we explicitly exclude type warnings because here we have a couple of false positives. So in this respect, it actually agrees with the article.

But then we also do code review and there we are strict about having it all correct.

Yes, I see the argument of the article that the typing in Python is not perfect and you can easily fool it if you want, so you cannot 100% trust the types. But given good standard practice, it will only rarely happen that the type is not as expected and typing helps a lot. And IDE type warnings, or mypy checks still are useful tools and catch bugs for you, just not maybe 100% of all typing bugs but still maybe 80% of them or so.

> Isn’t it better to detect at least some errors than to detect none at all?

I was trying to implement some new feature in some larger somewhat messy project (RETURNN but not so relevant).

So I created a new branch, also made a GitHub draft PR (here), and started working on it.