pyyaml VS lua-patterns

PyYAML knew about the breakage since january 2022[0], and nothing really happened. After a year and a half with lots of alphas and betas, I don't think there is much cython could do, short of fixing PyYAML themselves.

[0]: https://github.com/yaml/pyyaml/issues/601

They did finally change the default, in PyYAML 6, after many many bugs pointing out that their previous approach is broken (including one by yours truly), so the default is now safe.

Note some parsers, most notably pyyaml are still at yaml 1.1, because 13 years is just not enough time to update it.

That part of the YAML 1.2 spec is in conflict with reality, though. The base of YAML 1.1 documents is large enough that a backwards-incompatible change to default behavior is for practical purposes impossible.

YAML 1.1 was released in 2005, and 1.2 in 2009 -- only four years later. But here we are, in 2022, and YAML 1.1 is still the default (in many cases, only) version supported. That's why the "Norway problem" persists -- it's not possible for the parser to know whether an un-versioned YAML document containing "a: no" should parse the same as {"a": false} or {"a": "no"}.

Python (PyYAML) doesn't support 1.2 yet: https://github.com/yaml/pyyaml/issues/116

Ruby (Psych) ditto -- I can't even find a tracking issue to enable it.

Go (go-yaml) is a mixture of YAML 1.1 and 1.2, depending on the author's preferences.

Also, as a rough guideline, you can't have a backwards-incompatible revision of a versioned spec declare that it's the new default version, because that breaks all existing users.

Many parsers either default to YAML pre-1.2 or do not even expose a YAML 1.2 option. PyYAML has no 1.2 option, for example. So unless Ansible is using something other than PyYAML...

Relevant (open) PR: https://github.com/yaml/pyyaml/pull/555

Are you perhaps talking about decimal.Decimal? https://github.com/yaml/pyyaml/issues/255

If you want automatic built-in string validation, one option that seems particularly interesting is to use a variant of Lua patterns, which are weaker and easier to understand than regular expressions, but still provide a significant degree of "sanity" for something like an email. The original version works on bytes and not runes, but you could simply write a parser that works on runes instead, and the pattern-matching code is just 400 old and battle-tested lines of C89.

Lua patterns have also shown up in other places, such as BSD's httpd, and an implementation for Rust:

https://www.gsp.com/cgi-bin/man.cgi?section=7&topic=PATTERNS

https://github.com/stevedonovan/lua-patterns

http://lua-users.org/wiki/PatternsTutorial