markup-experiments VS treenotation.org

Responding to some points I left off here https://news.ycombinator.com/item?id=33336789

I guess the main one is this:

> If your audience is people like me, I think it would probably be worthwhile for you to spend some time up front describing the intended semantics of a data model, as I've attempted above, rather than leaving people to infer it from the grammar. (Maybe OCaml is not a good way to explain it, though.) You might also want to specify that leading and trailing whitespace in prefixes is not significant, though it is in the suffix ("body"); this would enable people to format their name-value pairs readably without corrupting the data. As far as I can tell, this addendum wouldn't interfere with any of your existing uses for Jevko, though in some cases it would simplify their implementations.

You're right, things should be explained more clearly (TODO). Especially the exact role of Jevko and treatment of whitespace. I'll try to improve that.

Here is a sketch of an explanation.

Plain Jevko is meant to be a low-level syntactic layer.

It takes care of turning a unicode sequence into a tree.

On this level, all whitespace is preserved in the tree.

To represent key-value pairs and other data, you most likely want another layer above Jevko -- this would be a Jevko-based format, such as queryjevko (somewhat explained below) or, a very similar one, easyjevko, implemented and very lightly documented here: https://github.com/jevko/easyjevko.js

Or you could have a markup format, such as https://github.com/jevko/markup-experiments#asttoxml5

This format layer defines certain restrictions which may make a subset of Jevkos invalid in it.

It also specifies how to interpret the valid Jevkos. This includes the treatment of whitespace, e.g. that a leading or trailing whitespace in prefixes is insignificant, but conditionally significant in suffixes, etc.

Different formats will define different restrictions and interpretations.

For example:

# queryjevko

queryjevko is a format which uses (a variant of) Jevko as a syntax. Only a subset of Jevko is valid queryjevko.

> I think this is a more useful level of abstraction, and it's more or less the level used by, for example, queryjevko.js's jevkoToJs, although that erroneously uses () instead of [].

The `()` are used on purpose -- queryjevko is meant to be used in URL query strings and be readable. If square brackets were used, things like JS' encodeURIComponent would escape them, making the string unreadable. Using `()` solves that. "~" is used instead of "`" for the same reason. So technically we are dealing not with a spec-compliant Jevko, but a trivial variant of it. Maybe I should write a meta-spec which allows one to pick the three special characters before instantiating itself into a spec. Anyway the parser implementation is configurable in that regard, so I simply configure it to use "~()" instead of "`[]".

> (Also, contrary to your assertion above that this is an example of "leaving [Jevko's data model] as-is", it forgets the order of the name-value pairs as well as I guess all but one of any duplicate set of fields with the same name and also the possibility that there could be both fields and a body.)

I meant [whitespace] rather than [Jevko's data model].

Again, queryjevko is a format which uses Jevko as an underlying syntax. It specifies how syntax trees are converted to JS values, by restricting the range of valid Jevkos. It also specifies conversion in the opposite direction, likewise placing restrictions on JS values that can be safely converted to queryjevko.

The order of name-value pairs happens to get preserved (because of the way JS works), but that's not necessarily relevant. If I were to write a cross-language spec for queryjevko, I'd probably specify that this shouldn't be relied upon.

Duplicate fields and Jevkos with both fields and a non-whitespace body will produce an error when converting Jevko->JS.

I hope this clarifies things somewhat.

Lastly, I'll respond to this for completeness:

> (By the way, if you want to attribute your JSON example for copyright reasons, you need to attribute it to its author or authors, not to the Wikipedia, which is just the site they posted it on.)

According to this:

https://en.wikipedia.org/wiki/Wikipedia:Reusing_Wikipedia_co...

there are 3 options, one of them being what I did, which is to include a link.

I think that's all.

Have a good one!

Thank you. :)

> I wonder if I should use it for something...

I'd be honored!

A couple of ideas:

How about a simple configuration format? https://gist.github.com/djedr/681e0199859874b3324eaa84192c42... (I should make a library out of this)

Or you can put it in your query strings to make them more humane: https://github.com/jevko/queryjevko.js

Or make up a markup DSL: https://github.com/jevko/markup-experiments#asttohtmltable

Or serialize game objects in your indie game. Or make it the interface of your experimental app. Or use it to shave off a few unnecessary characters off your data: https://jevko.github.io/compactness.html

No parser in your favorite language? A basic one should be only a couple dozen lines!

There are no specialized features here but also not many parsing challenges or edge cases.

Clearly, there are ways to simplify these things, but not many people seem to be really interested in that.

[1] https://github.com/jevko/markup-experiments#asttoxml7

> concatenating them changes the label for [b] from "a" to "z\na", and perhaps more damningly, erases the whitespace before "z". But, since none of the alternative formats (except ndjson and I guess plain uninterpreted binary, ASCII, or Unicode) is closed under concatenation, maybe that's less important.

Yes, being closed under concatenation is a feature I was aiming for and it indeed does bring with it this issue.

Just something to have in mind when devising formats. A simple solution here is to disallow having anything other than whitespace in the suffix of a Jevko with > 0 children. Then, if a format converts these labels to keys in a map, trimming leading and trailing whitespace, there is no problem. This is how I did it here:

https://github.com/jevko/easyjevko.js

> I don't know if you saw the last time this topic came up I linked to https://ogdl.org/, which seems pretty close to a minimal rose-tree notation.

Yes, I've seen OGDL before. It's pretty nice. A similar one is https://treenotation.org/

I have experimented with indentation-based syntaxes myself, before settling on brackets.

I have found them to be problematic, at least because:

* For complex structures they become less compact.

* A grammar that correctly captures significant indentation can't really be written in pure BNF. The way OGDL does it is this:

  [12] space(n) ::= char_space*n ; where n is the equivalent number of spaces (can be 0)

This reminds me a bit of Breck Yunits' Tree Notation (https://treenotation.org/). Both seem to have a ~totalizing energy. Maybe some common cause. :)

Hi, I'm a programmer and I've used quite a few different languages in my career. I've never studied compilers or language design, however it has always interested me from afar. Also I've always had a strong preference for simple syntax, what sane person wouldn't? Anyway I've scanned over the https://treenotation.org/ site. I get the general gist, that this provides a tool to easily create languages that use tree notation. Unfortunately I still don't really understand how to use it. If there was tutorial that held your hand that would be really useful. I suspect there a large number of people like myself that would benefit from this. Perhaps at some point I'll role up my sleeves and do it myself, but I'm sure someone else could do a better job.

> The way to fix this trend would be to reimagine the presentation layer of the browser as something other than a stack of hacks over hypertext, but so far nobody seems to have a good solution.

About a decade ago I had the start of a Eureka moment on how to do this (back then — https://medium.com/space-net/spacenet-51aca95d49a2, nowadays https://treenotation.org/). It seems to me we've missed a sort of fundamental universal notation of the universe, which you can think of as "two-dimensional binary". I predict we will soon see a Cambrian Explosion of new formats and notations that are simpler and more interoperable with each other, and some will have the opportunity to build new great languages for rendering stacks.

Awesome app. Do you plan on using it for anything in particular? Or are you just creating it as a passion project. It's totally cool.

Learning about https://treenotation.org/ (linking this for other people, not for you, Breck :P), and I like what I see. My first impression was "Lisp, but with python indenting"

> We no longer need to store our data in error prone CSV, XML, or JSON. Tree Notation gives us a simpler, more powerful encoding for data with lots of new advanced features

This is the one thing I didn't understand! Tree notation seems equivalent to these. Like at a certain level, it's all just data. Now, the major benefit is that you're supposed to think differently about what you're doing when using tree notation. Would love to hear your opinion about this conjecture.

Lots of code examples here: https://jtree.treenotation.org/designer/

And the source for that homepage is here: https://github.com/treenotation/treenotation.org

Always open to PR!