ferocity VS pidove

The advantage of the static import DSL is that it is more composable: anyone else can define operators that interoperate with my operators without the complexity of the scheme linked above not to mention more generality. This goes not just for the low level operators like "filter" that you might want to supplement but the many more operators you can write that are implemented out of mine.

I prototyped an API that works in the same direction as streams by having something like

https://paulhoule.github.io/pidove/apidocs/com/ontology2/pid...

in that it wraps all Iterables returned by my methods and has not just the teardown facility but also all the operators attached as instance methods which lets you write the chaining style you ask for that I know is in demand.

If I was going to go any further on pidove it would have involved more use of code generation and this system

https://github.com/paulhoule/ferocity

which was supposed to be a code generator for writing code generators, and it could code generate stubs that would let you write expression trees in Java as S-expressions and build them up into methods and either compile the code to real Java source code or execute the methods by evaluating the expression tree in place.

Like common LISP you can write syntactic macros in that that metalanguage because an Expression> can be evaluated at compile time, one of quite a few concepts like "quoting" that I encountered in that spike.

The idea was ferocity would get to the point where it synthesizes a more complete and perfect pidove.

but you can't have an overload that takes two differently parameterized expressions, this lets you write Java code in a lisp-like syntax that can be metaprogrammed on:

https://github.com/paulhoule/ferocity/blob/main/ferocity-std...

that project got me thinking a lot about the various ways types manifest in Java and I made the discovery that when you add Expression types you practically discover an extended type systems where there are many things that are implied by the existence of ordinary Java types.

I find this alien to my point of view. On the other hand, my side projects aren't driven by FOMO but are more like the "special interests" of autistic people.

Most of the time I have three side projects going on, maybe two of which are really getting the attention they deserve and one that is languishing. (See my profile to see about my current three.) Occasionally I get inspired to spend 1-4 weekends on some sudden inspiration, of which

https://github.com/paulhoule/pidove

came to completion but

https://github.com/paulhoule/ferocity

probably won't. The project I'm working the hardest on now is something that I was baffled that it didn't exist 18 years ago but felt compelled to do something out because of the Twitterinsanity last December and it turned out the technological conditions right now make it the perfect time to work on.

I have hacked off and on on this

https://github.com/paulhoule/ferocity

which lets you write extended Java in a lispy syntax. It generates stubs for the standard library and other packages you choose, unerasing types by putting them into method names. It works pretty well with IDEs but there are still problems w/ type erasure such that some kinds of type checking can't be done by the compiler working directly on the lispy Java, probably I wouldn't implement newer features such as pattern matching that are dependent on type inference to work, though lambda definitions are feasible if you give specific types.

The 'extended' bit was almost discovered instead of invented in that it is pretty obvious that you need quote and eval functions such that you can write lispy Java programs that manipulate Java expressions. Said expressions can be evaled at runtime with a primitive interpreter or incorporated into classes that are compiled w/ Javac. The motivation of the thing was to demonstrate Java-embedded-in-Java (an ugly kind of homoiconicity) and implement syntactic macros from Java which I think that prototype proves is possible but there is a lot more to be done on it to be really useful. Enough has been implemented in it right now in that you can use it to write the code generator that builds stubs. It might be good for balls-to-the-walls metaprogramming in Java but I think many will think it combines all the worse features of Java and Lisp.

There is this project

https://github.com/paulhoule/ferocity/

which I might finish up when I'm done with the report I'm writing. It is possible to create Java expression trees with trees of static method calls that look a lot like S-expressions and stick them together into statements, methods and classes.

You should be able to do the same tricks people do with LISP macros and it could work code generation miracles but it would have that "LISP curse" problem in spades.

The plan is to generate a code generator that is sufficient to generate the full DSL implementation (ferocity0) and use that to generate the full implementation (ferocity.)

I have some tests for ferocity0 writing .java files to get fed to javac and for ferocity0 running expression trees with a primitive interpreter. Already the type system is enriched over the type system because interpreted ferocity0 can handle Java expressions as a type at run time so you get issues like quoting and unquoting in LISP.

Internal or external?

I think Java is just fine for internal DSLs, see

https://www.jooq.org/

I was also hacking on this project

https://github.com/paulhoule/ferocity/

which was about making Java homoiconic. Namely in ferocity you can write

   Expression literal = of("Hello World");

You could process that Expression in a few different ways, for instance you could evaluate the tree using reflection or you could turn the tree into source code and compile it with javac.

Going down that path I found it was more about discovery rather than invention. That is, you will discover the issues that come up around quoting in LISP and develop some answers to them. Also for the exercise to be interesting at you will get into an execution model that is a bit bigger than the original language : for the metaprogramming to be fun at all you want to be able to write Expressions that manipulate Expressions so you end up introducing types that don't really exist in the base language. These are almost trivial to handle using the primitive interpreter style but could be more of a challenge to compile to source code.

Some source code is here

https://github.com/paulhoule/ferocity

It's been a long time since I worked on it and I need to see if I have more notes explaining the plan behind it, but the idea was to develop "ferocity0" which was a version of the DSL that was good enough to build stubs for some of the language and a large part of the standard library, and then use code generation techniques to create "ferocity1" which would be like "ferocity0" but would cover 100% of the Java language.

My current side project is being sidelined by supply chain problems so I might go back into that one.

Streams is too complex for what it does and it doesn’t even parallelize well. Here is something that does roughly the same thing but I think is way better

See https://github.com/paulhoule/pidove

https://central.sonatype.com/artifact/com.ontology2/pidove

(1) It's a bit of a bad smell (which he points out) that records aren't being used much at all in the Java stdlib, I wrote something that built out stubs for the 17 and 18 stdlibs and that stood out like a sore thumb. I do like using records though.

(2) I've looked at other ways to extend the collections API and related things, see

https://github.com/paulhoule/pidove

and I think the sequenced collections could have been done better.

(3) Virtual Threads are kinda cool but overrated. Real Threads in Java are already one of the wonders of the web and perform really well for most applications. The cases where Virtual Threads are really a win will be unusual but probably important for somebody. It's a good thing it sticks to the threads API as well as it did because I know in the next five years I'm going to find some case where somebody used Virtual Threads because they thought it was cool and I'll have to switch to Real Threads but won't have a hard time doing so.

I think that guy just made up a generator class for fun. It’s not too different from the integrator except it doesn’t have a hasNext() method so it either returns results forever or it has to return a sentinel value like null or return an exception to end iteration.

Somebody could make the case that returning a sentinel value or an exception is a better API since there is no risk somebody else is going to call the next() method after you call hasNext() and next(). Writing a generator that wraps a generator is a little simpler than writing an interest or that wraps an iteration because you don’t have to write a hasNext() function, which can occasionally be awkward.

That generator library has a few functions, like map that work on generators, unfortunately the Java stdlib doesn’t come with anything like that. (There is the streams API but it is over-complicated.)

I’ll point out this library I wrote

https://github.com/paulhoule/pidove

which does a lot of what the Steams library does but it works on iterators without creating streams. If you like those generator examples you might like pidove.

As for Python it is kinda accidental that generators would up related to coroutines, that is, generators were an easy way to implement coroutines, later async/await and stuff like that got added.

I find this alien to my point of view. On the other hand, my side projects aren't driven by FOMO but are more like the "special interests" of autistic people.

Most of the time I have three side projects going on, maybe two of which are really getting the attention they deserve and one that is languishing. (See my profile to see about my current three.) Occasionally I get inspired to spend 1-4 weekends on some sudden inspiration, of which

https://github.com/paulhoule/pidove

came to completion but

https://github.com/paulhoule/ferocity

probably won't. The project I'm working the hardest on now is something that I was baffled that it didn't exist 18 years ago but felt compelled to do something out because of the Twitterinsanity last December and it turned out the technological conditions right now make it the perfect time to work on.

When I saw sequenced collections earlier I didn’t like the design but I completely approve of the latest revision. One nice thing about the process they use to develop Java is that they really do work and rework new features to make them great.

I just wish that instead of Streams they’d made something more like

https://github.com/paulhoule/pidove

... or you can just use a sane FP library like

https://github.com/paulhoule/pidove

Some people don't like the Lispy signatures so I did start coding up a version with with a fluent interface but didn't quite finish.

Inner classes are pretty useful.

This library contains a huge number of Iterables, each of which has at least one Iterator implementation.

https://github.com/paulhoule/pidove

It is convenient to let the Iterator be immutable and the Iterator be an inner class that gets its configuration information out of the Iterable.

(That said, if people really thought seriously about Iterator being a Supplier people might think more rationally about error handling. Also in a slightly parallel universe the Iterator would only have one method since remove() hardly ever gets used and having both hasNext() and next() methods is asking for bugs.)

If you were programming in Python or many other languages you could download documentation locally.

In both Python and Java doing a mini-project I frequently challenge myself to only use the standard library. It's good for practicing HackerRank-rank style programming (the fun of single-file Java)

When I am waiting for builds I sometimes hack on this

https://github.com/paulhoule/pidove

because I don't really like the Streams API and want to perfect my mastery of generics and internal DSLs.

Now that I think of it, standard-library only for node seems like a good challenge for me because I code a lot of front-end Javascript but just barely know the node API.