unify-jdocs
fq
unify-jdocs | fq | |
---|---|---|
9 | 45 | |
71 | 9,441 | |
- | - | |
5.7 | 9.4 | |
4 months ago | 7 days ago | |
Java | Go | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
unify-jdocs
-
How to Use JSON Path
JSONPath is good when it comes to querying large JSON documents. But in my opinion, more than this is the need to simplify reading and writing from JSON documents. We use POJOs / model classes which can become a chore for large JSON documents. While it is possible to read paths, I had not seen any tool using which we could read and write JSON paths in a document without using POJOs. And so I wrote unify-jdocs - read and write any JSON path with a single line of code without ever using POJOs. And also use model documents to replace JSONSchema. You can find this library here -> https://github.com/americanexpress/unify-jdocs.
-
I created another JSON –> Java mapper (it's better)
I went the other way - because I so much disliked working with DTOs to work with JSON, I wrote a library that allows you to get rid of DTOs in the first place. No more POJO / DTO in order to work with JSON data and much more. You could take a look at https://github.com/americanexpress/unify-jdocs. I think you will find it interesting!
-
I spent a year building an App and failed – The Story of Taskwer
"And that’s when a real problem emerged; nobody knew who I was" and "I felt invisible, like I was all alone in this world. I could have had the best thing in the world, and no one would care": Could not agree with you more on this - over the past many years it has been more and more important to build a digital brand for yourself and have many followers - to be networked with people and know people who can provide you with reach - something which is difficult for introverts and challenging to do now since I have crossed 50. I personally have felt this helplessness in trying to promote a couple of opensource Java libraries I released on behalf of my employer (shameless plug here -> https://github.com/americanexpress/unify-jdocs and https://github.com/americanexpress/unify-flowret). I thought I had done something of value which would be readily adopted by people after they saw it - guess what - first I have not been able to get through to a wide audience and second - I underestimated what it takes to get people out of their comfort zone and their traditional thinking. It is so difficult for people to accept that there may be better ways of doing things once they get used to a certain way. Anyway, I don't think you failed - you learnt and it is never too late to learn and do something new. Something will click eventually and I wish you all the very best.
-
Show HN: Unify-jdocs – read / write any JSON path with a single line of code
Auther here - thanks for responding! I really do appreciate it.
> The trend seems to be in the opposite direction. People became frustrated with the lack of types in Python and JavaScript, hence we get Python with typing and TypeScript.
Regarding typing, what I have tried to achieve is the best of both worlds. In unify-jdocs, we have the concept of typed document. This is a document that defines the structure of the JSON document. It defines what the "type" of a leaf node is i.e. integer, string etc. The validation / determination against this type is however done at runtime. This, I feel is acceptable because whenever we add a JSON path to a document, the first thing we would do is to test the read / write of that path. And any type mismatch would get caught there immediately. And so, from the point of view of being able to read / write / validate in a single line of code (even though dynamically) provides much simplicity and ease of use as compared to using POJOs. Plus we always know the exact JSON path we are dealing with.
Usually, I would prefer static typing but, in a scenario, where we can have hundreds of JSON document types (we deal with more than 500 in the same application), complex JSON document (ours go down more than ten levels deep with hundreds of JSON paths) and where the document structures may undergo change over project lifetime, the use of read / writing using a single line of code has many benefits. I shudder to think of hundreds (if not thousands) of POJO classes, the writing of accessor methods (null / empty handling, namespace etc.) and what it would take to refactor in the face of change. Just my opinion based on my experience in the past.
> I think you will find less traction with this method of posting to HN. People want a clickable link to look at the project
The clickable link to the repo / documentation is actually in the text (https://github.com/americanexpress/unify-jdocs). In the case of this post, I felt it more important for people to read the text rather than be directly pointed to the contents of the link and hence this approach.
- Show HN: Unify-jdocs has come a long way
-
A Journey building a fast JSON parser and full JSONPath
Nice work! I see that that this is for processing / parsing large data sets and where documents do not conform to a fixed structure and for Go language.
I made something similar in Java - unify-jdocs - https://github.com/americanexpress/unify-jdocs - though this is not for parsing - it is more for reading and writing when the structure of the document is known - read and write any JSONPath in one line of code and use model documents to define the structure of the data document (instead of using JSONSchema which I found very unwieldy to use) - no POJOs or model classes - along with many other features. Posting here as the topic is relevant and it may help people in the Java world. We have used it intensively within Amex for a very large complex project and it has worked great for us.
-
Show HN: A tool to Convert JSON schemas into TypeScript classes
Nice! Talking of JSON schemas and validating JSON documents against schemas, for Java, I wrote unify-jdocs where I do not use JSON schemas but still do validations (I found them unwieldy to use and was looking for something simpler). You can find details here -> https://github.com/americanexpress/unify-jdocs. Also, no POJOs / model classes, just reading and writing JSON paths in a single line of code. It's helped us tremendously in managing complexity in a very large internal project. I am hoping it helps others.
-
On the Complexity of JSON Serialization
This article got my attention. Related to what you are saying, in Java, the problem that I was really fed up of was creating domain specific JSON object models to map the JSON documents into to use in code. In other words, mapping JSON to rigidly typed language structure. Its boiler plate, is tedious to do (as the author points out in the article), difficult to change and usually a pain. I solved this problem by creating unify-jdocs which completely eliminates the need to create object models or POJO classes to represent your JSON object. You can read more about it here -> https://github.com/americanexpress/unify-jdocs
fq
-
How to Use JSON Path
I see, thanks for replying and no worries! yeap some of the "self-describing" formats like msgpack, cbor etc will because of how fq works have to be decoded into something more of a meta-msgpack etc.
About blobs, if you want to change how (possibly large) binaries are represented as JSON you can use the bits_format options, see https://github.com/wader/fq/blob/master/doc/usage.md#options, so fq -o bits_format=md5 torepr ...
I can highly recommend to learn jq, it's what makes fq really useful, and as a bonus you will learn jq in general! :)
-
Reverse-engineering an encrypted IoT protocol
Hey! fq author here. I have a bunch of related tools in the readme https://github.com/wader/fq?tab=readme-ov-file#tools two suggestions: gnu poke and wireshark (can decode lots of more things then just network protocol)
-
To a Man with `Jq`, Everything Looks Like JSON
Did someone say let's represent structured data as json? a bit of shameless plug: https://github.com/wader/fq :) It's using a fork of gojq btw!
-
Jaq – A jq clone focused on correctness, speed, and simplicity
https://github.com/wader/fq has a REPL and can read JSON. Tip is to use "paste | from_json | repl" in a REPl to paste JSON into a sub-REPL, you can also use `` with fq which is a raw string literal
-
jq 1.7 Released
I do lots of exploratory work in various structure data, in my case often debugging media filea via https://github.com/wader/fq, which mean doing lots of use-once-queries on the command line or REPL. In those cases jq line-friendly and composable syntax and generators really shine.
- fq (jq for binary formats) has a new v0.7.0 version
- FLaNK Stack 5-June-2023
- fq: jq for binary formats - tool, language and decoders for working with binary and text formats
- Fq: Jq for Binary Formats
- GitHub - wader/fq: jq for binary formats - tool, language and decoders for working with binary and text formats
What are some alternatives?
jsonschema2pojo - Generate Java types from JSON or JSON Schema and annotate those types for data-binding with Jackson, Gson, etc
jq - Command-line JSON processor [Moved to: https://github.com/jqlang/jq]
REST Assured - Java DSL for easy testing of REST services
jq - Command-line JSON processor
jsog - JavaScript Object Graph
Kaitai Struct - Kaitai Struct: declarative language to generate binary data parsers in C++ / C# / Go / Java / JavaScript / Lua / Nim / Perl / PHP / Python / Ruby
hof - Framework that joins data models, schemas, code generation, and a task engine. Language and technology agnostic.
HexFiend - A fast and clever hex editor for macOS
unify-flowret - A lightweight Java based orchestration engine
nq - Unix command line queue utility
Paste JSON as Code • quicktype - Xcode extension to paste JSON as Swift, Objective-C, and more
miller - Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON