arc42.org-site VS json-schema-spec

There a various standards for documenting software architecture, like arc42 or C4. While useful and somewhat well-known (there is certainly a correlation here), here architecture documentation can be further simplified, particularly due to the self-similarity of project and component. Following is a small template, that can also serve as a project's and component's README:

I've been using https://structurizr.com/ to automatically generate C4 diagrams from a model (rather than drawing them by hand). It works well with the approach for written documentation as proposed in https://arc42.org/. It's very easy to embed a C4 diagram into a markdown document.

The result is a set of documents and diagrams under version control that can be rendered using the structurizr documentation server (for interactive diagrams and indexed search).

I also use https://d2lang.com/ for declarative diagrams in addition to C4, e.g., sequence diagrams and https://adr.github.io/ for architectural decision records. These are also well integrated into structurizr.

My general approach to documentation is a "software guidebook" (free e-book link) or arc42 ... complemented with diagrams and architecture decision records where necessary.

In addition to a small number of diagrams, I'd recommend something like arc42 or my "software guidebook" (link for a free copy of my book describing this), plus some architecture decision records.

Arc42 may give some inspiration https://arc42.org/

If you need more structure than ADRs then https://arc42.org is super useful.

There are plenty of good resources out there for learning modeling languages and architectural documentation structures. For more lightweight methods, I'd recommend checking out resources like Simon Brown's C4 model for visualizing software architecture, Martin Fowler's UML Distilled: A Brief Guide to the Standard Object Modeling Language, Scott Ambler's Agile Modeling site, and the arc42 template for architectural documentation.

As you need to, you can get into topics like different modeling languages or documentation template libraries. These are going to be highly dependent on your organization, though, as different organizations have different standards for what and how to document design decisions. Personally, I've found that the arc42 documentation template plus the C4 modeling language plus the UML modeling language (see UML as sketch and UML as notes for more on lightweight UML) for the most detailed models (which are rarely needed outside of the most complex parts of a system) tends to be a good, agile, lightweight model for the kinds of things to think about when it comes to creating design documentation.

We've started using Arc42 and C4 model & PlantUML for diagrams. Working out well so far.

arc42 - arc42

models.tf where I centralized all the Data model that API Gateway uses to perform input and output checks. Those use the JSON-schema specification. GitHub - psantus/serverless.api-gateway-dynamodb-integration.terraform

In the meantime, we are going to expand our backend with two endpoints: one for fetching data and another one for creating data. Fastify provides out-of-the-box support for API serialization and validation through its schema-based approach built on top of JSON Schema. Through the schema option, we can attach a schema definition to each route.

JSON-schema to define templates for request and response contents.

The syntax used to define the output is called JSON Schema. It is a standard way to define the structure of a JSON object. If you know zod, the spirit is similar. Based on Swarmion's roadmap, it will be possible to use zod schemas to defined contracts in the future, which will be super cool!

Essentially, all it does is attempt to generate the parameters to hypothetical or potential functions, which you using a JSON schema describe to ChatGPT.

Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming.

Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We can transform this DFA into a generative model: in each state we get a list of symbols which correspond to completions that partially match the regular expression. We mask the other symbols in the logits returned by a large language model, sample a new symbol and move to the next state. The subtelty is that language models work with tokens, not symbols, so we derive a new FSM whose alphabet is the model's vocabulary. We can do this in only one pass over the vocabulary.

Generating the token masks thus only requires a dictionary lookup at each state. Our method blows other libraries like Microsoft's guidance out of the water.

From there it was only a small leap to be able to generate text that follows a JSON schema (https://json-schema.org/), or is parseable into a Pydantic model (https://docs.pydantic.dev/latest/usage/models/). The method works with union types, optional types, nested schemas, arrays, everything. It is guaranteed that the output is parseable.

I think it's cool, and I've spent a lot of time watching even tiny models output valid JSON over the weekend. Hope you will to.

I look forward to feedback, bug reports, feature requests and discussions!