user-statistician VS sirix

The cicirello/user-statistician GitHub Action generates a detailed visual summary of your activity on GitHub in the form of an SVG suitable to display on your GitHub Profile README Although the intended use-case is to generate an SVG image for your GitHub Profile README you can also potentially link to the image from a personal website, or from anywhere else where you'd like to share a summary of your activity on GitHub. The SVG that the action generates includes statistics for the repositories that you own, your contribution statistics (e.g., commits, issues, PRs, etc), as well as the distribution of languages within public repositories that you own The user stats image can be customized, including the colors such as with one of the built-in themes or your own set of custom…

Contributing a language translation mostly involves creating a new JSON file named with the ISO 639-1 two-character code for the language, or for languages that don't have a two-character code, the ISO 639-2 three-character language code. Then within that JSON file, translating all of the string values. You also need to add the locale code to a Python set of supported locales within src/StatConfig.py.

I maintain several GitHub Actions, such as jacoco-badge-generator, generate-sitemap, javadoc-cleanup, and user-statistician. I've also written posts here on DEV about each of these if you'd like more information. GitHub's documentation for GitHub Action developers recommends maintaining a major release tag for the Action so that users can either reference the Action by its specific release tag, such as v1.2.3, or simply by the major release with v1. In fact, it is so commonplace that users will likely assume that your Action supports specifying full version tag or major tag only. Note that some Actions use major release branches (e.g., branch named v1) instead of tags. My intention in this post is not to discuss the advantages/disadvantages of each of these alternative approaches. In the Actions that I maintain, I use major release tags for the simple reason that it is what GitHub's documentation recommends.

The complete workflow for this project is found at: build.yml. The repository itself is:

I use GitHub Actions to automate a variety of things in nearly all of my repositories, such as running a build and tests during pull-requests and pushes, deploying artifacts to Maven Central, etc for my Java libraries, or to PyPI for a couple Python projects, building my personal website with my custom static site generator, among a variety of other tasks. In addition to using GitHub Actions for workflow automation, I also develop and maintain a few Actions (all implemented in Python), including jacoco-badge-generator, user-statistician, javadoc-cleanup, and generate-sitemap.

If you want to generate the equivalent to the above for your own GitHub profile, check out the cicirello/user-statistician GitHub Action.

The cicirello/user-statistician GitHub Action generates a detailed visual summary of your activity on GitHub in the form of an SVG suitable to display on your GitHub Profile README Although the intended use-case is to generate an SVG image for your GitHub Profile README you can also potentially link to the image from a personal website, or from anywhere else where you'd like to share a summary of your activity on GitHub. The SVG that the action generates includes statistics for the repositories that you own, your contribution…

The user-statistician GitHub Action is implemented in Python as a container action. You don't need to know any Docker to contribute, as you won't need to touch the Dockerfile, and the unit tests can run locally with Python alone (the unit tests don't actually query the GitHub API, instead using fake query results). If you know how to add elements to a Python dictionary, then your Python background is sufficient. There is a single Python file that you would need to edit: StatConfig.py. There is a comment within that has an itemized list of what is required to contribute a language translation, and mostly involves adding your translation of the various headings and labels, as well as a translation of the title template, to a couple of Python dictionaries, and adding the language code to a Python set.

Just in time for Halloween, and Hacktoberfest, I recently added a few Halloween themes to the user-statistician GitHub Action. I've posted about the user-statistician GitHub Action before. It generates an SVG with a detailed summary of your activity on GitHub suitable for inclusion in your GitHub Profile README or on a personal website. The intended use-case is to run on a schedule via a GitHub workflow in your GitHub Profile repository (repository with same name as your username). It is implemented in Python as a Container Action, and uses the GitHub CLI to query the GitHub GraphQL API to gather the data. For a more detailed summary of its functionality, see my earlier DEV post as well as other posts in this series:

In this post, I explain how to configure GitHub's Linguist within your repository to enable more accurate and more relevant repository language reporting, with examples from a few of my own repositories. Every repository on GitHub has a chart that shows the distribution of languages detected in the repository. GitHub's Linguist is responsible for detecting the language of each file within your repository, and the reported percentages are based on file sizes. For example, "Java 50%" means that 50% of the total size of all detected files in the repository are Java files. There are also third party tools that display language statistics, such as the user-statistician GitHub Action that I developed and maintain, which includes on an SVG (among other things) a pie chart summarizing the language distribution across all of your public repositories (excluding forks). The language data necessary to generate that language chart comes from GitHub's GraphQL API, which is as it is reported for each of your repositories by Linguist.

We're using a similar trie structure as the main document (node) index in SirixDB[1]. Lately, I got some inspiration for different page-sizes based on the ART and HAMT basically for the rightmost inner pages (as the node-IDs are generated by a simple sequence generator and thus also all inner pages (we call them IndirectPage) except for the rightmost are fully occupied (the tree height is adapted dynamically depending on the size of the stored data. Currently, always 1024 references are stored to indirect child pages, but I'll experiment with smaller sized, as the inner nodes are simply copied for each new revision, whereas the leaf pages storing the actual data are versioned themselfes with a novel sliding snapshot algorithm.

You can simply compute from a unique nodeId each data is assigned (64bit) the page and reference to traverse on each level in the trie through some bit shifting.

[1] https://github.com/sirixdb/sirix

I'm working on something similar for the JVM, however with no document semantics, but on a much more fine granular level.

JSON is shredded during an initial import into a tree structure with fine granular nodes. Thus, an import can be done with very low memory consumption (permitted that auto-commit issues a sync to disk before RAM space is exceeded). Furthermore, it doesn't require a WAL for consistency. Instead the indexes are stored in a log-structure using a persistent tree (as in every commit creates a new tree root). A sliding snapshot algorithm makes sure, that only a fragment of a page has to be copied on a write.

As thus, it's also a perfect candidate for an event store, storing both the (lightweight) snapshots and tracking the changes optionally.

https://github.com/sirixdb/sirix

The architecture is described over here:

https://sirix.io/docs/concepts.html

Furthermore I'm working on a tutorial for a local client usage (work in progress):

https://sirix.io/docs/jsoniq-tutorial.html

Kind regards

If anyone is up to building a new frontend, that would be awesome (of course, work could also be split between interested people) :-)

https://github.com/sirixdb/sirix/issues/627

I'm working on an append-only (immutable) (bi)temporal DBS[1] in my spare time, which transforms CRUD operations into an event store, automatically providing an audit log for each stored node, while the nodes are stored with immutable node-IDs, which never change. As the contents stored are based on a custom binary JSON format also a rolling hash can optionally be built, to check if a whole subtree has changed or not.

The system uses persistent index data structures to share unchanged pages between revisions.

The intermittant snapshots are omitted. Rather the snapshot is spread over several revisions, applying a sliding snapshot algorithm on the data pages (thus, avoiding write peaks, while at max a predefined number of page fragments has to be read in parallel to reconstruct a page in-memory).

[1] https://sirix.io | https://sirix.io/docs/concepts.html

I've already posted the project a couple of years ago and it gained some interest, but a lot of stuff has been done since then, especially regarding performance, a complete new JSON store, a REST API, various internals refactored, an improved JSONiq based query engine allowing updates, a now already dated web UI, a new Kotlin based CLI, a Python and TypeScript client to ease the use of Sirix...

First prototypes from a precursor stem already from 2005.

So, what is it all about?

I'm working on an evolutionary data store in my spare time[1]. It is based on the idea to get rid of the need for a second trx log (the WAL) by using a persistent tree of tries (preserving the previous revision through copy on write and path copying to the root) index as the log itself with only a single permitted read/write txn concurrently and in parallel to N read-only txns, which are bound to specific revisions during the start. The single writer is permitted on a resource (comparable to a table/relation in a relational DB) basis within a database, reads do not involve any locks at all.

The idea is, that the system atomically swaps the tree root to the new version (replicated). If something fails the log can simply be truncated to the former tree root.

Thus, the system has many similarities with Git (structural sharing of unchanged nodes/pages) and ZFS snapshots (regarding the latter the keyed trie has been inspired by ZFS, as well as that checksums for child pages are stored in parent pages in the references to the child pages)[2].

You can of course simply execute time travel queries on the whole revision history, add commit comments and the author to answer questions such as who committed what at which point in time and why...

The system not only copies full data pages, but it applies a sliding snapshot versioning algorithm to keep storage space to a minimum.

Thus, it's best suited for fast flash drives with fast random reads and sequential writes. Data is never overwritten, thus audit trails are given for free.

The system stores find granular JSON nodes, thus the structure and size of an object has almost no limits. A path summary is built, which is an unordered set of all paths to leaf nodes in the tree and enables various optimizations. Furthermore a rolling hash is optionally built, whereas during inserts all ancestor node hashes are adapted.

Furthermore it optionally keeps track of update operations and the ctx nodes involved during txn commits. Thus, you can easily get the changes between revisions, you can check the full history of nodes, as well as navigate in time to the first revision, the last revision, the next and previous revision of a node...

You can also open a revision at a specific system time revert to a revision and commit a new version while preserving all revisions in-between.

As said one feature is, that the objects can be arbitrarily nested, thus almost no limits in the number and updates are cheap.

A dated Jupyter notebook with some examples can be found in [3] and overall documentation in [4].

The query engine[5] Brackit is retargetable (a couple of interfaces and rewrite rules have to be implemented for DB systems) and especially finds implicit joins and applies known algorithms from the relational DB systems world to optimize joins and aggregate functions due to set-oriented processing of the operators.[6]

I've given an interview in [7], but I'm usually very nervous, so don't judge too harshly.

Give it a try and happy coding!

Kind regards

Johannes

[1] https://sirix.io | https://github.com/sirixdb/sirix

[2] https://sirix.io/docs/concepts.html

[3] https://colab.research.google.com/drive/1NNn1nwSbK6hAekzo1YbED52RI3NMqqbG#scrollTo=CBWQIvc0Ov3P

[4] https://sirix.io/docs/

[5] http://brackit.io

[6] https://colab.research.google.com/drive/19eC-UfJVm_gCjY--koOWN50sgiFa5hSC

[7] https://youtu.be/Ee-5ruydgqo?si=Ift73d49w84RJWb2

You can use Datomic for instance (mentioned already in your article IIRC!?) or SirixDB[1] on sich I'm working in my spare time.

The idea is an indexed append-only log-structure and to use a functional tree structure (sharing unchanged nodes between revisions) plus a novel algorithm to balance incremental and full dumps of database pages using a sliding window instead.

[1] https://sirix.io | https://github.com/sirixdb/sirix

Append-only database system (based on a persistent inddx structure): https://github.com/sirixdb/sirix or a retargetable query compiler https://github.com/sirixdb/brackit

You can work on a temporal data store called SirixDB: https://github.com/sirixdb/sirix