shlomi-fish-homepage VS generate-sitemap

Well, Plato considered craziness as divine (but excluded humour for example): https://en.wikipedia.org/wiki/Divine_madness . Later on, navi was mistranslated by the Septuaginte as "prophet", which also influenced later Jewish thought. Anyway, insane people at the time were likely used as sources of guidance and advice (see a WiP essay I wrote about that and "a navi" (נביא) was also a philosopher, educator, and entertainer (like the image macro based on https://www.youtube.com/watch?v=GoZ3bu23BMA jokes about).

generate-sitemap - Generate an XML sitemap for a GitHub pages site using GitHub Actions

I just released generate-sitemap 1.9.1, a GitHub Action for generating XML sitemaps for static websites. The generate-sitemap GitHub Action is implemented in Python, and generates an XML sitemap by crawling the GitHub repository containing the html of the site, using commit dates to generate tags in the sitemap.

I maintain several GitHub Actions, such as jacoco-badge-generator, generate-sitemap, javadoc-cleanup, and user-statistician. I've also written posts here on DEV about each of these if you'd like more information. GitHub's documentation for GitHub Action developers recommends maintaining a major release tag for the Action so that users can either reference the Action by its specific release tag, such as v1.2.3, or simply by the major release with v1. In fact, it is so commonplace that users will likely assume that your Action supports specifying full version tag or major tag only. Note that some Actions use major release branches (e.g., branch named v1) instead of tags. My intention in this post is not to discuss the advantages/disadvantages of each of these alternative approaches. In the Actions that I maintain, I use major release tags for the simple reason that it is what GitHub's documentation recommends.

We now need a way to detect if the results of the above integration tests are correct. The various actions that I maintain produce files (e.g., jacoco-badge-generator produces coverage badges, and generate-sitemap produces an XML sitemap) or edits existing files (e.g., javadoc-cleanup inserts canonical links and a few other things into the head of javadoc pages). In cases like these, I use Python's unittest module to validate the results. In this case, I define unit test cases in tests/integration.py that verify that the files produced by the action are correct. If any of those tests fail, then Python will exit with a non-zero exit code which will cause the workflow to fail.

On release and workflow_dispatch events, use the generate-sitemap GitHub Action, which I maintain, to generate an XML sitemap for the documentation website. The first step below generates the sitemap, and the second one below just logs some information about that action's run to the workflow run's log (e.g., number of URLs in the sitemap, and number excluded by either robots.txt or noindex directives).

I use GitHub Pages for my personal website, as well as for several project sites. Although some static site generators include support for sitemap generation (e.g., Jekyll has a plugin for sitemaps), my personal website is generated by a custom static site generator that I built for a few specialized reasons, and most of my project sites for Java libraries consist of a single hand-written HTML page combined with javadoc-generated documentation. So a while back I implemented a GitHub Action, generate-sitemap, that can generate an XML sitemap by crawling a GitHub repository containing the HTML of the site. It uses the last commit date of each file to produce the tags. By default, it includes URLs for HTML and PDF files in the sitemap, and skips other file extensions in the repository. But it can be configured to include URLs corresponding to whatever file extensions you want included. It checks the head of HTML pages for noindex meta tags, and excludes such files from the sitemap, and it likewise excludes files from the sitemap if they match a Disallow rule in your robots.txt. The generate-sitemap can be configured in a few other ways as well (see the documentation in the GitHub repository for all details). The generate-sitemap action is implemented in Python as a container action.

There are two primary ways of implementing a GitHub Action: JavaScript Actions and Container Actions. The latter of which enables implementing Actions in any language via a Docker container. My language of choice for implementing GitHub Actions is Python. The purpose of most of these actions is to produce files (e.g., jacoco-badge-generator produces test coverage badges as SVGs, and generate-sitemap produces an XML sitemap) or to edit files in some way (e.g., javadoc-cleanup can insert canonical links and other user-defined elements into the head of javadoc pages). However, all of these also produce workflow step outputs. For example, generate-sitemap has outputs for the number of pages in the sitemap, and the number of pages excluded from the sitemap due to noindex or robots.txt exclusions; and jacoco-badge-generator has workflow step outputs for the coverage and branches coverage percentages if a user had some reason to use those in later steps of their workflow.

Here is a basic example workflow template that combines my generate-sitemap action with other actions. When content is pushed to the repository, generate-sitemap walks the directory structure, using last commit dates to determine when each page was last modified, skipping any html files that have noindex directives, and outputting an xml sitemap. Another action is then used to generate a pull request if the sitemap changed.