Chips-n-Salsa VS generate-sitemap

name: docs on: push: branches: [ master ] paths: [ '**.java', '.github/workflows/docs.yml' ] pull_request: branches: [ master ] release: types: [created] workflow_dispatch: jobs: docs: runs-on: ubuntu-latest steps: - name: Checkout the repo uses: actions/checkout@v3 - name: Checkout the gh-pages branch uses: actions/checkout@v3 with: fetch-depth: 0 ref: gh-pages path: gh-pages - name: Set up JDK 17 uses: actions/setup-java@v3 with: distribution: 'adopt' java-version: '17' - name: Build docs with Maven run: mvn compile javadoc:javadoc - name: Copy to Documentation Website Location if: ${{ github.event_name == 'release' || github.event_name == 'workflow_dispatch' }} run: | rm -rf gh-pages/api cp -rf target/site/apidocs/. gh-pages/api - name: Tidy up the javadocs if: ${{ github.event_name == 'release' || github.event_name == 'workflow_dispatch' }} id: tidy uses: cicirello/javadoc-cleanup@v1 with: base-url-path: https://chips-n-salsa.cicirello.org/ path-to-root: gh-pages user-defined-block: | - name: Log javadoc-cleanup output if: ${{ github.event_name == 'release' || github.event_name == 'workflow_dispatch' }} run: | echo "modified-count = ${{ steps.tidy.outputs.modified-count }}" - name: Commit documentation changes without pushing yet if: ${{ github.event_name == 'release' || github.event_name == 'workflow_dispatch' }} run: | cd gh-pages if [[ `git status --porcelain` ]]; then git config --global user.name 'github-actions' git config --global user.email '41898282+github-actions[bot]@users.noreply.github.com' git add -A git commit -m "Automated API website updates." fi cd .. - name: Generate the sitemap if: ${{ github.event_name == 'release' || github.event_name == 'workflow_dispatch' }} id: sitemap uses: cicirello/generate-sitemap@v1 with: base-url-path: https://chips-n-salsa.cicirello.org/ path-to-root: gh-pages - name: Output stats if: ${{ github.event_name == 'release' || github.event_name == 'workflow_dispatch' }} run: | echo "sitemap-path = ${{ steps.sitemap.outputs.sitemap-path }}" echo "url-count = ${{ steps.sitemap.outputs.url-count }}" echo "excluded-count = ${{ steps.sitemap.outputs.excluded-count }}" - name: Commit documentation website sitemap and push all commits if: ${{ github.event_name == 'release' || github.event_name == 'workflow_dispatch' }} run: | cd gh-pages if [[ `git status --porcelain` ]]; then git config --global user.name 'github-actions' git config --global user.email '41898282+github-actions[bot]@users.noreply.github.com' git add -A git commit -m "Automated API website sitemap update." fi git push cd ..

Website: https://chips-n-salsa.cicirello.org/

The complete workflow file that this example is derived from is maven-publish.yml.

Let's look at a real example, again from Chips-n-Salsa, where I publish artifacts of the library on both Maven Central as well as to GitHub Packages. We'll build up to the full example. First, consider the configuration below:

To see a live example, you can consult the pom.xml of one of my projects. Here is the GitHub repository:

GitHub Language Chart From https://github.com/cicirello/Chips-n-Salsa

Website: https://chips-n-salsa.cicirello.org/

Important Note: Because of the way JitPack essentially builds a snapshot of your repository, even for release builds, the jitpack.yml must be present within the specific snapshot of your repository in order to apply. For example, for the library that this post is based upon, Chips-n-Salsa, the first release where the repository contains the configuration file is 5.0.1, so JitPack will fail to build any prior version, although earlier versions are available via Maven Central and GitHub Packages. Likewise, if you specify any commit hash prior to that of the commit where that configuration file was introduced, the JitPack build will also fail.

Chips-n-Salsa - A Java library of customizable, hybridizable, iterative, parallel, stochastic, and self-adaptive local search algorithms

generate-sitemap - Generate an XML sitemap for a GitHub pages site using GitHub Actions

I just released generate-sitemap 1.9.1, a GitHub Action for generating XML sitemaps for static websites. The generate-sitemap GitHub Action is implemented in Python, and generates an XML sitemap by crawling the GitHub repository containing the html of the site, using commit dates to generate tags in the sitemap.

I maintain several GitHub Actions, such as jacoco-badge-generator, generate-sitemap, javadoc-cleanup, and user-statistician. I've also written posts here on DEV about each of these if you'd like more information. GitHub's documentation for GitHub Action developers recommends maintaining a major release tag for the Action so that users can either reference the Action by its specific release tag, such as v1.2.3, or simply by the major release with v1. In fact, it is so commonplace that users will likely assume that your Action supports specifying full version tag or major tag only. Note that some Actions use major release branches (e.g., branch named v1) instead of tags. My intention in this post is not to discuss the advantages/disadvantages of each of these alternative approaches. In the Actions that I maintain, I use major release tags for the simple reason that it is what GitHub's documentation recommends.

We now need a way to detect if the results of the above integration tests are correct. The various actions that I maintain produce files (e.g., jacoco-badge-generator produces coverage badges, and generate-sitemap produces an XML sitemap) or edits existing files (e.g., javadoc-cleanup inserts canonical links and a few other things into the head of javadoc pages). In cases like these, I use Python's unittest module to validate the results. In this case, I define unit test cases in tests/integration.py that verify that the files produced by the action are correct. If any of those tests fail, then Python will exit with a non-zero exit code which will cause the workflow to fail.

On release and workflow_dispatch events, use the generate-sitemap GitHub Action, which I maintain, to generate an XML sitemap for the documentation website. The first step below generates the sitemap, and the second one below just logs some information about that action's run to the workflow run's log (e.g., number of URLs in the sitemap, and number excluded by either robots.txt or noindex directives).

I use GitHub Pages for my personal website, as well as for several project sites. Although some static site generators include support for sitemap generation (e.g., Jekyll has a plugin for sitemaps), my personal website is generated by a custom static site generator that I built for a few specialized reasons, and most of my project sites for Java libraries consist of a single hand-written HTML page combined with javadoc-generated documentation. So a while back I implemented a GitHub Action, generate-sitemap, that can generate an XML sitemap by crawling a GitHub repository containing the HTML of the site. It uses the last commit date of each file to produce the tags. By default, it includes URLs for HTML and PDF files in the sitemap, and skips other file extensions in the repository. But it can be configured to include URLs corresponding to whatever file extensions you want included. It checks the head of HTML pages for noindex meta tags, and excludes such files from the sitemap, and it likewise excludes files from the sitemap if they match a Disallow rule in your robots.txt. The generate-sitemap can be configured in a few other ways as well (see the documentation in the GitHub repository for all details). The generate-sitemap action is implemented in Python as a container action.

There are two primary ways of implementing a GitHub Action: JavaScript Actions and Container Actions. The latter of which enables implementing Actions in any language via a Docker container. My language of choice for implementing GitHub Actions is Python. The purpose of most of these actions is to produce files (e.g., jacoco-badge-generator produces test coverage badges as SVGs, and generate-sitemap produces an XML sitemap) or to edit files in some way (e.g., javadoc-cleanup can insert canonical links and other user-defined elements into the head of javadoc pages). However, all of these also produce workflow step outputs. For example, generate-sitemap has outputs for the number of pages in the sitemap, and the number of pages excluded from the sitemap due to noindex or robots.txt exclusions; and jacoco-badge-generator has workflow step outputs for the coverage and branches coverage percentages if a user had some reason to use those in later steps of their workflow.

Here is a basic example workflow template that combines my generate-sitemap action with other actions. When content is pushed to the repository, generate-sitemap walks the directory structure, using last commit dates to determine when each page was last modified, skipping any html files that have noindex directives, and outputting an xml sitemap. Another action is then used to generate a pull request if the sitemap changed.