hadolint VS incubation-engineering

To skip the "move your scripts to standalone files" step some devs don't like, consider something like https://github.com/hadolint/hadolint which runs Shellcheck over inline scripts within Containerfiles.

This is neat :)

I love going and making containers smaller and faster to build.

I don't know if it's useful for alpine, but adding a --mount=type=cache argument to the RUN command that `apk add`s might shave a few seconds off rebuilds. Probably not worth it, in your case, unless you're invalidating the cached layer often (adding or removing deps, intentionally building without layer caching to ensure you have the latest packages).

Hadolint is another tool worth checking out if you like spending time messing with Dockerfiles: https://github.com/hadolint/hadolint

With Depot, we make use of two Dockerfile linters, hadolint and a set of Dockerfile linter rules that Semgrep has written to make a bit of a smarter Dockerfile linter.

# Download hadolint wget https://github.com/hadolint/hadolint/releases/download/v2.12.0/hadolint-Linux-x86_64 # Download SHA256 checksum wget https://github.com/hadolint/hadolint/releases/download/v2.12.0/hadolint-Linux-x86_64.sha256 # Validate the checksum sha256sum -c hadolint-Linux-x86_64.sha256 # Make the file executable chmod + ./hadolint-Linux-x86_64 # Rename the file mv hadolint-Linux-x86_64 hadolint

The most common linter I've seen and used it Hadolint, which does: https://github.com/hadolint/hadolint/wiki/DL3002 I didn't bother checking to see if alternatives also support this as well though.

Some discussion on that here:

https://github.com/koalaman/shellcheck/issues/58

The hadolint project does shell checking for Dockerfiles and it uses shellcheck:

https://github.com/hadolint/hadolint

So the approach is definitely feasible, but you do need a new project and probably it needs to be written in Haskell.

how does this compare to something like hadolint?

Also, have you run across Hadolint for linting? https://github.com/hadolint/hadolint

However if you really want to optimize data currently residing in Postgres for analytical workloads, as the original comment suggests - consider moving to a dedicated OLAP DB like ClickHouse.

See results from Gitlab benchmarking ClickHouse vs TimescaleDB: https://gitlab.com/gitlab-org/incubation-engineering/apm/apm...

Key findings:

ansible: stage: configure image: alpine rules: - if: $ANSIBLE_SETUP_VM != "" && $ANSIBLE_SETUP_HOST != "" variables: ANSIBLE_HOST_KEY_CHECKING: "False" script: - apk add curl bash openssh python3 py3-pip - pip3 install ansible paramiko - ansible-galaxy collection install -r ansible/requirements.yml - curl --silent "https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/download-secure-files/-/raw/main/installer" | bash - mkdir /root/.ssh && cp .secure_files/ansible.priv /root/.ssh/id_rsa && chmod 600 /root/.ssh/id_rsa - ansible-playbook ansible/main.yml -i ansible/inventory --extra-vars vyos_host=$ANSIBLE_SETUP_VM --limit $ANSIBLE_SETUP_HOST,$ANSIBLE_SETUP_VM ```

Interesting to match with the observations from the practice of using ClickHouse[1][2] for time series:

1. Reordering to SOA helps a lot - this is the whole point of column-oriented databases.

2. Specialized codecs like Gorilla[3], DoubleDelta[4], and FPC[5] lose to simply using ZSTD[6] compression in most cases, both in compression ratio and in performance.

3. Specialized time-series DBMS like InfluxDB or TimescaleDB lose to general-purpose relational OLAP DBMS like ClickHouse [7][8][9].

[1] https://clickhouse.com/blog/optimize-clickhouse-codecs-compr...

[2] https://github.com/ClickHouse/ClickHouse

[3] https://clickhouse.com/docs/en/sql-reference/statements/crea...

[4] https://clickhouse.com/docs/en/sql-reference/statements/crea...

[5] https://clickhouse.com/docs/en/sql-reference/statements/crea...

[6] https://github.com/facebook/zstd/

[7] https://arxiv.org/pdf/2204.09795.pdf "SciTS: A Benchmark for Time-Series Databases in Scientific Experiments and Industrial Internet of Things" (2022)

[8] https://gitlab.com/gitlab-org/incubation-engineering/apm/apm... https://gitlab.com/gitlab-org/incubation-engineering/apm/apm...

[9] https://www.sciencedirect.com/science/article/pii/S187705091...

Documentation of rules is now available: https://gitlab.com/gitlab-org/incubation-engineering/ai-assist/dokter/-/blob/main/docs/overview.md