manylinux VS docker-node

I use manylinux containers as the OS for compilation. It tries to ensure as much cross-os / libc / etc.. as much as possible for precompiled libraries. https://github.com/pypa/manylinux

Just to clarify for anyone who isn't aware, the "compiling issues", at least historically, have been that that Alpine uses musl, and PyPI's manylinux wheels are built against old glibc versions. So stuff like numpy that would trivially and quickly install from whl on glibc distros (like a bare-bones Ubuntu image) trigger compilations and the installation of build-only dependencies on Alpine.

That said, it looks like as of late-2021, at least some projects are offering musllinux wheels as well, per the discussion here: https://github.com/pypa/manylinux/issues/37 (not numpy, though: https://pypi.org/project/numpy/#files)

It's very hard. Incompatible glibc ABIs make this nigh impossible, there's a reason Steam installs a vcredistributable.dll for pretty much every game on Windows.

Look no further than the hoops you need jump through to distribute a Linux binary on PyPI [1]. Despite tons of engineering effort, and tons of hoop jumping from packagers, getting a non-trivial binary to run across all distros is still considered functionally impossible.

[1]: https://github.com/pypa/manylinux

As a user, if you build every python package from source, it's ok. But if you a maintainer of an OSS project and you need to publish binary packages for it, then you will hit the trouble. Binaries built on Ubuntu 20.04 can only support Ubuntu 20.04 and newer. So you'd better to choose an older Linux release to target broader users. Now most python packages choose CentOS 6 or 7. See https://github.com/pypa/manylinux/issues/1012 for more details. They need help!

I recently learned that Clang supports this kind of cross-compiling out of the box. https://mcilloni.ovh/2021/02/09/cxx-cross-clang/

The main difference is that Clang does not ship with headers/libraries for different platforms, as Zig appears to do. You need to give Clang a "sysroot" -- a path that has the headers/libraries for the platform you want to compile for.

If you create a bunch of sysroots for various architectures, you can do some pretty "easy" cross-compiling with just a single compiler binary. Docker can be a nice way of packaging up these sysroots (especially combined with Docker images like manylinux: https://github.com/pypa/manylinux). Gone are the days when you had to build a separate GCC cross-compiler for each platform you want to target.

Now you come and use manylinux to build. (https://github.com/pypa/manylinux) so you are based on the CentOS 7 toolchain (at best if you use manylinux2014) or Debian 9 toolchain (if you use manylinux_2_24).

I think the generally accepted way to do that would be a container image running a relatively old distribution. This is exactly what python packages do when they need to distribute binary packages on linux [0]. You are supposed to compile the package in a container (or VM) that runs CentOS 7 (or older if you want broader support), although now the baseline is moving gradually to Debian 9.

[0]: https://github.com/pypa/manylinux

FROM node:16-alpine AS base FROM base AS deps # Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed. RUN apk add --no-cache libc6-compat WORKDIR /app COPY package.json package-lock.json* ./ RUN npm ci FROM base AS builder WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY . . RUN npm run build # Production image, copy all the files and run next FROM base AS runner WORKDIR /app ENV NODE_ENV production COPY --from=builder /app/.next/standalone ./ COPY --from=builder /app/.next/static ./.next/static COPY --from=builder /app/public ./public EXPOSE 3000 ENV PORT 3000 ENV HOSTNAME localhost CMD ["node", "server.js"]

This instructs docker to start building our image from an existing node image based on Alpine Linux. Alpine distribution is the smallest Linux distribution which allows building lightweight images.

FROM node:21-alpine AS base FROM base AS builder Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed. RUN apk add --no-cache libc6-compat RUN apk update Set working directory WORKDIR /app Install pnpm with corepack RUN corepack enable && corepack prepare pnpm@latest --activate Enable pnpm add --global on Alpine Linux by setting home location environment variable to a location already in $PATH https://github.com/pnpm/pnpm/issues/784#issuecomment-1518582235 ENV PNPM_HOME=/usr/local/bin RUN pnpm install turbo --global COPY . . RUN turbo prune web --docker Add lockfile and package.json's of isolated subworkspace FROM base AS installer RUN apk add --no-cache libc6-compat RUN apk update WORKDIR /app First install the dependencies (as they change less often) COPY .gitignore .gitignore COPY --from=builder /app/out/json/ . COPY --from=builder /app/out/pnpm-workspace.yaml ./pnpm-workspace.yaml COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml RUN pnpm install Build the project COPY --from=builder /app/out/full/ . RUN pnpm turbo run build --filter=web FROM base AS runner WORKDIR /app Don't run production as root RUN addgroup --system --gid 1001 nodejs RUN adduser --system --uid 1001 nextjs USER nextjs COPY --from=installer /app/apps/web/next.config.js . COPY --from=installer /app/apps/web/package.json . Automatically leverage output traces to reduce image size https://nextjs.org/docs/advanced-features/output-file-tracing COPY --from=installer --chown=nextjs:nodejs /app/apps/web/.next/standalone ./ COPY --from=installer --chown=nextjs:nodejs /app/apps/web/.next/static ./apps/web/.next/static COPY --from=installer --chown=nextjs:nodejs /app/apps/web/public ./apps/web/public CMD node apps/web/server.js

FROM node:18-alpine AS base # Install dependencies only when needed FROM base AS deps # Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed. RUN apk add --no-cache libc6-compat WORKDIR /app # Install dependencies based on the preferred package manager COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./ RUN \ if [ -f yarn.lock ]; then yarn --frozen-lockfile; \ elif [ -f package-lock.json ]; then npm ci; \ elif [ -f pnpm-lock.yaml ]; then yarn global add pnpm && pnpm i --frozen-lockfile; \ else echo "Lockfile not found." && exit 1; \ fi # Rebuild the source code only when needed FROM base AS builder WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY . . # Next.js collects completely anonymous telemetry data about general usage. # Learn more here: https://nextjs.org/telemetry # Uncomment the following line in case you want to disable telemetry during the build. # ENV NEXT_TELEMETRY_DISABLED 1 RUN yarn build # If using npm comment out above and use below instead # RUN npm run build # Production image, copy all the files and run next FROM base AS runner WORKDIR /app ENV NODE_ENV production # Uncomment the following line in case you want to disable telemetry during runtime. # ENV NEXT_TELEMETRY_DISABLED 1 RUN addgroup --system --gid 1001 nodejs RUN adduser --system --uid 1001 nextjs COPY --from=builder /app/public ./public # Set the correct permission for prerender cache RUN mkdir .next RUN chown nextjs:nodejs .next # Automatically leverage output traces to reduce image size # https://nextjs.org/docs/advanced-features/output-file-tracing COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./ COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static USER nextjs EXPOSE 3000 ENV PORT 3000 # set hostname to localhost ENV HOSTNAME "0.0.0.0" # server.js is created by next build from the standalone output # https://nextjs.org/docs/pages/api-reference/next-config-js/output CMD ["node", "server.js"]

Official docker image for node is built from Alpine or Debian [1]

Forgive me if I don't believe that running a full OS on a host OS to run a single node command doesn't amount to running a VM.

[1] https://github.com/nodejs/docker-node/tree/main/20

This is Node's Docker image.

Since the project is written in Node.js, we need to find a Node.js environment on Docker Hub. We can find the official Node image on Docker Hub by searching for "Node.js".

This node image is just one of a range published by the Node.js team (https://hub.docker.com/_/node), and they also have the Dockerfile for their build available

do you mean container image? npm comes in the "node" container image https://hub.docker.com/_/node

I'm trying to setup a docker container for running a node js app. I'm a bit of a newbie when it comes to docker, but from what I've read all I need to do is "Launch docker image" and enter the image name from the docker repo. In this case: https://hub.docker.com/_/node My config looks like this. After that the UI just says deploying forever. I must be missing something obvious, any ideas?