Kedro VS Docusaurus

Interesting, thanks for sharing. I'll definitely take a look, although at this point I am so comfortable with Snakemake, it is a bit hard to imagine what would convince me to move to another tool. But I like the idea of composable pipelines: I am building a tool (too early to share) that would allow to lay Snakemake pipelines on top of each other using semi-automatic data annotations similar to how it is done in kedro (https://github.com/kedro-org/kedro).

# pyproject.toml [project] dependencies = [ "kedro @ git+https://github.com/kedro-org/kedro@3ea7231", "kedro-datasets[pandas.CSVDataSet,polars.CSVDataSet] @ git+https://github.com/kedro-org/kedro-plugins@3b42fae#subdirectory=kedro-datasets", ]

So there's 2 sides to pipeline management: the actual definition of the pipelines (in code) and how/when/where you run them. Some tools like prefect or airflow do both of them at once, but for the actual pipeline definition I'm a fan of https://kedro.org. You can then use most available orchestrators to run those pipelines on whatever schedule and architecture you want.

We have set up a milestone on GitHub so you can check in on our progress and contribute if you want to. To suggest features to us, report bugs, or just see what we're working on right now, visit the Kedro projects on GitHub.

you could use a project framework like kedro to force you to be more disciplined about how you structure your projects. I'd also recommend checking out this book: Edna Ridge - Guerrilla Analytics: A Practical Approach to Working with Data

Recently I started a position as Developer Advocate for Kedro, an opinionated data science framework, and one of the things we're doing is exploring what are the best open source tools we can use to create our documentation.

You can also check out Kedro, it’s like the Flask for data science projects and helps apply clean code principles to data science code.

An often overlooked framework used by NASA among others is Kedro https://github.com/kedro-org/kedro. Kedro is probably the simplest set of abstractions for building pipelines but it doesn't attempt to kill Airflow. It even has an Airflow plugin that allows it to be used as a DSL for building Airflow pipelines or plug into whichever production orchestration system is needed.

Docusaurus is a popular open-source documentation tool primarily designed for product documentation and other technical documentation needs. It was first released in 2017 by Facebook Open Source (now Meta Open Source). Just recently, Docsaurus version 3.0 was released.

// @ts-check // `@type` JSDoc annotations allow editor autocompletion and type checking // (when paired with `@ts-check`). // There are various equivalent ways to declare your Docusaurus config. // See: https://docusaurus.io/docs/api/docusaurus-config import { themes as prismThemes } from "prism-react-renderer"; /** @type {import('@docusaurus/types').Config} */ const config = { title: "My Site", tagline: "Dinosaurs are cool", url: "https://your-docusaurus-test-site.com", baseUrl: "/", onBrokenLinks: "throw", onBrokenMarkdownLinks: "warn", favicon: "img/favicon.ico", organizationName: "facebook", // Usually your GitHub org/user name. projectName: "docusaurus", // Usually your repo name. presets: [ [ "docusaurus-preset-openapi", /** @type {import('docusaurus-preset-openapi').Options} */ ({ docs: { sidebarPath: require.resolve("./sidebars.js"), // Please change this to your repo. editUrl: "https://github.com/facebook/docusaurus/tree/main/packages/create-docusaurus/templates/shared/", }, blog: { showReadingTime: true, // Please change this to your repo. editUrl: "https://github.com/facebook/docusaurus/tree/main/packages/create-docusaurus/templates/shared/", }, theme: { customCss: require.resolve("./src/css/custom.css"), }, }), ], ], themeConfig: /** @type {import('docusaurus-preset-openapi').ThemeConfig} */ ({ navbar: { title: "My Site", logo: { alt: "My Site Logo", src: "img/logo.svg", }, items: [ { type: "doc", docId: "intro", position: "left", label: "Tutorial", }, { to: "/api", label: "API", position: "left" }, { to: "/blog", label: "Blog", position: "left" }, { href: "https://github.com/facebook/docusaurus", label: "GitHub", position: "right", }, ], }, footer: { style: "dark", links: [ { title: "Docs", items: [ { label: "Tutorial", to: "/docs/intro", }, ], }, { title: "Community", items: [ { label: "Stack Overflow", href: "https://stackoverflow.com/questions/tagged/docusaurus", }, { label: "Discord", href: "https://discordapp.com/invite/docusaurus", }, { label: "Twitter", href: "https://twitter.com/docusaurus", }, ], }, { title: "More", items: [ { label: "Blog", to: "/blog", }, { label: "GitHub", href: "https://github.com/facebook/docusaurus", }, ], }, ], copyright: `Copyright © ${new Date().getFullYear()} My Project, Inc. Built with Docusaurus.`, }, prism: { theme: prismThemes.github, darkTheme: prismThemes.dracula, }, }), }; export default config;

Facebook's React/Markdown SSG docusaurus does those things: https://docusaurus.io/

Though you may have to use a plugin for responsive images: https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-id...

Docusaurus is an open-source static site generator built on React and has emerged as a popular tool for developing and maintaining product documentation. Its ease of use, extensive features, and robust community support make it a compelling choice for many organizations.

Wondering why Docusaurus (https://docusaurus.io) did not match their needs. Works perfectly fine as a blogging engine for our tech blog.

This is developed by Meta. You can create really nice-looking documentation websites super fast.

Docusaurus, a documentation tool by Facebook, hosts a showcase of other websites that use Docusaurus on their Homepage. The list of websites of this showcase is a typescript files that is maintained by Docusaurus devs, and that you can add your website to through PR: https://github.com/facebook/docusaurus/blob/main/website/src/data/users.tsx

Fix "Edit this page" links at the bottom of each doc (Problem with the Docusaurus build I guess)