CKAN VS ArchiveBox

CKAN The Open Source Data Portal Software

We use Kubernetes as our deployment platform. Any feedback on one of these open source data catalogs ? - https://atlas.apache.org/#/ - https://opendatadiscovery.org/ - https://open-metadata.org/ - https://marquezproject.github.io/marquez/ - https://datahubproject.io/ - https://www.amundsen.io/ - https://ckan.org/ - https://magda.io/

CKAN (https://ckan.org/) is what data.gov and most state governments use.

Our first instinct is to use [CKAN](https://ckan.org) for cataloging (and storage, with modifications), especially since we know it and know that it has been used successfully elsewhere. However, we suspect that more specialized/better tools exist for this, thus why I kindly ask for your insights.

Ckan

We publish all our data on the [Data Portal](https://data.nhm.ac.uk), a Museum project that's been running since 2014. Instead of MediaWiki it runs on an open-source Python framework called [CKAN](https://ckan.org), which is designed for hosting datasets - though we've had to adapt it in various ways so that it can handle such large amounts of data.

A bit off topic, but are there any self hosted open source archiving servers people are using for personal usage?

I think ArchiveBox[1] is the most popular. I will give it a shot, but it's a shame they don't support URL rewriting[2], which would be pretty important to me. I read a lot of blog and news articles that are split across multiple pages, and it's quite annoying to have to individually search through the archive for each page one by one instead of the "next page" button going to the next archived page.

1: https://archivebox.io/

2: https://github.com/ArchiveBox/ArchiveBox/discussions/1395

I run an ArchiveBox instance locally. Recommended! https://archivebox.io/

Friendly reminder that archive box exists to let you self host your own archive service.

https://github.com/ArchiveBox/ArchiveBox

I dream of a day where archivebox becomes a fleet of homelabs all over the world making it drastically harder to block them all.

https://archivebox.io/ could be a solution for that

I've used https://historio.us since 2011 and still pay for it to keep access to all the pages I've archived over the years. The price has been kept low enough that I can't bring myself to cancel it even though I've been using self-hosted https://archivebox.io/ for the last few years.

I always include an archived link whenever I reference something in documentation. That's my main use at the moment.

However, I also feel like I've gotten a lot of really good value when trying to learn a new development topic. Whenever I find something that looks like it might be useful, I archive it and, because everything is searchable, I end up with a searchable index of really high quality content once I actually know what I'm doing.

I find it hard to rediscover content via web search these days and there's so much churn that having a personal archive of useful content is going to increase in value, at least in my opinion.

The links the reporter are trying to use already don't work on mobile. If you want to link to the README file, link to the README file, e.g. https://github.com/ArchiveBox/ArchiveBox/blob/dev/README.md

I'll concede that this latter link is much longer than it perhaps should be, but I don't think the links the reporter used previous should have ever been used as they don't work for a lot of people.

2. Drop the link into my instance of ArchiveBox [0] and will return to it a few weeks/months later or, more often than not, never again

[0] https://archivebox.io/