CKAN VS Flask

CKAN The Open Source Data Portal Software

We use Kubernetes as our deployment platform. Any feedback on one of these open source data catalogs ? - https://atlas.apache.org/#/ - https://opendatadiscovery.org/ - https://open-metadata.org/ - https://marquezproject.github.io/marquez/ - https://datahubproject.io/ - https://www.amundsen.io/ - https://ckan.org/ - https://magda.io/

CKAN (https://ckan.org/) is what data.gov and most state governments use.

Our first instinct is to use [CKAN](https://ckan.org) for cataloging (and storage, with modifications), especially since we know it and know that it has been used successfully elsewhere. However, we suspect that more specialized/better tools exist for this, thus why I kindly ask for your insights.

Ckan

We publish all our data on the [Data Portal](https://data.nhm.ac.uk), a Museum project that's been running since 2014. Instead of MediaWiki it runs on an open-source Python framework called [CKAN](https://ckan.org), which is designed for hosting datasets - though we've had to adapt it in various ways so that it can handle such large amounts of data.

I'd suggest Flask or some of the smaller projects in the Pallets ecosystem:

https://github.com/pallets/flask

#!/usr/bin/python # # https://flask.palletsprojects.com/en/3.0.x/installation/ # from flask import Flask, jsonify, request contacts = [ { "id": "1", "firstname": "Lorem", "lastname": "Ipsum", "email": "[email protected]", }, { "id": "2", "firstname": "Mauris", "lastname": "Quis", "email": "[email protected]", }, { "id": "3", "firstname": "Donec Purus", "lastname": "Purus", "email": "[email protected]", } ] app = Flask(__name__, static_url_path='', static_folder='public',) @app.route("/contact//save", methods=["PUT"]) def save_contact(id): data = request.json contacts[id - 1] = data return jsonify(contacts[id - 1]) @app.route("/contact/", methods=["GET"]) @app.route("/contact//edit", methods=["GET"]) def get_contact(id): return jsonify(contacts[id - 1]) @app.route('/') def root(): return app.send_static_file('index.html') if __name__ == '__main__': app.run(debug=True)

I was looking at all the big projects like numpy, pytorch, flask, etc.

Explore here.

I may use Flask-Admin initially to offload the "CRUD" operations to have an initial prototype fast but then drop it ASAP because I don't want to write a "flask-admin application" to fight against later on. If the application is mainly "CRUD", then Flask-Admin is suitable.

Now...

Would you do a breakdown/list of all the jobs you've done by sector/vertical and by function/role and by application functionality?

- [0]: https://flask.palletsprojects.com

- [1]: https://flask-admin.readthedocs.io/en/latest

- [2]: https://flask.palletsprojects.com/en/2.3.x/patterns/celery

- [3]: https://sentry.io

- [4]: https://posthog.com

- [5]: https://www.docker.com

In the lab to follow, we will be setting up an end-to-end DevOps workflow for a Flask microservice with GitHub Actions, using a self-managed custom runner for maximal control over the pipeline execution environment and automating deployments to a local Kubernetes cluster. Furthermore, we will construct separate pipelines for our "development" and "production" environments to further elaborate on the concepts of continuous deployment and delivery.

Flask Micro web Framework: You will use Flask to build a Flask web application.

In an earlier post I mentioned a bunch of Open Source web applications. Let's now focus on the ones written in Python using Flask the light-weight web framework.