public-api-lists VS isort

I have a few e2e projects, if that might help. While they can be deployed to AWS via the included terraform file (costs money) you can run them locally with docker. I list the projects from simplest to more complicated 1. I’d recommend starting at https://www.startdataengineering.com/post/data-engineering-project-to-impress-hiring-managers/ this is the simplest. 2. Once you have it running, and get an overview of the components( docker, ec2, Postgres), then I’d recommend looking at this article https://www.startdataengineering.com/post/data-engineering-projects-with-free-template/ to understand how the components work together. 3. Try out the pipeline with a data source if your choosing. I use https://github.com/public-api-lists/public-api-lists to get some data API. 4. Once you get a good understanding of how data is pulled and loaded along with how it’s scheduled, then I’d recommend looking at this airflow project https://www.startdataengineering.com/post/data-engineering-project-for-beginners-batch-edition/ I posted about this a while back https://www.reddit.com/r/dataengineering/comments/ygieh8/data\_engineering\_projects\_with\_template\_airflow/ Hope this helps. LMK if you have any questions.

And then try out the pipeline with a data source if your choosing. I use https://github.com/public-api-lists/public-api-lists to get some data API.

Suggest to take a look through https://github.com/public-api-lists/public-api-lists

Sounds like you're stuck in tutorial purgatory. It's time to drop the training wheels and apply the knowledge you've learned to some personal projects. Pickup pygame, or build a web scraper. Here's a list of free public api's. This might get you started. https://github.com/public-api-lists/public-api-lists

Public-api-lists (fork of original one I linked)

public-api-lists Another great repository to find good APIs for development. It has more than 4K stars on GitHub. These APIs are more inclined towards the development side. It is also organized into suitable categories.

isort: This library sorts your imports alphabetically, and automatically separates them into sections and by type. It provides a cleaner and more organised way to manage project imports.

isort will sort the imports for you

isort is a Python utility that helps in sorting and organizing import statements in Python code to create readable and consistent code. It automatically formats import statements in accordance with PEP 8.

repos: - repo: https://github.com/ambv/black rev: 23.3.0 hooks: - id: black args: [--config=./pyproject.toml] language_version: python3.11 - repo: https://github.com/pycqa/flake8 rev: 6.0.0 hooks: - id: flake8 args: [--config=./tox.ini] language_version: python3.11 - repo: https://github.com/pycqa/isort rev: 5.12.0 hooks: - id: isort args: ["--profile", "black", "--filter-files"] language_version: python3.11 - repo: https://github.com/pre-commit/pre-commit-hooks rev: v4.4.0 hooks: - id: requirements-txt-fixer language_version: python3.11 - id: debug-statements - id: detect-aws-credentials - id: detect-private-key

repos: ... pre-commmit stuff ... black stuff - repo: https://github.com/pycqa/isort rev: 5.12.0 hooks: - id: isort name: isort (python)

isort

isort , A Python utility / library to sort imports.

Install black and isort and just don't worry about it. :-)

Sure. Also, and don't take this the wrong way, but there are some code smells in your project that could be partially mitigated with some basic linting/formatting. I suggest black as a code formatter, flake8 for basic linting, and isort for sorting imports (for example, you have local imports mixed in with standard library and third party imports). You can install these via pip and most editors (like VS Code) can autoformat on save and show you linting problems as you edit. And you can integrate these into your workflow by using pre-commit.