responses VS isort

I use responses for this (for HTTP/HTTPS requests), to great affect. It's API is very nice. We have an API layer at work that's basically just a proxy to micro services, and responses is what we use to test it.

responses

Please format your code appropriately. But if you are using python requests you can use responses for mocking requests: https://github.com/getsentry/responses

You can use this https://github.com/getsentry/responses

The repsonses library is designed for mocking requests during tests https://github.com/getsentry/responses

I like using responses for mocking API calls in unit tests. It's easy to use and has less boilerplate than doing the mocks yourself. The bulk of your unit tests should be around any data transformation you do, and unit testing them should be fairly straightforward. I'd recommend writing integration tests as well, especially as IME the most common reason why data pipelines with API sources break is due to breaking API changes out of your control. I'd make them as high level as possible and make real API calls, preferably using credentials for a designated testing account. Assert the response from the API call conforms to the contract you expect. Additionally to integration test your code, you can test the main entrypoint to your script and make sure it spits out a file with the shape and contents you expect.

HTTP client library flexibility. Yes, requests pairs well with responses or requests-mock, and HTTPX has RESPX or pytest_httpx. Those testing helpers are an excellent match for the corresponding library, and should certainly be recommended. However, I don't always want to face the risk of rewriting all my tests if I replace the client library some day. And sometimes I am using an altogether different tool (even though both requests and HTTPX are quite awesome) such as urlopen, urllib3, httplib2, tornado, or aiohttp.

isort: This library sorts your imports alphabetically, and automatically separates them into sections and by type. It provides a cleaner and more organised way to manage project imports.

isort will sort the imports for you

isort is a Python utility that helps in sorting and organizing import statements in Python code to create readable and consistent code. It automatically formats import statements in accordance with PEP 8.

repos: - repo: https://github.com/ambv/black rev: 23.3.0 hooks: - id: black args: [--config=./pyproject.toml] language_version: python3.11 - repo: https://github.com/pycqa/flake8 rev: 6.0.0 hooks: - id: flake8 args: [--config=./tox.ini] language_version: python3.11 - repo: https://github.com/pycqa/isort rev: 5.12.0 hooks: - id: isort args: ["--profile", "black", "--filter-files"] language_version: python3.11 - repo: https://github.com/pre-commit/pre-commit-hooks rev: v4.4.0 hooks: - id: requirements-txt-fixer language_version: python3.11 - id: debug-statements - id: detect-aws-credentials - id: detect-private-key

repos: ... pre-commmit stuff ... black stuff - repo: https://github.com/pycqa/isort rev: 5.12.0 hooks: - id: isort name: isort (python)

isort

isort , A Python utility / library to sort imports.

Install black and isort and just don't worry about it. :-)

Sure. Also, and don't take this the wrong way, but there are some code smells in your project that could be partially mitigated with some basic linting/formatting. I suggest black as a code formatter, flake8 for basic linting, and isort for sorting imports (for example, you have local imports mixed in with standard library and third party imports). You can install these via pip and most editors (like VS Code) can autoformat on save and show you linting problems as you edit. And you can integrate these into your workflow by using pre-commit.