Our great sponsors
- InfluxDB - Access the most powerful time series database as a service
- ONLYOFFICE ONLYOFFICE Docs — document collaboration in your environment
- Sonar - Write Clean Python Code. Always.
-
httptoolkit
HTTP Toolkit is a beautiful & open-source tool for debugging, testing and building with HTTP(S) on Windows, Linux & Mac :tada: Open an issue here to give feedback or ask for help.
Thanks, that's super useful.
> If you try to use go's package manager, example: `go get golang.org/x/oauth2`
I just tested, and `go get golang.org/x/oauth2` seems to work fine for me, I can see all the requests being happily intercepted immediately: https://imgur.com/a/Cb1y9Q2
Can you see the 500 in HTTP Toolkit, and any more info there (in the body or as an error at the top) related to that? Or can you see a "certificate rejected" message? If nothing turns up there at all then yes, something must be overriding the proxy configuration.
Maybe you have some other Go package manager configuration that conflicts with this? I'd be very interested to know about that if so, I'm sure there's others with the same thing. It's always very hard to know if my configuration is representative of normal devs for any given language/tool.
Probably best to debug this outside of a HN thread though :-). You can file a proper issue about this at https://github.com/httptoolkit/httptoolkit/issues/new, I'd love to know what's going on there and get this fixed.
> I ended up using https://github.com/hmgle/graftcp which somehow manages to force tcp traffic through a socks5 proxy.
Really interesting, thanks! I'll look into that.
-
For the time being it's still OK. It asked me once to try the "premium" version or whatever it's called, I said no, and it never bothered me again.
I don't know what licence Postman uses. A quick search on GitHub didn't turn up the source of the app on the Postman Labs page. Insomnia is MIT [0], so it could still be forked if Kong got decided to stop supporting the free version.
-
InfluxDB
Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.
-
I'm the author, that's exactly it! The contents of that interceptors folder should give you an idea how it all works: https://github.com/httptoolkit/httptoolkit-server/tree/maste...
It actually doesn't install system certificates at all though. It doesn't change any system configuration whatsoever, and it doesn't need any admin/root privileges. The deb package doesn't do anything different to any others.
That's because the key differentiator of HTTP Toolkit vs Fiddler/Charles/mitmproxy etc, is that it provides targeted interception, rather than intercepting your entire system at once.
That works by injecting cert & proxy config into a single browser window, intercepting specific Android apps, targeting individual Docker containers etc. That way you get much less noisy intercepted traffic for your debugging, and you can freely add rules to rewrite/break traffic without interfering with anything else.
You can even open two HTTP Toolkit windows on one machine, and intercept things separately into each one.
If you want, you can still do the normal steps to do full system interception manually if you'd prefer that, but by default it uses entirely transient and permissionless targeted interception instead, and that's almost always the better approach.
-
Thanks, that's super useful.
> If you try to use go's package manager, example: `go get golang.org/x/oauth2`
I just tested, and `go get golang.org/x/oauth2` seems to work fine for me, I can see all the requests being happily intercepted immediately: https://imgur.com/a/Cb1y9Q2
Can you see the 500 in HTTP Toolkit, and any more info there (in the body or as an error at the top) related to that? Or can you see a "certificate rejected" message? If nothing turns up there at all then yes, something must be overriding the proxy configuration.
Maybe you have some other Go package manager configuration that conflicts with this? I'd be very interested to know about that if so, I'm sure there's others with the same thing. It's always very hard to know if my configuration is representative of normal devs for any given language/tool.
Probably best to debug this outside of a HN thread though :-). You can file a proper issue about this at https://github.com/httptoolkit/httptoolkit/issues/new, I'd love to know what's going on there and get this fixed.
> I ended up using https://github.com/hmgle/graftcp which somehow manages to force tcp traffic through a socks5 proxy.
Really interesting, thanks! I'll look into that.
-
> why would I prefer this to mitmproxy?
Compared to mitmproxy, HTTP Toolkit:
- Has fully automated setup for most browsers, docker containers, Android, all Node.js/Ruby/Python/PHP/Go applications run from intercepted terminal windows, all JVM processes, any Electron apps etc etc. Some of these automated setup steps are very difficult to do manually (e.g. intercepting Android devices, where you can't normally install your own certificates nowadays, or intercepting Node.js, which completely ignores system proxy settings) so this can make a huge difference in non-trivial case.
- Supports targeted interception (intercept just one app/container/browser window) whilst all mitmproxy's manual setup steps are generally focused on helping you intercept your whole machine at once. Intercepting the whole machine means very noisy interception and means that rewriting traffic interferes with all other usage of your machine. Targeted interception means you can do neat things like run two HTTP Toolkit instances independently at the same time, and means you don't need root privileges or permanent configuration settings.
- Has generally friendlier UI & UX (imo). For example, mitmproxy uses a unique custom syntax (https://docs.mitmproxy.org/stable/concepts-filters/) of special characters to define matching & rewriting rules, or requires you to write a full python script. HTTP Toolkit lets you click 'new rule' -> 'GET requests' -> 'match regex ' -> 'then reply with ', and then immediately start injecting automated fake responses. From HTTP Toolkit you can then build named groups or these rules, and import & export them (as JSON) to build libraries you can share with your colleagues.
- Provides lots more background information automatically: e.g. built-in documentation for all standard HTTP headers, body autoformatting for lots more formats, syntax highlighting, code folding, regex searching etc of request & response bodies, plus 'this is how and why this response could be cached' caching explanations, OpenAPI-powered docs for recognized endpoints on 1400+ APIs, etc.
- Includes advanced features to do things like exporting requests as ready-to-use code for various languages & tools, or automatically testing the performance of different compression algorithms on a given response body.
- Is more easily scriptable for automation & end-to-end testing, because all the HTTP-handling internals are usable as a standalone open-source JS library: https://github.com/httptoolkit/mockttp
That said, mitmproxy has been around longer, it's definitely more mature, and it was a big inspiration in many places. It's a great project! It does have some advantages of its own:
- If you strongly prefer a CLI interface, mitmproxy is very focused on that, and HTTP Toolkit is not. HTTP Toolkit could support that too in theory (the backend & frontend are independent) but it definitely doesn't right now, and it's not high on my todo list (contributions welcome though!)
- Mitmproxy is primarily scriptable in Python. You can build automation around HTTP Toolkit's internals using mockttp, but that's JS, and it's mostly usable standalone right now, rather than integrated into normal workflows within the app. If you want very complex scripted rules, mitmproxy has a few more options right now, and lets you do things in python instead of JS, which some people will prefer.
- WebSocket debugging - this is coming for HTTP Toolkit soon, but it's not available today. WebSockets get passed through fine, but they don't appear in the UI, and you can't set up mock rules for them.
> I'd be interested both in why I'd prefer the open source httptoolkit and pro?
There's a list of Pro features at https://httptoolkit.tech/pricing/. Note that it's all open source, even the Pro code, everything.
The general idea is that everything you need to intercept, inspect and manually fiddle with traffic is totally free. Anything optional that most users don't need, but which is helpful for advanced usage or enterprise use cases, requires Pro.
-
mockoon
Mockoon is the easiest and quickest way to run mock APIs locally. No remote deployment, no account required, open source.
-
I recommend https://paw.cloud for just the Postman clone - it has native UI, not an Electron shell.
For doing proxies, there is https://proxyman.io, which I think is also native (haven't used it a lot, not sure)
-
ONLYOFFICE
ONLYOFFICE Docs — document collaboration in your environment. Powerful document editing and collaboration in your app or environment. Ultimate security, API and 30+ ready connectors, SaaS or on-premises
-
100% agree. After postman asked me to make an account I uninstalled it and have been building my own. It's not all wired together yet but I'm using this fork of https://github.com/dylanowen/dot-http/tree/wasm-library
I wanted something where I could store and share my requests over git instead of creating some random account. I also wanted the api description to be text not a UI+JSON blob.
I also made this
-
I'm really surprised no one else has mentioned this so far, but Hoppscotch (formerly postwoman) is an excellent open alternative: https://github.com/hoppscotch/hoppscotch
-
-
mitmproxy
An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
MITM proxy doesnt include any builtin way to install a system certificate. So if whatever you are trying to monitor doesnt accept user certificate (which is most stuff), then MITM proxy wont even work:
-
-
httpie
🥧 HTTPie for Terminal — modern, user-friendly command-line HTTP client for the API era. JSON support, colors, sessions, downloads, plugins & more.
How about HTTPie: https://httpie.io/
Well known for the CLI, they now have a web/desktop client as well.
-
-
Sonar
Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
