Scraping data from an app: real world example

• httptoolkit

  36 2,424 4.1

  HTTP Toolkit is a beautiful & open-source tool for debugging, testing and building with HTTP(S) on Windows, Linux & Mac :tada: Open an issue here to give feedback or ask for help.

  

Previously i had some success with this https://httptoolkit.tech/ and running the app on android emulator

• mitmproxy

  152 34,347 9.4 Python

  An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.

  

I recently did something similar with good results (I found the api endpoints I was interested in) using the official Android emulator and https://github.com/mitmproxy/mitmproxy

I did have to jump through some hoops with the emulator and pushing my own ssl cert to it's RO system partition. But it was a few commands and easy enough.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• PCAPdroid

  16 1,888 9.0 Java

  No-root network monitor, firewall and PCAP dumper for Android

There's also PCAPdroid [1] which you can run straight from your phone with no root. Works with https traffic too when you enable the mitm setting.

[1]: https://github.com/emanuele-f/PCAPdroid

Suggest a related project