scrapingfish-blog-projects VS uule_grabber

This is definitely possible. I’m not sure if we’re going to have time for this as we’re occupied by work on Scraping Fish but we shared the code for scraping nutrition facts data from Walmart on github: https://github.com/pawelkobojek/scrapingfish-blog-projects/t.... Feel free to take it and build such app/website on top of it.

A script with command-line interface is shared in this GitHub repository: https://github.com/pawelkobojek/scrapingfish-blog-projects/tree/main/google-serp. To run it, you will need Scraping Fish API key which you can get here: https://scrapingfish.com/buy

There's a fairly simple algorithm behind generating uule parameter. It involves a fixed prefix, a special key which depends on location string length, and base64 encoding. If you're interested, here is a blog post from MOZ (a SEO tool) which describes it in detail: https://moz.com/blog/geolocation-the-ultimate-tip-to-emulate-local-search. If you need to automate uule parameter generation, there's a python package for this: https://github.com/ogun/uule_grabber. It generates uule value for you given the location string. Example usage:

from urllib.parse import quote_plus import requests API_KEY = "YOUR SCRAPING FISH API KEY" # https://scrapingfish.com/buy url_prefix = f"https://scraping.narf.ai/api/v1/?api_key={API_KEY}&render_js=true&url=" # to get uule for location you can use: https://github.com/ogun/uule_grabber # or https://site-analyzer.pro/services-seo/uule/ uule_usa = "w+CAIQICIDVVNB" keyword = "kitchen sink" search_url = f"https://www.google.com/search?q={quote_plus(keyword)}&uule={uule_usa}&gl=us&hl=en" response = requests.get(f"{url_prefix}{quote_plus(search_url)}", timeout=90) # add your response processing/parsing logic with open("google.html", "wb") as f: f.write(response.content)