Top 23 Python selenium-python Projects

• SeleniumBase

  9 4,286 9.7 Python

  📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.

  

Project mention: The new pdbp (Pdb+) Python debugger! | dev.to | 2023-08-02

And for Python browser automation, see the SeleniumBase GitHub page!

• Edu-Mail-Generator

  1 788 0.0 Python

  Generate Free Edu Mail(s) within minutes

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• browserpilot

  3 363 7.5 Python

  Natural language browser automation

Project mention: LaVague: Open-source Large Action Model to automate Selenium browsing | news.ycombinator.com | 2024-03-13

• pyleniumio

  2 259 4.8 Python

  Bring the best of Selenium and Cypress into a single Python package

• impf-botpy

  9 246 2.6 Python

  Impf Bot.py 🐍⚡ – Automatisierung für den Corona ImpfterminService Bot

• captcha_bypass

  2 232 2.6 Python

  Captcha bypassing methods for Selenium

• Selenium-Profiles

  2 232 7.2 Python

  undetected Selenium using chromedriver and emulation / device profiles

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• Python-Selenium-Action

  17 150 0.0 Python

  Run Selenium with Python via Github Actions using Headless or Non-Headless browsers!

  

Project mention: Saturday Daily Thread: Resource Request and Sharing! Daily Thread | /r/Python | 2023-06-24

Shared this multiple times - but always seems to be helpful to someone. A GitHub action / template to run your Selenium based scripts on GitHub with ease. https://github.com/MarketingPipeline/Python-Selenium-Action

• BrowserBruter

  1 128 8.1 Python

  BrowserBruter is a powerful web form fuzzing automation tool designed for web security professionals and penetration testers. This Python-based tool leverages Selenium and Selenium-Wire to automate web form fuzzing, making it easier to identify potential vulnerabilities in web applications.

  

Project mention: The Browser Bruter – First Ever Browser based web application fuzzing tool | news.ycombinator.com | 2024-04-08

• WhatSoup

  4 108 2.6 Python

  A web scraper that exports your entire WhatsApp chat history.

• website-to-gif

  3 104 7.7 Python

  GitHub Action to turn your website into a GIF :camera:

Project mention: website-to-gif: A GH Action to turn a webpage into a GIF | /r/opensource | 2023-10-23

• google_maps_scraper

  1 77 0.0 Python

  Script to scrape data from Google Maps places.

Project mention: How to scrape google maps using Python, Selenium and Bose Framework | dev.to | 2023-05-21

git clone https://github.com/patxijuaristi/google_maps_scraper google-maps-scraper cd google-maps-scraper

• GMaps-Crawler

  5 70 5.3 Python

  Google Maps crawler using Selenium. All extracted data is forwarded to a SQS queue.

Project mention: GMaps-Crawler VS google-maps-scraper - a user suggested alternative | libhunt.com/r/GMaps-Crawler | 2023-05-14

• linkedin-comments-scraper

  2 70 1.0 Python

  Script to scrape comments (including name, profile link, pfp, designation, email(if present), and comment) from a LinkedIn post from the URL of the post.

• Instagram-Like-Comment-Bot

  1 62 0.0 Python

  📷 An Instagram bot written in Python using Selenium on Google Chrome. It will go through posts in hashtag(s) and like and comment on them.

• selenium-python-pytest-bdd

  1 23 4.8 Python

  Example code for a simple Selenium Python Pytest-BDD project using the Page Object Model. Dependency management is handled by pip. Supports Chrome & Firefox

• fanfictionnet_ff_proxy

  5 21 4.2 Python

  fanfictionnet_ff_proxy: an experimental "proxy" for fanfiction.net piloted by FanFicFare

Project mention: How to update your offline downloaded fanfiction collection? | /r/HPfanfiction | 2023-05-21

And there is another workaround … https://github.com/nsapa/fanfictionnet_ff_proxy … works for me, but I admit it is seriously hackers—only solution.

• deepl-srt

  1 13 2.6 Python

  Use Selenium to automate translation of a Chinese srt to English on Deepl website

• selenium_driver_updater

  1 10 4.4 Python

  Download or update your Selenium driver binaries and their browsers automatically with this package

• s-tool

  1 10 5.0 Python

  Selenium wrapper to make your life easy.

  

• AutoParaSelenium

  1 9 4.7 Python

  A library to make concurrent selenium tests that automatically download and setup webdrivers

• web-scraping-with-python

  2 5 7.6 Python

  Demonstration of Web Scraping using Selenium Python (Pytest & Pyunit) and Beautiful Soup

Project mention: Pyppeteer Tutorial: The Ultimate Guide to Using Puppeteer with Python | dev.to | 2024-02-05

import asyncio import pytest from pyppeteer.errors import PageError from urllib.parse import quote import json import os import sys from os import environ from pyppeteer import connect, launch exec_platform = os.getenv('EXEC_PLATFORM') # Get username and access key of the LambdaTest Platform username = environ.get('LT_USERNAME', None) access_key = environ.get('LT_ACCESS_KEY', None) test1_url = 'https://ecommerce-playground.lambdatest.io/' test2_url = 'https://scrapingclub.com/exercise/list_infinite_scroll/' # Usecase - 1 # loc_ecomm_1 = ".order-1.col-lg-6 div:nth-of-type(1) > div:nth-of-type(1) > div:nth-of-type(1) > div:nth-of-type(1) > div:nth-of-type(1) div:nth-of-type(1) > img:nth-of-type(1)" loc_ecomm_1 = "[aria-label='1 / 2'] div:nth-of-type(1) > [alt='Nikon D300']" target_url_1 = "https://ecommerce-playground.lambdatest.io/index.php?route=product/product&product_id=63" # Usecase - 2 (Click on e-commerce sliding banner) loc_ecomm_2 = "[alt='Canon DSLR camera']" target_url_2 = "https://ecommerce-playground.lambdatest.io/index.php?route=product/product&product_id=30" # Usecase - 3 Automating interactions on https://scrapingclub.com/exercise/list_infinite_scroll/ loc_infinite_src_prod1 = ".grid .p-4 [href='/exercise/list_basic_detail/93926-C/']" target_url_3 = "https://scrapingclub.com/exercise/list_basic_detail/93926-C/" # Usecase - 4 Automating interactions on https://scrapingclub.com/exercise/list_infinite_scroll/ # when the images are lazy loaded loc_infinite_src_prod2 = "div:nth-of-type(31) > .p-4 [href='/exercise/list_basic_detail/94967-A/']" target_url_4 = "https://scrapingclub.com/exercise/list_basic_detail/94967-A/" # Set timeout in ms timeOut = 60000 async def scroll_to_element(page, selector): # Scroll until the element is detected await page.evaluateHandle( '''async (selector) => { const element = document.querySelector(selector); if (element) { element.scrollIntoView(); } }''', selector ) return selector async def scroll_carousel(page, scr_count): for scr in range(1, scr_count): elem_next_button = "#mz-carousel-213240 > ul li:nth-child(" + str(scr) + ")" await asyncio.sleep(1) elem_next_button = await page.querySelector(elem_next_button) await elem_next_button.click() # Replica of https://github.com/hjsblogger/web-scraping-with-python/blob/ # main/tests/beautiful-soup/test_infinite_scraping.py#L67C5-L80C18 async def scroll_end_of_page(page): start_height = await page.evaluate('document.documentElement.scrollHeight') while True: # Scroll to the bottom of the page await page.evaluate(f'window.scrollTo(0, {start_height})') # Wait for the content to load await asyncio.sleep(1) # Get the new scroll height scroll_height = await page.evaluate('document.documentElement.scrollHeight') if scroll_height == start_height: # If heights are the same, we reached the end of the page break # Add an additional wait await asyncio.sleep(2) start_height = scroll_height # Additional wait after scrolling await asyncio.sleep(2) @pytest.mark.asyncio @pytest.mark.order(1) async def test_lazy_load_ecomm_1(page): # The time out can be set using the setDefaultNavigationTimeout # It is primarily used for overriding the default page timeout of 30 seconds page.setDefaultNavigationTimeout(timeOut) await page.goto(test1_url, {'waitUntil': 'load', 'timeout': timeOut}) # Set the viewport - Apple MacBook Air 13-inch # Reference - https://codekbyte.com/devices-viewport-sizes/ # await page.setViewport({'width': 1440, 'height': 770}) await asyncio.sleep(2) if exec_platform == 'local': # Scroll until the element is detected elem_button = await scroll_to_element(page, loc_ecomm_1) # await page.click(elem_button) # Wait until the page is loaded # https://miyakogi.github.io/pyppeteer/reference.html#pyppeteer.page.Page.waitForNavigation navigationPromise = asyncio.ensure_future(page.waitForNavigation()) await page.click(elem_button) await navigationPromise elif exec_platform == 'cloud': elem_button = await page.waitForSelector(loc_ecomm_1, {'visible': True}) await asyncio.gather( elem_button.click(), page.waitForNavigation({'waitUntil': 'networkidle2', 'timeout': 30000}), ) # Assert if required, since the test is a simple one; we leave as is :D current_url = page.url print('Current URL is: ' + current_url) try: assert current_url == target_url_1 print("Test Success: Product checkout successful") except PageError as e: print("Test Failure: Could not checkout Product") print("Error Code" + str(e)) @pytest.mark.asyncio @pytest.mark.order(2) async def test_lazy_load_ecomm_2(page): carousel_len = 4 # The time out can be set using the setDefaultNavigationTimeout # It is primarily used for overriding the default page timeout of 30 seconds page.setDefaultNavigationTimeout(timeOut) await page.goto(test1_url, {'waitUntil': 'load', 'timeout': timeOut}) # Set the viewport - Apple MacBook Air 13-inch # Reference - https://codekbyte.com/devices-viewport-sizes/ # await page.setViewport({'width': 1440, 'height': 770}) await asyncio.sleep(2) # Approach 1: Directly click on the third button on the carousel # elem_carousel_banner = await page.querySelector("#mz-carousel-213240 > ul li:nth-child(3)") # await asyncio.sleep(1) # await elem_carousel_banner.click() # Approach 2 (Only for demo): Serially click on every button on carousel await scroll_carousel(page, carousel_len) await asyncio.sleep(1) # elem_prod_1 = await page.querySelector(loc_ecomm_2) elem_prod_1 = await page.waitForSelector(loc_ecomm_2, {'visible': True}) await asyncio.gather( elem_prod_1.click(), page.waitForNavigation({'waitUntil': 'networkidle2', 'timeout': 60000}), ) # Assert if required, since the test is a simple one; we leave as is :D current_url = page.url print('Current URL is: ' + current_url) try: assert current_url == target_url_2 print("Test Success: Product checkout successful") except PageError as e: print("Test Failure: Could not checkout Product") print("Error Code" + str(e)) @pytest.mark.asyncio @pytest.mark.order(3) async def test_lazy_load_infinite_scroll_1(page): # The time out can be set using the setDefaultNavigationTimeout # It is primarily used for overriding the default page timeout of 30 seconds page.setDefaultNavigationTimeout(timeOut) await page.goto(test2_url, {'waitUntil': 'load', 'timeout': timeOut}) # Set the viewport - Apple MacBook Air 13-inch # Reference - https://codekbyte.com/devices-viewport-sizes/ # await page.setViewport({'width': 1440, 'height': 770}) await asyncio.sleep(1) elem_prod1 = await page.querySelector(loc_infinite_src_prod1) await asyncio.gather( elem_prod1.click(), page.waitForNavigation({'waitUntil': 'networkidle2', 'timeout': 60000}), ) # await asyncio.sleep(1) # await elem_carousel_banner.click() # elem_button = scroll_to_element(page, loc_infinite_src_prod1) # print(elem_button) # await asyncio.sleep(2) # await elem_button.click() # Assert if required, since the test is a simple one; we leave as is :D current_url = page.url print('Current URL is: ' + current_url) try: assert current_url == target_url_3 print("Test Success: Product checkout successful") except PageError as e: print("Test Failure: Could not checkout Product") print("Error Code" + str(e)) @pytest.mark.asyncio @pytest.mark.order(4) async def test_lazy_load_infinite_scroll_2(page): # The time out can be set using the setDefaultNavigationTimeout # It is primarily used for overriding the default page timeout of 30 seconds page.setDefaultNavigationTimeout(timeOut) # Tested navigation using LambdaTest YouTube channel # await page.goto("https://www.youtube.com/@LambdaTest/videos", await page.goto(test2_url, {'waitUntil': 'load', 'timeout': timeOut}) # Set the viewport - Apple MacBook Air 13-inch # Reference - https://codekbyte.com/devices-viewport-sizes/ # await page.setViewport({'width': 1440, 'height': 770}) await asyncio.sleep(1) await scroll_end_of_page(page) await page.evaluate('window.scrollTo(0, 0)') await asyncio.sleep(1) # elem_prod = await page.querySelector(loc_infinite_src_prod2) # asyncio.sleep(1) # await asyncio.gather( # elem_prod.click(), # page.waitForNavigation({'waitUntil': 'load', 'timeout': 60000}), # ) elem_button = await scroll_to_element(page, loc_infinite_src_prod2) await asyncio.sleep(1) # await page.click(elem_button) await asyncio.gather( page.click(elem_button), page.waitForNavigation({'waitUntil': 'networkidle2', 'timeout': 60000}), ) # Assert if required, since the test is a simple one; we leave as is :D current_url = page.url print('Current URL is: ' + current_url) try: assert current_url == target_url_4 print("Test Success: Product checkout successful") except PageError as e: print("Test Failure: Could not checkout Product") print("Error Code" + str(e))

• 2-captcha-solver-python

  1 5 3.6 Python

  A python program that automates Recaptcha solving using the 2Captcha service.

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Python selenium-python related posts

• Selenium-Python but Lighter: Helium

  1 project | news.ycombinator.com | 27 Sep 2023

• How to update your offline downloaded fanfiction collection?

  3 projects | /r/HPfanfiction | 21 May 2023

• GMaps-Crawler VS google-maps-scraper - a user suggested alternative

  2 projects | 14 May 2023

• What are your favorite fics found exclusively on FF.net?

  2 projects | /r/HPfanfiction | 30 Mar 2023

• It’s just dawned on me how insanely effective the gamification strategy of Duolingo is. I thought it was very kid-like at first until I realised how much I’ve learned using their approach.

  1 project | /r/duolingo | 13 Feb 2023

• Linkedin Comments Scraper - Script to scrape comments (including name, profile picture, designation, email(if present), and comment) from a LinkedIn post from the URL of the post.

  1 project | /r/webscraping | 5 Jan 2023

• Citronella: Webdriver Extension with Page Object Wrapper - Python

  1 project | /r/coolgithubprojects | 22 Dec 2022
• A note from our sponsor - InfluxDB
  www.influxdata.com | 10 May 2024

  Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Python projects with our weekly report!