Python selenium-python

Open-source Python projects categorized as selenium-python

Top 23 Python selenium-python Projects

selenium-python
  • SeleniumBase

    📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.

    Project mention: The new pdbp (Pdb+) Python debugger! | dev.to | 2023-08-02

    And for Python browser automation, see the SeleniumBase GitHub page!

  • Scout Monitoring

    Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

    Scout Monitoring logo
  • Edu-Mail-Generator

    Generate Free Edu Mail(s) within minutes

  • browserpilot

    Natural language browser automation

    Project mention: LaVague: Open-source Large Action Model to automate Selenium browsing | news.ycombinator.com | 2024-03-13
  • pyleniumio

    Bring the best of Selenium and Cypress into a single Python package

  • Selenium-Profiles

    undetected Selenium using chromedriver and emulation / device profiles

  • captcha_bypass

    Discontinued Captcha bypassing methods for Selenium

  • impf-botpy

    Impf Bot.py 🐍⚡ – Automatisierung für den Corona ImpfterminService Bot

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • BrowserBruter

    BrowserBruter is a powerful web form fuzzing automation tool designed for web security professionals and penetration testers. This Python-based tool leverages Selenium and Selenium-Wire to automate web form fuzzing, making it easier to identify potential vulnerabilities in web applications.

    Project mention: The Browser Bruter – First Ever Browser based web application fuzzing tool | news.ycombinator.com | 2024-04-08
  • Python-Selenium-Action

    Run Selenium with Python via Github Actions using Headless or Non-Headless browsers!

  • WhatSoup

    A web scraper that exports your entire WhatsApp chat history.

  • website-to-gif

    GitHub Action to turn your website into a GIF :camera:

    Project mention: website-to-gif: A GH Action to turn a webpage into a GIF | /r/opensource | 2023-10-23
  • google_maps_scraper

    Script to scrape data from Google Maps places.

  • linkedin-comments-scraper

    Script to scrape comments (including name, profile link, pfp, designation, email(if present), and comment) from a LinkedIn post from the URL of the post.

  • GMaps-Crawler

    Google Maps crawler using Selenium. All extracted data is forwarded to a SQS queue.

  • Instagram-Like-Comment-Bot

    📷 An Instagram bot written in Python using Selenium on Google Chrome. It will go through posts in hashtag(s) and like and comment on them.

  • selenium-python-pytest-bdd

    Example code for a simple Selenium Python Pytest-BDD project using the Page Object Model. Dependency management is handled by pip. Supports Chrome & Firefox

  • fanfictionnet_ff_proxy

    fanfictionnet_ff_proxy: an experimental "proxy" for fanfiction.net piloted by FanFicFare

  • deepl-srt

    Use Selenium to automate translation of a Chinese srt to English on Deepl website

  • selenium_driver_updater

    Download or update your Selenium driver binaries and their browsers automatically with this package

  • kicktipp-bot

    A bot which can submit tips for a Kicktipp competition based on quotes.

  • s-tool

    Selenium wrapper to make your life easy.

  • AutoParaSelenium

    A library to make concurrent selenium tests that automatically download and setup webdrivers

  • web-scraping-with-python

    Demonstration of Web Scraping using Selenium Python (Pytest & Pyunit) and Beautiful Soup

    Project mention: Pyppeteer Tutorial: The Ultimate Guide to Using Puppeteer with Python | dev.to | 2024-02-05

    import asyncio import pytest from pyppeteer.errors import PageError from urllib.parse import quote import json import os import sys from os import environ from pyppeteer import connect, launch exec_platform = os.getenv('EXEC_PLATFORM') # Get username and access key of the LambdaTest Platform username = environ.get('LT_USERNAME', None) access_key = environ.get('LT_ACCESS_KEY', None) test1_url = 'https://ecommerce-playground.lambdatest.io/' test2_url = 'https://scrapingclub.com/exercise/list_infinite_scroll/' # Usecase - 1 # loc_ecomm_1 = ".order-1.col-lg-6 div:nth-of-type(1) > div:nth-of-type(1) > div:nth-of-type(1) > div:nth-of-type(1) > div:nth-of-type(1) div:nth-of-type(1) > img:nth-of-type(1)" loc_ecomm_1 = "[aria-label='1 / 2'] div:nth-of-type(1) > [alt='Nikon D300']" target_url_1 = "https://ecommerce-playground.lambdatest.io/index.php?route=product/product&product_id=63" # Usecase - 2 (Click on e-commerce sliding banner) loc_ecomm_2 = "[alt='Canon DSLR camera']" target_url_2 = "https://ecommerce-playground.lambdatest.io/index.php?route=product/product&product_id=30" # Usecase - 3 Automating interactions on https://scrapingclub.com/exercise/list_infinite_scroll/ loc_infinite_src_prod1 = ".grid .p-4 [href='/exercise/list_basic_detail/93926-C/']" target_url_3 = "https://scrapingclub.com/exercise/list_basic_detail/93926-C/" # Usecase - 4 Automating interactions on https://scrapingclub.com/exercise/list_infinite_scroll/ # when the images are lazy loaded loc_infinite_src_prod2 = "div:nth-of-type(31) > .p-4 [href='/exercise/list_basic_detail/94967-A/']" target_url_4 = "https://scrapingclub.com/exercise/list_basic_detail/94967-A/" # Set timeout in ms timeOut = 60000 async def scroll_to_element(page, selector): # Scroll until the element is detected await page.evaluateHandle( '''async (selector) => { const element = document.querySelector(selector); if (element) { element.scrollIntoView(); } }''', selector ) return selector async def scroll_carousel(page, scr_count): for scr in range(1, scr_count): elem_next_button = "#mz-carousel-213240 > ul li:nth-child(" + str(scr) + ")" await asyncio.sleep(1) elem_next_button = await page.querySelector(elem_next_button) await elem_next_button.click() # Replica of https://github.com/hjsblogger/web-scraping-with-python/blob/ # main/tests/beautiful-soup/test_infinite_scraping.py#L67C5-L80C18 async def scroll_end_of_page(page): start_height = await page.evaluate('document.documentElement.scrollHeight') while True: # Scroll to the bottom of the page await page.evaluate(f'window.scrollTo(0, {start_height})') # Wait for the content to load await asyncio.sleep(1) # Get the new scroll height scroll_height = await page.evaluate('document.documentElement.scrollHeight') if scroll_height == start_height: # If heights are the same, we reached the end of the page break # Add an additional wait await asyncio.sleep(2) start_height = scroll_height # Additional wait after scrolling await asyncio.sleep(2) @pytest.mark.asyncio @pytest.mark.order(1) async def test_lazy_load_ecomm_1(page): # The time out can be set using the setDefaultNavigationTimeout # It is primarily used for overriding the default page timeout of 30 seconds page.setDefaultNavigationTimeout(timeOut) await page.goto(test1_url, {'waitUntil': 'load', 'timeout': timeOut}) # Set the viewport - Apple MacBook Air 13-inch # Reference - https://codekbyte.com/devices-viewport-sizes/ # await page.setViewport({'width': 1440, 'height': 770}) await asyncio.sleep(2) if exec_platform == 'local': # Scroll until the element is detected elem_button = await scroll_to_element(page, loc_ecomm_1) # await page.click(elem_button) # Wait until the page is loaded # https://miyakogi.github.io/pyppeteer/reference.html#pyppeteer.page.Page.waitForNavigation navigationPromise = asyncio.ensure_future(page.waitForNavigation()) await page.click(elem_button) await navigationPromise elif exec_platform == 'cloud': elem_button = await page.waitForSelector(loc_ecomm_1, {'visible': True}) await asyncio.gather( elem_button.click(), page.waitForNavigation({'waitUntil': 'networkidle2', 'timeout': 30000}), ) # Assert if required, since the test is a simple one; we leave as is :D current_url = page.url print('Current URL is: ' + current_url) try: assert current_url == target_url_1 print("Test Success: Product checkout successful") except PageError as e: print("Test Failure: Could not checkout Product") print("Error Code" + str(e)) @pytest.mark.asyncio @pytest.mark.order(2) async def test_lazy_load_ecomm_2(page): carousel_len = 4 # The time out can be set using the setDefaultNavigationTimeout # It is primarily used for overriding the default page timeout of 30 seconds page.setDefaultNavigationTimeout(timeOut) await page.goto(test1_url, {'waitUntil': 'load', 'timeout': timeOut}) # Set the viewport - Apple MacBook Air 13-inch # Reference - https://codekbyte.com/devices-viewport-sizes/ # await page.setViewport({'width': 1440, 'height': 770}) await asyncio.sleep(2) # Approach 1: Directly click on the third button on the carousel # elem_carousel_banner = await page.querySelector("#mz-carousel-213240 > ul li:nth-child(3)") # await asyncio.sleep(1) # await elem_carousel_banner.click() # Approach 2 (Only for demo): Serially click on every button on carousel await scroll_carousel(page, carousel_len) await asyncio.sleep(1) # elem_prod_1 = await page.querySelector(loc_ecomm_2) elem_prod_1 = await page.waitForSelector(loc_ecomm_2, {'visible': True}) await asyncio.gather( elem_prod_1.click(), page.waitForNavigation({'waitUntil': 'networkidle2', 'timeout': 60000}), ) # Assert if required, since the test is a simple one; we leave as is :D current_url = page.url print('Current URL is: ' + current_url) try: assert current_url == target_url_2 print("Test Success: Product checkout successful") except PageError as e: print("Test Failure: Could not checkout Product") print("Error Code" + str(e)) @pytest.mark.asyncio @pytest.mark.order(3) async def test_lazy_load_infinite_scroll_1(page): # The time out can be set using the setDefaultNavigationTimeout # It is primarily used for overriding the default page timeout of 30 seconds page.setDefaultNavigationTimeout(timeOut) await page.goto(test2_url, {'waitUntil': 'load', 'timeout': timeOut}) # Set the viewport - Apple MacBook Air 13-inch # Reference - https://codekbyte.com/devices-viewport-sizes/ # await page.setViewport({'width': 1440, 'height': 770}) await asyncio.sleep(1) elem_prod1 = await page.querySelector(loc_infinite_src_prod1) await asyncio.gather( elem_prod1.click(), page.waitForNavigation({'waitUntil': 'networkidle2', 'timeout': 60000}), ) # await asyncio.sleep(1) # await elem_carousel_banner.click() # elem_button = scroll_to_element(page, loc_infinite_src_prod1) # print(elem_button) # await asyncio.sleep(2) # await elem_button.click() # Assert if required, since the test is a simple one; we leave as is :D current_url = page.url print('Current URL is: ' + current_url) try: assert current_url == target_url_3 print("Test Success: Product checkout successful") except PageError as e: print("Test Failure: Could not checkout Product") print("Error Code" + str(e)) @pytest.mark.asyncio @pytest.mark.order(4) async def test_lazy_load_infinite_scroll_2(page): # The time out can be set using the setDefaultNavigationTimeout # It is primarily used for overriding the default page timeout of 30 seconds page.setDefaultNavigationTimeout(timeOut) # Tested navigation using LambdaTest YouTube channel # await page.goto("https://www.youtube.com/@LambdaTest/videos", await page.goto(test2_url, {'waitUntil': 'load', 'timeout': timeOut}) # Set the viewport - Apple MacBook Air 13-inch # Reference - https://codekbyte.com/devices-viewport-sizes/ # await page.setViewport({'width': 1440, 'height': 770}) await asyncio.sleep(1) await scroll_end_of_page(page) await page.evaluate('window.scrollTo(0, 0)') await asyncio.sleep(1) # elem_prod = await page.querySelector(loc_infinite_src_prod2) # asyncio.sleep(1) # await asyncio.gather( # elem_prod.click(), # page.waitForNavigation({'waitUntil': 'load', 'timeout': 60000}), # ) elem_button = await scroll_to_element(page, loc_infinite_src_prod2) await asyncio.sleep(1) # await page.click(elem_button) await asyncio.gather( page.click(elem_button), page.waitForNavigation({'waitUntil': 'networkidle2', 'timeout': 60000}), ) # Assert if required, since the test is a simple one; we leave as is :D current_url = page.url print('Current URL is: ' + current_url) try: assert current_url == target_url_4 print("Test Success: Product checkout successful") except PageError as e: print("Test Failure: Could not checkout Product") print("Error Code" + str(e))

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Python selenium-python discussion

Log in or Post with

Python selenium-python related posts

  • Selenium-Python but Lighter: Helium

    1 project | news.ycombinator.com | 27 Sep 2023
  • How to update your offline downloaded fanfiction collection?

    3 projects | /r/HPfanfiction | 21 May 2023
  • GMaps-Crawler VS google-maps-scraper - a user suggested alternative

    2 projects | 14 May 2023
  • What are your favorite fics found exclusively on FF.net?

    2 projects | /r/HPfanfiction | 30 Mar 2023
  • It’s just dawned on me how insanely effective the gamification strategy of Duolingo is. I thought it was very kid-like at first until I realised how much I’ve learned using their approach.

    1 project | /r/duolingo | 13 Feb 2023
  • Linkedin Comments Scraper - Script to scrape comments (including name, profile picture, designation, email(if present), and comment) from a LinkedIn post from the URL of the post.

    1 project | /r/webscraping | 5 Jan 2023
  • Citronella: Webdriver Extension with Page Object Wrapper - Python

    1 project | /r/coolgithubprojects | 22 Dec 2022
  • A note from our sponsor - SaaSHub
    www.saashub.com | 18 Jul 2024
    SaaSHub helps you find the best software and product alternatives Learn more →

Index

What are some of the best open-source selenium-python projects in Python? This list will help you:

Project Stars
1 SeleniumBase 4,499
2 Edu-Mail-Generator 795
3 browserpilot 419
4 pyleniumio 263
5 Selenium-Profiles 255
6 captcha_bypass 247
7 impf-botpy 243
8 BrowserBruter 190
9 Python-Selenium-Action 158
10 WhatSoup 116
11 website-to-gif 106
12 google_maps_scraper 79
13 linkedin-comments-scraper 74
14 GMaps-Crawler 72
15 Instagram-Like-Comment-Bot 64
16 selenium-python-pytest-bdd 23
17 fanfictionnet_ff_proxy 21
18 deepl-srt 13
19 selenium_driver_updater 11
20 kicktipp-bot 12
21 s-tool 10
22 AutoParaSelenium 9
23 web-scraping-with-python 5

Sponsored
Free Django app performance insights with Scout Monitoring
Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
www.scoutapm.com