crawler
An easy to use, powerful crawler implemented in PHP. Can execute Javascript. (by spatie)
Guzzle
Guzzle, an extensible PHP HTTP client (by guzzle)
| crawler | Guzzle | |
|---|---|---|
| 3 | 15 | |
| 2,472 | 22,992 | |
| 0.7% | 0.1% | |
| 6.8 | 6.7 | |
| about 2 months ago | 19 days ago | |
| PHP | PHP | |
| MIT License | MIT License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
crawler
Posts with mentions or reviews of crawler.
We have used some of these posts to build our list of alternatives
and similar projects.
-
How to Build a Robust Web Scraper with Laravel: and Catch 'Em All
The spatie/crawler package is a powerful tool developed by Spatie, a web development agency known for creating high-quality, open-source packages for the Laravel community. This crawler is designed to simplify the process of building web scrapers and bots in PHP, particularly within the Laravel framework. It provides a flexible and easy-to-use interface to crawl websites and extract needed data efficiently. Click me if you love spatie/crawler, give them a star too, they are the reason that this article exists.
-
Can I use Laravel Dusk to automate downloading an HTML version of a website?
You probably want something like this: https://github.com/spatie/crawler
-
How to setup Wordpress to not crash due to 404 DDoS attack?
Alternatively, write your own basic crawler or adapt an existing package like https://github.com/spatie/crawler.
Guzzle
Posts with mentions or reviews of Guzzle.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-12-06.
-
Open Source Projects You Can Lay Your Hand On
Guzzle is a PHP HTTP client library. It’s a simple and effective solution for sending HTTP requests and managing HTTP/1.1 and HTTP/2.0 responses. This versatile tool excels in several areas, allowing developers to build query strings quickly, send POST requests, upload JSON data, and handle other HTTP-related tasks. Moreover, Guzzle facilitates both synchronous and asynchronous request handling, providing flexibility for different scenarios.
-
What GraphQL client package are you using?
Symfony HTTP Client or Guzzle. If new build then Symfony, but have a lot of existing implementations with Guzzle. Both have worked very well. Worst case fallback to cURL.
-
How to integrate Microsoft Graph API into Symfony?
but if you'd rather make raw requests, guzzle is a good option (though I'd opt for the sdk): https://github.com/guzzle/guzzle
-
PHP, REST API and Mikrotik Routers
PHP has built-in cURL support, but I never use it. I like using Guzzle or Symfony's http-client.
-
Why is the cURL package missing libz?
I just ran into multiple errors regarding the pre-packaged cURL: It does not feature accepting compressed responses (which I suppose is due to absense of `libz` being compiled/linked in).
- CVE-2022-29248 for guzzlehttp/guzzle: Cross-domain cookie leakage
- GitHub - guzzle/guzzle: Guzzle, an extensible PHP HTTP client
-
Queues vs Schedule to monitor websites
I am busy building a small application that monitors websites using guzzle. My idea is to run through the list of websites alphabetically and make a guzzle request to each and update my table with the http response codes of each site.
- How do you test your code ?
-
API Client Design Across Languages - Part 2 - Making Requests
Like Node.js, the PHP ecosystem has quite a number of good HTTP request libraries. Guzzle is perhaps one of the most well known, but there are many other popular libraries out there. Luckily, PHP also has some interface standards around HTTP clients and messages, particularly PSR-7, PSR-17, and PSR-18,
What are some alternatives?
When comparing crawler and Guzzle you can also consider the following projects:
RED_HAWK - All in one tool for Information Gathering, Vulnerability Scanning and Crawling. A must have tool for all penetration testers
Requests - Requests for PHP is a humble HTTP request library. It simplifies how you interact with other sites and takes away all your worries.
scrape - Scrape any website, article or RSS/Atom Feed with ease!
HTTPFul - A Chainable, REST Friendly, PHP HTTP Client. A sane alternative to cURL.
aws-elixir - AWS clients for Elixir
Buzz - PHP's lightweight HTTP client
plug_wait1 - Plug adapter for the wait1 protocol
PHP VCR - Record your test suite's HTTP interactions and replay them during future test runs for fast, deterministic, accurate tests.
diskover-community - Diskover Community Edition - Open source file indexer, file search engine and data management and analytics powered by Elasticsearch
zend-diactoros
gen_retry - Utilities for retrying Elixir functions, with configurable delay and backoff.
HTTPlug - HTTPlug, the HTTP client abstraction for PHP