Crawly
Crawler
Crawly | Crawler | |
---|---|---|
2 | 3 | |
840 | 918 | |
1.2% | - | |
6.6 | 7.9 | |
8 days ago | 7 months ago | |
Elixir | Elixir | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Crawly
- Crawly – Elixir web scraping framework
-
Squirm - This was the night of the crawling terror!
I would say that the usage of Squirm is for crawling purposes and it is a close analogue of https://github.com/elixir-crawly/crawly
Crawler
-
Ask HN: Freelancer? Seeking freelancer? (April 2022)
SEEKING WORK | Melbourne, Australia | Remote Preferred
- Technologies: Elixir, Ruby, React, JS
- Résumé/CV: https://fredwu.me/cv.pdf
- Email: ifredwu at gmail dot com
My name’s Fred Wu, I’m an experienced Elixir and Ruby developer who has worked on multiple commercial projects as well as having released multiple open source Hex packages and Rubygems.
I work at a fintech startup as CTO, leading a small team. During Covid I've been pumping out around 40 extra hours per week on freelancing work. My day job involves mostly the non-coding part of problem solving so it's a nice mix of pace for me to keep my coding skills sharp during the evenings and on weekends.
I’ve been using Elixir for half a decade, ruby for over a decade, lead and built multiple commercial B2B & B2C SaaS projects. I’ve always been very hands on, and have worked with multiple tech stacks in the past, including JS/React, PHP, Golang, and most recently Clojure at the startup I’m currently working at.
- My blog and talks: https://fredwu.me/
- My Github profile: https://github.com/fredwu
- My LinkedIn profile: https://www.linkedin.com/in/wufred/
As you probably noticed I have quite a few projects on Github. Some of the more interesting ones are:
- Crawler, a high performance web crawler built in Elixir: https://github.com/fredwu/crawler
- Simple Bayes, a naive bayes machine learning implementation in Elixir: https://github.com/fredwu/simple_bayes
- OPQ, a simple in-memory queue with worker pooling and rate limiting in Elixir: https://github.com/fredwu/opq
- And a few years ago when I was heavily involved in the ruby/rails community, I had done an experimental project building a "layer 0" ORM on top of ActiveRecord and Sequel: https://github.com/fredwu/datamappify
If you think my skills and experience could add value to the project I’d love to chat more. You could reach me at ifredwu at gmail dot com. Thanks!
-
[For Hire] Expert-level Elixir/Ruby freelancer - with established open source projects and decades of experience
Crawler, a high performance web crawler built in Elixir: https://github.com/fredwu/crawler
-
Complete, Production-Ready Phoenix Reference Applications
Crawler
What are some alternatives?
scrape - Scrape any website, article or RSS/Atom Feed with ease!
Ace - HTTP web server and client, supports http1 and http2
gun - HTTP/1.1, HTTP/2, Websocket client (and more) for Erlang/OTP.
explode - An easy utility for responding with standard HTTP/JSON error payloads in Plug- and Phoenix-based applications
finch - Elixir HTTP client, focused on performance
web_socket - An exploration into a stand-alone library for Plug applications to easily adopt WebSockets.
yuri - Elixir module for easier URI manipulation.
river - An HTTP/2 client for Elixir (a work in progress!)
mint - Functional HTTP client for Elixir with support for HTTP/1 and HTTP/2 🌱
uri_template - RFC 6570 compliant URI template processor for Elixir
sparql_client - A SPARQL client for Elixir
Ralitobu.Plug - Elixir Plug for Ralitobu, the Rate Limiter with Token Bucket algorithm