Crawler
Crawly
Our great sponsors
| Crawler | Crawly | |
|---|---|---|
| 3 | 2 | |
| 918 | 840 | |
| - | 3.2% | |
| 7.9 | 6.6 | |
| 7 months ago | 5 days ago | |
| Elixir | Elixir | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Crawler
-
Ask HN: Freelancer? Seeking freelancer? (April 2022)
SEEKING WORK | Melbourne, Australia | Remote Preferred
- Technologies: Elixir, Ruby, React, JS
- Résumé/CV: https://fredwu.me/cv.pdf
- Email: ifredwu at gmail dot com
My name’s Fred Wu, I’m an experienced Elixir and Ruby developer who has worked on multiple commercial projects as well as having released multiple open source Hex packages and Rubygems.
I work at a fintech startup as CTO, leading a small team. During Covid I've been pumping out around 40 extra hours per week on freelancing work. My day job involves mostly the non-coding part of problem solving so it's a nice mix of pace for me to keep my coding skills sharp during the evenings and on weekends.
I’ve been using Elixir for half a decade, ruby for over a decade, lead and built multiple commercial B2B & B2C SaaS projects. I’ve always been very hands on, and have worked with multiple tech stacks in the past, including JS/React, PHP, Golang, and most recently Clojure at the startup I’m currently working at.
- My blog and talks: https://fredwu.me/
- My Github profile: https://github.com/fredwu
- My LinkedIn profile: https://www.linkedin.com/in/wufred/
As you probably noticed I have quite a few projects on Github. Some of the more interesting ones are:
- Crawler, a high performance web crawler built in Elixir: https://github.com/fredwu/crawler
- Simple Bayes, a naive bayes machine learning implementation in Elixir: https://github.com/fredwu/simple_bayes
- OPQ, a simple in-memory queue with worker pooling and rate limiting in Elixir: https://github.com/fredwu/opq
- And a few years ago when I was heavily involved in the ruby/rails community, I had done an experimental project building a "layer 0" ORM on top of ActiveRecord and Sequel: https://github.com/fredwu/datamappify
If you think my skills and experience could add value to the project I’d love to chat more. You could reach me at ifredwu at gmail dot com. Thanks!
-
[For Hire] Expert-level Elixir/Ruby freelancer - with established open source projects and decades of experience
Crawler, a high performance web crawler built in Elixir: https://github.com/fredwu/crawler
-
Complete, Production-Ready Phoenix Reference Applications
Crawler
Crawly
- Crawly – Elixir web scraping framework
-
Squirm - This was the night of the crawling terror!
I would say that the usage of Squirm is for crawling purposes and it is a close analogue of https://github.com/elixir-crawly/crawly
What are some alternatives?
Ace - HTTP web server and client, supports http1 and http2
scrape - Scrape any website, article or RSS/Atom Feed with ease!
explode - An easy utility for responding with standard HTTP/JSON error payloads in Plug- and Phoenix-based applications
gun - HTTP/1.1, HTTP/2, Websocket client (and more) for Erlang/OTP.
web_socket - An exploration into a stand-alone library for Plug applications to easily adopt WebSockets.
finch - Elixir HTTP client, focused on performance
river - An HTTP/2 client for Elixir (a work in progress!)
yuri - Elixir module for easier URI manipulation.
uri_template - RFC 6570 compliant URI template processor for Elixir
mint - Functional HTTP client for Elixir with support for HTTP/1 and HTTP/2 🌱
Ralitobu.Plug - Elixir Plug for Ralitobu, the Rate Limiter with Token Bucket algorithm
sparql_client - A SPARQL client for Elixir