|7 days ago||about 18 hours ago|
|Apache License 2.0||GNU General Public License v3.0 or later|
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
JORLDY: OpenSource Reinforcement Learning Framework
2 projects | reddit.com/r/reinforcementlearning | 8 Nov 2021
Distributed RL algorithms are provided using ray
Python stands to lose its GIL, and gain a lot of speed
5 projects | reddit.com/r/programming | 20 Oct 2021
I had a similar use case and ended up using ray. https://github.com/ray-project/ray
How to deploy a rllib-trained model?
3 projects | reddit.com/r/reinforcementlearning | 16 Oct 2021
Currently, rllib's "--export-formats" does nothing; I have folders of checkpoints, but no models. Looks like currently the internal export_model function isn't implemented: https://github.com/ray-project/ray/issues/190213 projects | reddit.com/r/reinforcementlearning | 16 Oct 2021
[HELP] Converting many individual workstations into a HPC cluster
1 project | reddit.com/r/HPC | 11 Oct 2021
Unless you have infiniband, you might want to build it as a kubernetes cluster and look at something like (ray-project)[https://github.com/ray-project/ray] it has a ton of distributed plugin packages that are Ethernet based.
Show HN: SpotML – Managed ML Training on Cheap AWS/GCP Spot Instances
6 projects | news.ycombinator.com | 3 Oct 2021
Neat. Congratulations on the launch!
Apart from the fact that it could deploy to both GCP and AWS, what does it do differently than AWS Batch ?
When we had a similar problem, we ran jobs on spots with AWS Batch and it worked nicely enough.
Some suggestions (for a later date):
1. Add built-in support for Ray  (you'd essentially be then competing with Anyscale, which is a VC funded startup, just to contrast it with another comment on this thread) and dbt .
2. Support deploying coin miners (might be good to widen the product's reach; and stand it up against the likes of consensys).
3. Get in front of many cost optimisation consultants out there, like the Duckbill Group.
If I may, where are you building this product from? And how many are on the team?
Writing your First Distributed Python Application with Ray (without multiprocessing)
4 projects | reddit.com/r/Python | 23 Aug 2021
Here is an older discussion on dask vs ray from the creators of both projects: https://github.com/ray-project/ray/issues/642
[D] Kubeflow vs. Argo for ML Pipeline Tool
2 projects | reddit.com/r/MachineLearning | 17 Aug 2021
Here is link number 1 - Previous text "Ray"2 projects | reddit.com/r/MachineLearning | 17 Aug 2021
If you are looking for a developer-friendly tool, I'd ditch the task/workflow orchestration paradigm altogether and use something like Ray. It's made by and for ML practitioners, it's much more versatile, has no unwarranted DSLs (pure python), and you can test locally before deploying with pretty much the same code.
1 project | news.ycombinator.com | 8 Jun 2021
Old guy programmer here, need to brush up on Python quickly!
13 projects | reddit.com/r/Python | 6 Dec 2021
scrapy for reading and processing data on websites
Web scraping data with pagination?
1 project | reddit.com/r/learnpython | 2 Dec 2021
What you want to use instead, is a web crawling framework like scrapy, which provides methods and classes to deal with all the common web scraping requirements. It has functions for pagination, it supports callbacks for using different parsers for different sub sites, it provides link extractors to find and follow urls, asynchronous request handling, logging, automatic request throttling, file exports for your results and many more. In short, it was written to provide all the tools you need to make writing web scrapers as comfortable and easy as possible.
What are the best 5 Web Scrapping API/Tool to scrape data?
1 project | reddit.com/r/u_ScrapperExpert | 1 Dec 2021
- Scrapfly (Probably the best web scraping api on the market) - If you liked ScraperAPI you should test them - you will see the difference - Scrapy (Web Scraping Framework from Zyte) - Browserless (Automation)
5 ways to keep your skills fresh after finishing a coding bootcamp
5 projects | dev.to | 28 Nov 2021
One way to improve your projects and coding skills is to try new models and libraries. For example, if you did classification with logistic regression, try also with random forest; if you used Tensorflow, now try Keras; if you scraped a website with BeautifulSoup, now do it with Scrapy. You get the point.
Good way to create a web scraper for multiple different sites
1 project | reddit.com/r/webdev | 18 Nov 2021
working on the parser of a website - in order to get this running on x pages
1 project | reddit.com/r/learnpython | 18 Nov 2021
Use scrapy to do this.
[OC] Which programming language is required to land a data job at Meta (Facebook)
3 projects | reddit.com/r/dataisbeautiful | 17 Nov 2021
Download Files with Scrapy Crawl Spider - Tutorial and Source Code
2 projects | reddit.com/r/webscraping | 29 Oct 2021
I assume that you have at least working knowledge of Python though. This tutorial also assumes that you have at the very least, have played around with Scrapy.
Wanting to build a web scraper with no prior coding knowledge. Where do I start as fast as possible?
1 project | reddit.com/r/webscraping | 23 Oct 2021
Check out https://scrapy.org/ it’s a Python framework for web scraping and then look at https://youtube.com/c/JohnWatsonRooney channel to learn the syntax. Finally, go to https://www.zyte.com/scrapy-cloud/ to deploy your crawler to the cloud!
Why on earth is the Modpack button on curseforge always swapped so much? Are the devs having an argument or something?
1 project | reddit.com/r/feedthebeast | 30 Sep 2021
What are some alternatives?
requests-html - Pythonic HTML Parsing for Humans™
pyspider - A Powerful Spider(Web Crawler) System in Python.
MechanicalSoup - A Python library for automating interaction with websites.
Faust - Python Stream Processing
stable-baselines3 - PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.
gevent - Coroutine-based concurrency library for Python
Grab - Web Scraping Framework
SCOOP (Scalable COncurrent Operations in Python) - SCOOP (Scalable COncurrent Operations in Python)
portia - Visual scraping for Scrapy
Thespian Actor Library - Python Actor concurrency library
feedparser - Parse feeds in Python