rtila-releases VS parsel

Rtila [1]

Created by an indy/solo developer-on-fire cranking out user-requested features quite quickly... check the releases page [2]

I have used (or at lelast trialled) the vast majority acraping tech and written hundreds of scrapers since my first VB5 controlling IE and dumping to SQLserver in the 90's and then moving to various php and python libs/frameworks and a handful of windows apps like ubot and imacros (both of which were useful to me at some point but I never use those nowadays)

A recent release of Rtila allows creating standalone bots you can run using it's built-in local Node.js server (which also has it's own locally hosted server API you can program anything else against using any language you like)

[1] www.rtila.net

[2] https://github.com/IKAJIAN/rtila-releases/releases

An alternative for beautifulsoup is https://github.com/scrapy/parsel also from the scrapy team.

variable.css(".X5PpBb::text").get() # returns a text value variable.css(".gs_a").xpath("normalize-space()").get() # https://github.com/scrapy/parsel/issues/192#issuecomment-1042301716 variable.css(".gSGphe img::attr(srcset)").get() # returns a attribute value variable.css(".I9Jtec::text").getall() # returns a list of strings values variable.xpath('th/text()').get() # returns text value using xpath

Something I don't see discussed when this topic is brought up is that Scrapy's HTML parsing library, parsel, can be installed separately from scrapy itself. You can use it in place of beautifulsoup and, imo, it's much easier to use.