rbspy
lexbor
Our great sponsors
rbspy | lexbor | |
---|---|---|
10 | 10 | |
2,456 | 881 | |
0.4% | 1.7% | |
8.6 | 8.5 | |
2 days ago | 6 days ago | |
Rust | C | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
rbspy
- Flameshow: A Terminal Flamegraph Viewer
-
When counting lines in Ruby randomly failed SerpApi deployments
We used rbspy to generate the flamegraph:
-
EventMachine Performance Spikes
You could use rbspy to profile the EventMachine process as it's doing the workload, and try to see there where most of the time is being spent.
-
Rails Resque - AWS ECS task randomly stuck
Not sure what your issue is and you got some of this info from strace, but FYI rbspy can also help profile where a running Ruby process is spending its time: https://rbspy.github.io/. I think it's a little more helpful than strace in that it should bring it back to Ruby code rather than just showing the raw syscalls.
-
Optimizing your tests in 5 steps
Even the most general profiler tool will show you each statement's accumulated time. These are called statistical profilers and give you a panoramic view of what the test is doing. An example of such a profiler is rbspy:
-
Fantastic Symbols and Where to Find Them - Part 2
As a result, you need to craft a specific routine for each interpreter runtime (in some cases, each version of that runtime) to obtain symbol information. Educated eyes might have already noticed, it's not an easy undertaking considering the sheer amount of interpreted languages out there. For example, a very well known Ruby profiler, rbspy, generates code for reading internal structs of the Ruby runtime for each version.
-
How to use strace on threads managed by supervisor? i.e. i want to ´supervisorctl restart someService´ and strace someService.
For production, it depends on what the process is built with. There are possibly better tracing tools than just strace. For example Java has JMX, Go has pprof. There's also things like rbspy for Ruby.
-
Why Is JRuby Slow?
I was a bit surprised to find out that the tool used to generate the flamegraph for Ruby (rbspy [1]) is written in Rust. lol
-
Profiling Rails app that uses websockets
Maybe https://github.com/rbspy/rbspy would allow you to profile the running ruby process handling web sockets?
-
How SerpApi sped up data extraction from HTML from 3s to 800ms (or How to profile and optimize Ruby code and C extension)
c function is not very helpful to find the performance problem, so we dug deeper.
lexbor
-
Modest: A fast HTML renderer implemented as a pure C99 library
Project is deprecated in favour of the same developer's lexbor project[0].
-
Created a performance-focused HTML5 parser for Ruby, trying to be API-compatible with Nokogiri
It supports both CSS selectors and XPath like Nokogiri, but with separate engines - parsing and CSS engine by Lexbor, XPath engine by libxml2. (Nokogiri internally converts CSS selectors to XPath syntax, and uses XPath engine for all searches).
- Lexbor: Fast HTML Renderer library in C
-
Andreas Kling (of SerenityOS fame) is building a new Linux browser using SerenityOS libraries
An HTML parser, probably the simplest relatively modern example I could find is 1MB https://github.com/lexbor/lexbor (haven't used it, but might look more into it now that I know it exists.)
- Lexbor: Open-source HTML Renderer library in C
-
The State of Web Scraping in 2021
Lazyweb link: https://github.com/rushter/selectolax
although I don't follow the need to have what appears to be two completely separate HTML parsing C libraries as dependencies; seeing this in the readme for Modest gives me the shivers because lxml has _seen some shit_
> Modest is a fast HTML renderer implemented as a pure C99 library with no outside dependencies.
although its other dep seems much more cognizant about the HTML5 standard, for whatever that's worth: https://github.com/lexbor/lexbor#lexbor
---
> It looks like the author of the article just googled some libraries for each language and didn't research the topic
Heh, oh, new to the Internet, are you? :-D
-
Libraries for retrivieng html data from website
Lexbor is here: https://github.com/lexbor/lexbor
-
What second language to learn after Python?
Well, regarding HTML5, what I've found was libxml (does not support tag-soup HTML5), https://github.com/lexbor/lexbor, for which I was unable to find good documentation ( see https://lexbor.com/docs/lexbor/#dom), Apache Xerces (appears to not support tag-soup HTML5 as well), and Gumbo, which does not appear to be active and to support selectors and XPath (although there are libraries that add that).
-
You can't parse [X]HTML with regex
I think we've all (mostly?) tried it. It really is the Wild West of the web when you're trying to parse other people's HTML, though.
I've played around with this parser which is extremely quick. https://github.com/lexbor/lexbor
-
How SerpApi sped up data extraction from HTML from 3s to 800ms (or How to profile and optimize Ruby code and C extension)
I’m glad to have the opportunity to contribute to an open-source project that is used by thousands of people. Hopefully, we will speed up Nokogiri (or XML parser it uses) to match the performance of html5ever or lexbor at some point in the future. 800 ms to extract data from HTML is still too much.
What are some alternatives?
flamegraph - Easy flamegraphs for Rust projects and everything else, without Perl or pipes <3
myhtml - Fast C/C++ HTML 5 Parser. Using threads.
bcc - BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more
selectolax - Python binding to Modest and Lexbor engines (fast HTML5 parser with CSS selectors).
parca-agent - eBPF based always-on profiler auto-discovering targets in Kubernetes and systemd, zero code changes or restarts needed!
gumbo-parser - An HTML5 parsing library in pure C99
ruby-ll
Xerces-C++ - Apache Xerces-C validating XML parser
perf-map-agent - A java agent to generate method mappings to use with the linux `perf` tool
nokogiri-rust - Ruby FFI wrapper around scraper crate to be used instead of Nokogiri. Status: proof of concept.
flamescope - FlameScope is a visualization tool for exploring different time ranges as Flame Graphs.
pyppeteer - Headless chrome/chromium automation library (unofficial port of puppeteer)