go-sbb-api VS GoQuery

Interesting article: I did something similar but for Zürich (finding a low-price, but good apartment there is almost impossible).

I did follow a different approach though, and reverse engineered the APIs of the most common rental listing websites here.

I didn't publish the code (or better said, I didn't make it public), but it's similar to yours. Instead of SQLite I used a PostGis database where I stored the apartments with their point to point public transport distance from my office (pre-covid search). I did it in Rust.

Whilst the application is not available to the public (yet), all of the libraries I've created are now available.

I should have both a Rust and a Golang version for most / all of them.

On a side note, I did a similar project (scraper + visualizer / search) for finding a job for my girlfriend by scraping LinkedIn, Xing and a couple of other local job posting websites.

[0]: https://github.com/denysvitali/sbb-api-rs

[1]: https://github.com/denysvitali/homegate-rs

[2]: https://github.com/denysvitali/flatfox-rs

[3]: https://github.com/denysvitali/go-sbb-api

Your comment was posted 4 minutes ago. That means you still have enough time to edit your comment to change it so it contains real URLs:

<https://github.com/PuerkitoBio/goquery>

<https://github.com/dop251/goja>

(Please do not reply to this comment—I won't be able to delete it once the previous post is fixed if it contains replies.)

How about - https://github.com/PuerkitoBio/goquery

Assuming you're asking about https://github.com/PuerkitoBio/goquery , to interpret your printout you want to look at what Find is defined to return, a *Selection. https://github.com/PuerkitoBio/goquery/blob/39fb6d4dc47a07e5782494b6defc89a194b1f906/traversal.go#L23

github.com/PuerkitoBio/goquery - library that provides a convenient and concise way to query HTML and XML documents. It provides a jQuery-like API for selecting elements and extracting data, making it a popular choice for web scraping in Go.

Yep, I thing I can add it. Add new goquery alternative will be good :)

If I want to parse the structure of HTML and not interact with it from a browser point of view, I use this in Go: https://github.com/PuerkitoBio/goquery

goquery

I had a similar problem that I solved with goquery and otto. You can use goquery to traverse the DOM and otto to execute the script fragment. Then just grab the data from otto's VM.

Your scraping being slow and using Chrome might be a blessing in disguise though. If you aren't careful you can get detected as a bot and banned from the site.

https://github.com/PuerkitoBio/goquery

If you're willing to add constraints around the goal, you can catch this type of error with semgrep rules and/or unit tests using qoquery.

RUN go get github.com/PuerkitoBio/goquery