scrapeghost VS ddgr

Not my project but an ex-colleague has been having some success in this direction: https://jamesturk.github.io/scrapeghost/

Yes, there is something like that available - ScrapeGhost.

With AI scraping, tools can be far more resilient than soon enough to minor dom changes. See - https://jamesturk.github.io/scrapeghost/.

Their ToS mentions scraping but it pertains to scraping their frontend instead of using their API, which they don't want you to do.

Also - this library requests the HTML by itself [0] and ships it as a prompt but with preset system messages as the instruction [1].

[0] - https://github.com/jamesturk/scrapeghost/blob/main/src/scrap...

[1] - https://github.com/jamesturk/scrapeghost/blob/main/src/scrap...

Using the shell commands plugin and ddgr, I managed to create a nice trick, which allows you to add the link to selected text without having to google it (hard to describe, but the gif should give you the idea). Here is how it works: - install the shell commands plugin https://obsidian.md/plugins?id=obsidian-shellcommands - install ddgr https://github.com/jarun/ddgr - add the following code as shell command: bash query="{{selection}}" link=$(ddgr --num=1 --json "$query" | grep "url" | cut -d'"' -f4) mdlink="[$query]($link)" echo -n "$mdlink" - in the settings for that shell command, set the stdout to "current file: caret position" - you are good to go. (Maybe give it a hotkey)

search from the CLI is hardly new.. some examples by jarun: - google: https://github.com/jarun/googler - duckduckgo: https://github.com/jarun/ddgr

Leveraging filter commands (i.e. :!) to easily/quickly manipulate lines or entire buffers/files. For example, :!date will run the external date and show you results, but :.!date (which is done by typing !! then date will run the external date command and put the result on the current line. But, also, if you have the word date on a line, then you can run :.!bash (which is done by typing !! then bash), which will execute the command date and replace the current line with the result. There are infinite uses for this, like :!sort (for sorting text), :!column -t (for aligning/tabulating text), :!awk for text manipulation, :!ddgr duckduckgo cli, ...etc

I like ddgr https://github.com/jarun/ddgr. Duckduckgo for the terminal. googler (https://github.com/jarun/googler) was great but google seem to have crippled it, at least for me. ddgr --site youtube.com QUERY .... You can use the builtin URL handler, or the JSON output, to pass the URL to a video player like mpv. yt-dlp is required for mpv to launch the video from URL.

Here is link number 1 - Previous text "Yes"