pywb VS 22120

I have thousands of bookmarks on raindrop.io that I've been wanting to archive for a while. However, I've archived ~150 pages so far with Pywb and it ended up being 500MB across two WARCs, even with the dedupe setting specified in my settings file. It dedupes while archiving pages. I want software to get any spots missed and be sure that WARCs are actually deduped.

I don't know if the solution would be related or relevant to this, but I would also want to be able to remotely launch and access a web server, Pywb, on Safari on my iPad, also no matter what wifi I'm on. On a Mac, it would be launched with the command wayback and the server would be accessed on the Browser with localhost:8080.

Check their github site. I would try "git clone https://github.com/webrecorder/pywb `

I've been trying to open those large warc files these days. I've tried webrecorder, replayweb, pywb and warcat before but none of these worked well for me.

pywb, but it requires creating a full copy of the data: https://github.com/webrecorder/pywb/issues/408

I found it to be good with loading small WARCs quickly, but it can longer if the WARC is larger. Webarchive player, while it's old and discontinued, I've found it work better than Webrecorder Player and replayweb.page. If you want newer software to replay WARCs, try Pywb. I find it to be the best WARC player.

I use pywb in proxy recording mode.

Here. An archivist browser controller that caches everything you browse, a library server with full text search to serve your archive.

Following your "hilarious" and disrespectful answer here https://github.com/i5ik/22120/issues/63#issuecomment-7275272..., I would prefer that you remove any reference to SingleFile in the description of your project. I could not open an issue because you blocked me. And please don't accuse people without proof.

https://github.com/i5ik/22120 is a tool that archives as you browse that you can then view offline later

My solution to this (it's been done before), is to use the existing browser engine (not the system webview) installed. So far I only utilize Chrome, but as the way I connect to it is over the Chrome DevTools protocol which is somewhat fluent with the Remote Debugging Protocol[0] that Firefox is doing, this is a reasonable approach.

So far my "tool" to do this is simply a template repository with some conveniences, providing in essence a skeleton for these types of apps. I hope to flesh this out a little more, and expose a much richer API, as well as convert some of my existing popular apps (like 22120[1]) to the "framework".

The benefit of this is Graderjs has a built in 'app builder' that can create a cross-platform binary (excluding or ignoring the necessity (on MacOS) and near-necessity (on Windows) to sign your executable somehow, that lets you display your UI in JS/HTML/CSS using the already installed browser engine, as well as run code in NodeJS and using the rich APIs[2] of the browser engine itself. I'm really happy with this project and think that, even tho it's small now, it will in time become my most popular and powerful one: even bigger than my remote browser and popular web archiver.

Just give it time! :)

[0]: https://firefox-source-docs.mozilla.org/remote/index.html

[1]: https://github.com/i5ik/22120

[2]: https://chromedevtools.github.io/devtools-protocol/tot/Brows...

The GraderJS: https://github.com/i5ik/graderjs

I use this to backup pages automatically

https://github.com/i5ik/22120

Does this potentially help? https://github.com/c9fe/22120

From the blog comments, I think this is what you’re after https://github.com/c9fe/22120