pywb
conifer
pywb | conifer | |
---|---|---|
7 | 5 | |
1,303 | 1,457 | |
1.2% | -0.3% | |
7.2 | 0.0 | |
15 days ago | 6 months ago | |
JavaScript | Python | |
GNU General Public License v3.0 only | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pywb
-
Is there any good software for deduping (deduplicating) content in WARC files?
I have thousands of bookmarks on raindrop.io that I've been wanting to archive for a while. However, I've archived ~150 pages so far with Pywb and it ended up being 500MB across two WARCs, even with the dedupe setting specified in my settings file. It dedupes while archiving pages. I want software to get any spots missed and be sure that WARCs are actually deduped.
-
Is there a way to easily and reliably SSH to my laptop no matter what wifi the laptop is connected to? I have no clue.
I don't know if the solution would be related or relevant to this, but I would also want to be able to remotely launch and access a web server, Pywb, on Safari on my iPad, also no matter what wifi I'm on. On a Mac, it would be launched with the command wayback and the server would be accessed on the Browser with localhost:8080.
-
I can't install a Python package, pywb, looks like a problem with brotlipy. What can I do?
Check their github site. I would try "git clone https://github.com/webrecorder/pywb `
-
Purevolume archives?
I've been trying to open those large warc files these days. I've tried webrecorder, replayweb, pywb and warcat before but none of these worked well for me.
-
Ran grab-site now have some warc.gz files etc, the site in question was originally hosted in a mixture of html and javascript, what's the best and easiest way to make this accessible as a user for offline personal use?
pywb, but it requires creating a full copy of the data: https://github.com/webrecorder/pywb/issues/408
-
How good is ArchiveWeb.page?
I found it to be good with loading small WARCs quickly, but it can longer if the WARC is larger. Webarchive player, while it's old and discontinued, I've found it work better than Webrecorder Player and replayweb.page. If you want newer software to replay WARCs, try Pywb. I find it to be the best WARC player.
-
Saving all browsed websites automatically
I use pywb in proxy recording mode.
conifer
- YaCy, a distributed Web Search Engine, based on a peer-to-peer network
- I have no idea how Github works. I need to download a Conifer archiving tool. Can someone explain how to do? https://github.com/Rhizome-Conifer/conifer
-
[HELP] I´m looking for some self-hosted solution where users can connect to the website, connect to a page, browse it and save it.
I saw Rhizome-conifer(https://github.com/Rhizome-Conifer/conifer). It look great but there is a lot of open issues on github, some from 2016, and i´m afraid if it will not keep up (it would be very sad).
- Looking for a tool that generates reader mode of articles
-
Ask HN: What browser extensions are a must-have for HNers in 2021?
I would personally recommend
https://github.com/rhizome-conifer/conifer
The intent is a webrecorder for the internet.
It records all js libraries, loads videos, and all else.
Once stored, you can review a snapshot at that point in time.
They have a service option, webrecorder.io, but this one let's you store directly locally.
What are some alternatives?
warcio - Streaming WARC/ARC library for fast web archive IO
Wallabag - wallabag is a self hostable application for saving web pages: Save and classify articles. Read them later. Freely.
awesome-selfhosted - A list of Free Software network services and web applications which can be hosted on your own servers
Reddit-Enhancement-Suite - Reddit Enhancement Suite
replayweb.page - Serverless replay of web archives directly in the browser
SingleFile - Web Extension for saving a faithful copy of a complete web page in a single HTML file
SingleFileZ - Web Extension to save a faithful copy of an entire web page in a self-extracting ZIP file
ArchiveBox - 🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
22120 - 💾 Diskernet - Your preferred backup solution. It's like you're still online! Full text search archive from your browsing and bookmarks. Weclome! to the Diskernet: an internet on yer disk. Disconnect with Diskernet, an internet for the post-online apocalypse. Or the airplane WiFi. Or the site goes down. Or ... You get the picture. Get Diskernet. 80s logo. Formerly 22120 (project codename) ;P ;) xx;p [Moved to: https://github.com/i5ik/Diskernet]
temporal-shift-module - [ICCV 2019] TSM: Temporal Shift Module for Efficient Video Understanding
webarchiveplayer - NOTE: This project is no longer being actively developed.. Check out Webrecorder Player for the latest player. https://github.com/webrecorder/webrecorderplayer-electron) (Legacy: Desktop application for browsing web archives (WARC and ARC)
nmigen - A refreshed Python toolbox for building complex digital hardware. See https://gitlab.com/nmigen/nmigen