Gmail to SQLite

1. gmvault

   1 9 3,599 0.0 Python

   gmail backup software

   

   > the link to gmvault in your readme is now a dead end, is this it https://github.com/gaubert/gmvault.

   Funnily enough, the gmvault.org domain _that_ page points to is simply a parked GoDaddy placeholder. It's also not been updated in 10+ years except for two non-source files.

2. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

3. gmail-to-sqlite

   2 1 11 - Python

   Index your Gmail account to a SQLite DB and play with the data.

   

4. postsack

   3 10 555 0.0 Rust

   Visually cluster your emails by sender, domain, and more to identify waste

   

   I build something to visualize huge amounts of email (such as from Gmail) some years ago:

   https://github.com/terhechte/postsack

5. gbackup-rs

   4 2 24 10.0 Rust

   Fast and configurable CLI tool to back-up your GMail data - written in Rust

   

   Would love a comparison to gbackup-rs[0].

   To me having to install a tool through Python is a show-stopper.

   [0] https://github.com/djipko/gbackup-rs

6. got-your-back

   5 91 2,847 7.2 Python

   Got Your Back (GYB) is a command line tool for backing up your Gmail messages to your computer using Gmail's API over HTTPS.

   

   https://github.com/GAM-team/got-your-back

   - Open source

   - Resume (so backups/restores will eventually complete)

   Honorable mention: https://www.mailstore.com/en/products/mailstore-home/

   - Not open source

   - GUI with index: nice for searching mail locally

   - Resume only for backup (so restores generally fail)

7. mail2db

   6 1 2 7.4 Go

   A CLI tool that processes Maildir format email directories and stores email metadata in SQLite database.

   

   having sqlite exporters for platforms is great help for Archiving, but also general questions: I used https://github.com/ltdangle/mail2db to see how much mail volume I still have on a mail account that I want to mostly move away from. A top10 of senders directed my un- and resubscribe actions.

8. recordlite

   7 2 29 - Go

   

   And since view and indexes don't change the data, you can use tools like https://github.com/fsaintjacques/recordlite to automate schema management.

9. Stream

   getstream.io featured

   Stream - Scalable APIs for Chat, Feeds, Moderation, & Video. Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure.

   

10. email-oauth2-proxy

    8 20 1,108 7.4 Python

    An IMAP/POP/SMTP proxy that transparently adds OAuth 2.0 authentication for email clients that don't support this method.

    

    I built a proxy a while ago to make this easier - it lets you stick with IMAP/POP/SMTP as-is. No need for your client to even know that OAuth exists. See here: https://github.com/simonrob/email-oauth2-proxy

11. datasette

    9 198 10,155 8.2 Python

    An open source multi-tool for exploring and publishing data

    

    A couple of reasons which pop to mind:

    - Searching a plain text data file is O(n). Searching a SQLite database that has been properly indexed, which is very easy to do nowadays with FTS5, is O(log n) worst case scenario and O(1) in the best case. This doesn't explain why SQLite over a dataframe or anything, but it definitely justifies it over plain text for large email collections.

    - SQLite is really easy to write custom views and programs around. Virtually every major programming language can work with it without issue. See also: simonw's wonderful https://datasette.io/ .

    - SQLite is an accepted archival format by the Library of Congress, if you ever want to go down the rabbit hole of digital preservation.

12. aox

    10 1 116 5.3 C++

    Archiveopteryx—An advanced PostgreSQL-based IMAP/POP server

    

    I am reminded a little of the Postgres-backed IMAP server Archiveopteryx: https://github.com/aox/aox

    The schema from AOX always looked really good to me, but I never have gotten to really giving it a try. I wanted to use it, primarily, to get analytics about my mail and for search (not a daily-driver IMAP server).

13. Django-link-archive

    11 33 103 9.6 Python

    Self-hostable link database

    

    I recently tried to integrate Gmail in my app [0], and I poured too much time on it. I decided it is not worth to support Gmail.

    Gmail to SQLite describes 6 steps to get credentials working, but it is not true for me. After 6 steps:

    - that Google said that my app was not published, so I published it

    - Google said that app cannot be internal, because I am not a workspace user

    - for external apps

    - then it said I cannot use the app until it is verified

    - in verification they wanted to know domain, address, other details

    - they wanted to have my justification for scopes

    - they wanted to have video explaining how the app is going to be used

    - they will take some time to verify the data I provided them

    It all looks like a maze of settings, where requiring any of users to go above the hoops required by Google is simply too much.

    Links:

    [0] https://github.com/rumca-js/Django-link-archive

14. gmail-sqlite-db

    12 1 2 6.7 Python

    Fetch emails into a local SQLite Database

    

    Funny... I did the same thing yesterday, just because I wanted to list my recipient emails by domain. Code is awful, but here it is: https://github.com/hugoferreira/gmail-sqlite-db

15. gmvaultdb

    13 3 5 2.0 Python

    Store emails from a GMvault backup into an sqlite DB and extracts attachments in a dedicated folder

    

    FWIW, I used gmvault a long time ago (N.B. I typically deactivated individual .eml.gz compression in favor of a more global compression with a squashfs archive of the gmvault backup). Since I found not very practical to search through that archive, I developed https://github.com/karteum/gmvaultdb in order to convert it to a local sqlite DB (it also works with https://takeout.google.com/ with the caveat that in some case Takeout permanently losses some information because of a bug with older encodings so I'd always prefer a backup using gmvault or imap)

16. ImapSync

    14 143 3,675 2.6 Shell

    Imapsync is an IMAP transfers tool. The purpose of imapsync is to migrate IMAP accounts or to backup IMAP accounts. IMAP is one of the three current standard protocols to access mailboxes, the two others are POP3 and HTTP with webmails, webmails are often tied to an IMAP server. Upstream website is

    

17. SaaSHub

    www.saashub.com featured

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    

Suggest a related project