go-imap VS gluon

The way I eventually solved it was to create a custom script in Go (using the emersion/go-imap library) that could interact with the IMAP server, retrieve all of the emails and then checking the headers and then moving them to the bin. Took about 60 seconds to process 44,000 emails and remove over 1,000 drafts that were in the wrong place.

They are going to replace go-imap with gluon. That's an effort which has been going on for quite a while.

Yeah, this really need attention now. I'm spotting the go-imap (which seems to be the real culprit) replacement being worked on, which is great! But, u/ProtonMail and u/bartbutler, please push all you can internally.

they seem to blame go-imap (https://github.com/emersion/go-imap). at a quick glance the library itself doesn't seem to have an open issue describing the problem.

curious if that author's (deprecated) protonmail bridge hydroxide (https://github.com/emersion/hydroxide) has the same problem.

github.com/emersion/go-imap

Check our blog to learn more: https://proton.me/blog/gluon-imap-library. You can peek into it yourself on our GitHub: https://github.com/ProtonMail/gluon.

They are going to replace go-imap with gluon. That's an effort which has been going on for quite a while.

Hi all, we want to share more info about the reported issue with Proton Mail Bridge, our desktop IMAP/SMTP gateway to Proton Mail encrypted email. The fact that Bridge and its client can become desynchronized sporadically for some users is a high priority issue we have been working on. Bridge is open source, and as a result relies upon open-source components, and the root cause is an architectural issue in a library that Bridge uses to implement IMAP. When there are network issues, this library returns errors to email clients. Unfortunately, there are hundreds of email clients, and some email clients don’t handle errors properly, and this leads to desynchronization. Our error tracking shows this does not happen often (1-2% of Bridge users) and the symptom is usually incorrect display of messages or read/unread status which is fixed with an inbox resynchronization. There are cases where a combination of a desynchronized mailbox and a specific series of user actions can lead to accidental email deletion, but this is far rarer than desynchronization. Our implementation tries as hard as possible to avoid this. If you find you are missing an email, our implementation works around the issue by placing it in your All Mail folder, so please check there. If you believe you have an email that is accidentally deleted, please contact our support team and we will do our best to help you undelete it. As Bridge is open source, updates on this issue have always been publicly posted on GitHub. IMAP-compatible encryption is a very difficult task and Bridge must interface with many different email clients, most of which are not open-source making it almost impossible to address all cases where certain clients don’t do error handling properly. Addressing this issue at the source requires replacing the core IMAP library. Unfortunately, there is not exactly a large selection of open-source IMAP libraries to pick from. None in fact are sufficiently well maintained to ensure we don’t run into this issue again. And a commercial library does not work because of Proton’s open-source nature. Therefore, the solution is to build our own IMAP library called Gluon, which we have been focusing on since this issue was reported to us. You can follow the progress or participate in this open-source effort here: https://github.com/ProtonMail/gluon

This is Bart, Proton CTO here. For clarity, the issue mentioned here only impacts Proton Mail Bridge, our desktop IMAP/SMTP gateway to Proton Mail encrypted email.

The fact that Bridge and its client can become desynchronized sporadically for some users is a high priority issue we have been working on. Bridge is open source, and as a result relies upon open-source components, and the root cause is an architectural issue in a library that Bridge uses to implement IMAP. When there are network issues, this library returns errors to email clients.

Unfortunately, there are hundreds of email clients, and some email clients don’t handle errors properly, and this leads to desynchronization.

Our error tracking shows this does not happen often (1-2% of Bridge users) and the symptom is usually incorrect display of messages or read/unread status which is fixed with an inbox resynchronization. There are cases where a combination of a desynchronized mailbox and a specific series of user actions can lead to accidental email deletion, but this is far rarer than desynchronization. Our implementation tries as hard as possible to avoid this. If you find you are missing an email, our implementation works around the issue by placing it in a users’ All Mail folder.

As Bridge is open source, updates on this issue have always been publicly posted on GitHub. Addressing this issue at the source requires replacing the core IMAP library. Unfortunately, there are no FOSS IMAP libraries that are sufficiently well maintained. Therefore, the solution is to build our own IMAP library called Gluon, which we have been focusing on since this issue was reported to us. You can follow the progress of this open-source project here: https://github.com/ProtonMail/gluon

We are not refusing to fix the problem. The only possible solution is writing a new open-source IMAP library which we can maintain ourselves to ensure this class of errors cannot occur again. We have doubled the size of the team working on this this year so it is a priority for us.

We’re confident that this addresses the main sources of desynchronization and will be available in the beta version of Bridge by the end of the year.