Our great sponsors
-
edlib
Lightweight, super fast C/C++ (& Python) library for sequence alignment using edit (Levenshtein) distance.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
I’ve got a similar situation. I was implementing the Smith-Waterman algorithm when I figured someone had to have already written a “fast” version of this. I found the edlib package (https://github.com/Martinsos/edlib) which does sequence alignment using Levenshtein distance. Essentially same DP algorithm as your traditional NW or SW only this is a C++ implementation with a Python wrapper. (I’m assuming you’re using Python, could be wrong though). The pertinent aspects of the output of this function contains the distance (dissimilarity) and the location (what index does the alignment start and end). This tool may go a ways to helping your pipeline. You could also look to metagenomic papers for inspiration as this is a problem (find a substring in a huge amount of data) that the community contends with all the time. Kmer based approach may also be useful if you want to attempt the alignment free path. Cheers.