WordDumb VS rake-nltk

Manual here: https://xxyzz.github.io/WordDumb/

X-ray. You want X-ray. For sideloaded books you can try https://github.com/xxyzz/WordDumb

What part are you failing on? The earliest one, assuming you have Calibre installed already, is that you should be downloading from the releases page and not trying to download the entire code repository.

First option is to use Count pages plugin, and second option is to use kfx output, which you can enable option to create approximate page numbers. Also worth mentioning that the kfx outputs also supports x-ray and word wise, if you use WordDumb plugin.

github page here, mobileread page here

Since that itself is pretty opaque to non-techies, the Wordumb home forum is here, and it looks like the github page you need page is here.

WordDumb You can use this to generate Word Wise and X-ray for your ebooks.

go to calibre prefrences, click on plugin, click on install new plugin search for kfx output, kfx input and worddumb (documentation for these plugin is availabe here kfx output , kfx input and worddumb ) (note: for worddumb you also need to install python)

I'm not sure what you mean by tokenizing phrases or concepts. Specifically extracting institution names would fall under NER. You can do this with spaCy. Extracting commonly used phrases would fall under keyword extraction. For this, you can study frequencies of n-grams of length > 1 and optionally filter based on POS (i.e. NOUN+ADJ). I've never used RAKE (https://github.com/csurfer/rake-nltk) but I've heard this is also a popular method.