lit VS markup

I'd also suggest looking at your data sample, and how your model handles it, with some kind of exploratory analysis tool. Google's Language Interpretability Tool might work for your scenario. This can give you a lot of ideas about how to prepare the data better.

Visualize and understand NLP models with the Language Interpretability Tool The Language Interpretability Tool (LIT) is for researchers and practitioners looking to understand NLP model behavior through a visual, interactive, and extensible tool. Use LIT to ask and answer questions like: What kind of examples does my model perform poorly on? Why did my model make this prediction? Can it attribute it to adversarial behavior, or undesirable priors from the training set? Does my model behave consistently if I change things like textual style, verb tense, or pronoun gender? LIT contains many built-in capabilities but is also customizable, with the ability to add custom interpretability techniques, metrics calculations, counterfactual generators, visualizations, and more.

Use LIT https://github.com/PAIR-code/lit

Hey HN! I'm super excited to share Markup with you, which is a totally free & open-source annotation tool that helps you transform unstructured text (e.g. news articles) into structured data that you can use for building, training, or fine-tuning ML models!

Check it out: https://github.com/samueldobbie/markup

Just to preface this summary, it's all a bit hacked together at the moment, and I'm in the process of rewriting the tool from scratch so this description is privy to change.

To generate the suggestions there's an active learner with an underlying random forest classifier, that has been fed ~60 seed sentences [1], to classify positive sentences (e.g. contains a prescription) and negative sentences (e.g. doesn't contain a prescription).

All positive sentences are fed into a sequence-to-sequence RNN model, that has been trained on ~50k synthetic rows of data [2] which maps unstructured sentences (e.g. patient is on pheneturide 250mg twice a day) to a structured output with the desired features (e.g. name: pheneturide; dose: 285; unit: g; frequency: 2). These synthetic sentences were generated with the in-built data generator [3].

The outputs of the RNN are validated to ensure they meet the expected structure and are valid for the sentence (e.g. the predicted drug name must exist somewhere within the sentence).

All non-junk predictions are shown to the user who can accept, edit, or reject each. Based on the users' response, the active learner is refined (currently nothing is fed back into the RNN).

[1] https://github.com/samueldobbie/markup/blob/master/data/text...

[2] https://raw.githubusercontent.com/samueldobbie/markup/master...

[3] https://www.getmarkup.com/tools/data-generator/