Are there any tools for seeing / understanding what a fine-tuned BERT model is looking at for a downstream task?

• bertviz

  15 6,377 3.9 Python

  BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)

From what I can tell, BertViz is one such visualization tool, but (1) I'm not sure what reading those outputs adds (maybe "this article is about cats" has a strong connection to the [CLS] token used for classification, and the rest of the document has less attention paid to it?); (2) it looks like it's only for pretrained models, not fine-tuned ones.

• lit

  3 3,394 9.3 TypeScript

  The Learning Interpretability Tool: Interactively analyze ML models to understand their behavior in an extensible and framework agnostic interface. (by PAIR-code)

  

Use LIT https://github.com/PAIR-code/lit

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project