Our great sponsors
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
gtn_applications
Discontinued Applications using the GTN library and code to reproduce experiments in "Differentiable Weighted Finite-State Transducers"
This uses dense (soft/weighted) transitions from any state to any state, and then some regularization to guide it to more sparse solutions.
In practice, the number of states can be huge (thousands, maybe millions), so representing this as a dense matrix (a 1Mx1M matrix is way too big) is not going to work. It must be sparse, and in practice (all FST you usually deal with) it is. So it's very much a waste to represent it as a dense matrix.
That's why there are many specialized libraries to deal with FSTs. Also in combination with deep learning tools. See e.g. K2 (https://github.com/k2-fsa/k2).
FB research has their own version of automatic differentiation of WFSTs: https://github.com/gtn-org/gtn
See also https://github.com/facebookresearch/gtn_applications which contains examples of applications such as handwriting recognition and speech recognition.
FB research has their own version of automatic differentiation of WFSTs: https://github.com/gtn-org/gtn
See also https://github.com/facebookresearch/gtn_applications which contains examples of applications such as handwriting recognition and speech recognition.
If you're interested in these kinds of things, many years ago we created TerpreT (https://arxiv.org/pdf/1608.04428.pdf and https://github.com/51alg/TerpreT) to look into generic program synthesis problems, using a set of very different techniques (gradient descent, ILP, SMT) on different problem settings (turing machines, boolean circuits, LLVM IR-style basic blocks, and straight assembly).