nanobind VS spacy-experimental

Take a look at https://github.com/wjakob/nanobind

> More concretely, benchmarks show up to ~4× faster compile time, ~5× smaller binaries, and ~10× lower runtime overheads compared to pybind11.

People should not take that an endorsement of Swig.

Please use ctypes, cffi or https://github.com/wjakob/nanobind

Beazley himself is amazed that it (Swig) is still in use.

Nanobind made by the creator of PyBind11, it has a similar interface, but it takes leverage of C++17 and it aims to have more efficient bindings in space and speed.

I would recommend using NanoBind, the follow up of PyBind11 by the same author (Wensel Jakob), and move as much performance critical code to C or C++. https://github.com/wjakob/nanobind

If you really care about performance called from Python, consider something like NVIDIA Warp (Preview). Warp jits and runs your code on CUDA or CPU. Although Warp targets physics simulation, geometry processing, and procedural animation, it can be used for other tasks as well. https://github.com/NVIDIA/warp

Jax is another option, by Google, jitting and vectorizing code for TPU, GPU or CPU. https://github.com/google/jax

Trying to work with the newly released coreference resolution pipeline

I think the details are mentioned here: https://github.com/explosion/spacy-experimental/releases/tag/v0.6.0

I can't speak for the parent commenter, but there is ofte. code 'around' the machine learning code that benefits from high-performance implementations. To give two examples:

1. We recently implemented an edit tree lemmatizer for spaCy. The machine learning model predicts labels that map to edit trees. However, in order to lemmatize tokens, the trees need to be applied. I implemented all the tree wrangling in Cython to speed up processing and save memory (trees can be encoded as compact C unions):

https://github.com/explosion/spaCy/blob/master/spacy/pipelin...

2. I am working on a biaffine parser for spaCy. Most implementations of biaffine parsing use a Python implementation of MST decoding, which is unfortunately quite slow. Some people have reported it to dominate parsing time (rather than a rather expensive transformer + biaffine layer). I have implemented MST decoding in Cython and it barely shows up in profiles:

https://github.com/explosion/spacy-experimental/blob/master/...

Nós iremos utilizar o template do edit_tree_lemmatizer contido da pasta de projetos do repositório https://github.com/explosion/spacy-experimental e modificaremos para treinar um modelo em português em vez de alemão.