Does the DataCollatorForLanguageModelling class mask statically or dynamically?

• transformers

  175 124,557 10.0 Python

  🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

  

So I have been working closely with DataCollators, and I am unable to understand whether or not the DataCollatorForLanguageModelling class masks statically or dynamically. For reference, here is the code%20%2D%3E%20Tuple%5BAny%2C%20Any%5D%3A) for the aforementioned class. As per my understanding so far, I find it to be a simple static masking, but this issue posted on the repo says otherwise. If someone has any experience regarding this, please do help out, and if it is indeed static masking, how does one go about implementing dynamic masking?

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project