Our great sponsors
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Hi eesmith, thanks for your insight! I've applied a lot of the improvements you mentioned, the collections.Counter one didn't even cross my mind- very neat:)
The lines are processed independently as they are separate sentences. One line may be from one source, and the next from a completely different source. The problem with simply concatenating them is that the target words on the boundaries between sentences can end up with context words from unrelated sentences.
>I liked seeing the 'η' - I haven't yet used non-ASCII in my code!
Then you may enjoy this april fool's python issue I made https://github.com/python/cpython/issues/103172
Yeah, I like using utf-8 chars for mathematical symbols, it makes my brain hurt a little less when mapping between the two! I also like using ŷ for predictions in ML contexts as that's the canonical symbol used in the literature.