Markov Chains Are the Original Language Models

• markov-basics

  1 6 4.7 Jupyter Notebook

  Basic demonstration of Markov chains

It's true that Markov chains are very limited in their capabilities. But one thing I love about them is that they are one of the simplest and intuitive ways to write code that *learns* from input data.

If you're never written something the *learns*, try it out! Here's a very primitive one I wrote recently to explain the basic idea and explains it along the way.

https://github.com/unoti/markov-basics/blob/main/markov-basi...

• llama

  184 53,053 8.1 Python

  Inference code for Llama models

  

Predicting subsequent text is pretty much exactly what they do. Lots of very cool engineering that’s a real feat, but at its core it’s argmax(P(token|token,corpus)):

https://github.com/facebookresearch/llama/blob/main/llama/ge...

The engineering feats are up there with anything, but it’s a next token predictor.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project