esm
beartype
esm | beartype | |
---|---|---|
5 | 18 | |
2,871 | 2,444 | |
4.9% | 3.4% | |
4.6 | 9.4 | |
3 months ago | 1 day ago | |
Python | Python | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
esm
-
Large language models generate functional protein sequences across families
When evaluating this work, it’s important to remember that the functional labels on each of the 290 million input sequences were originally assigned by HMM as part of the pfam project, so the model is predicting a prediction.
Furthermore, the authors must engage a lot of human curation to ensure the sequences they generate are active. First, they pick an easy target. Second, they employ by-hand classical bioinformatics techniques on their predicted sequences after they are generated. For example, they manually align them and select those which contain specific important amino acids at specific positions which are present in 100% of functional proteins of that class, and are required for function. This is all done by a human bioinformatics expert before they test the “generated” sequences.
One other comment, in protein science, a sequence with 40% identity to another sequence is not “very different” if it is homologous. Since this model is essentially generating homologs from a particular class, it’s no surprise at a pairwise amino acid level, the generated sequences have this degree of similarity. Take proteins in any functional family and compare them. They will have the same overall 3-D structure—called their “fold”—yet have pairwise sequence identities much lower than 30–40%.
Not to be negative. I really enjoyed reading this paper and I think the work is important. Some related work by Meta AI is the ESM series of models [1] trained on the same data (the UniProt dataset [2]).
One thing I wonder is about the vocabulary size of this model. The number of tokens is 26 for the 20 amino acids and some extras, whereas for a LLM like Meta’s LLaMa the vocab size is 32,000. I wonder how that changes training and inference, and how we can adopt the transformer architecture for this scenario.
1. https://github.com/facebookresearch/esm
2. https://www.uniprot.org/help/downloads
- Google DeepMind CEO Says Some Form of AGI Possible in a Few Years
-
Can anyone suggest some 3D protein function prediction software? I was using 3DLigandSite and they’ve gone down indefinitely.
What's your input data look like? If you're predicting structures of mutants where there's a wild type structure available you can use variant prediction tools like ESM-IF or some of the protein language models like ESM-2
-
RFdiffusion: Diffusion model generates protein backbones
Such an explosion of protein AI lately. It’s the absolute best time to be a protein scientist with an interest in ML. Every new model type is inevitably tried out on proteins. In this case, by grad students at a very famous protein design lab (Baker Lab at University of Washington). And they usually find some interesting application. Protein design presents tons of interesting challenges.
The very largest plain transformer models trained on protein sequences (analogous to plain text) are about 15B parameters (I am thinking of Meta AI’s ESM-2 [1]). These can do for protein sequences what LLMs do for text (that is, they can “fill in the blank” to design variations, generate new proteins that look like their training data—which consists of all natural protein sequences), and tell you how likely it is that a given sequence exists.
Some cool variations of transformers have applications for protein design, like the now-famous SE(3) equivariant transformer used in the structure prediction module of AlphaFold [2], now appearing in TFA
1. https://github.com/facebookresearch/esm
-
Returning to snake's nest after a long journey, any major advances in python for science ?
Likewise PyTorch is seeing a lot of sciml work, in particular to do with protein design. (See e.g. ESM2.)
beartype
-
Writing Python Like Rust
https://github.com/beartype/beartype
I wish more people started using Beartype, it makes Python bearable
-
ChatGPT Git Hook Writes Your Commit Messages
I saw this on /r/Python the other day...
- When the client's management is happy but their dev team is a pain
-
Returning to snake's nest after a long journey, any major advances in python for science ?
As other folks have commented, type hints are now a big deal. For static typing the best checker is pyright. For runtime checking there is typeguard and beartype. These can be integrated with array libraries through jaxtyping. (Which also works for PyTorch/numpy/etc., despite the name.)
-
What are some features you wish Python had?
Maybe you're looking for https://github.com/beartype/beartype for runtime type enforcement; it's only at function calls, though, but probably a decent solution for codebases that are not completely typed for MyPy or pyright.
-
svg.py: Type-safe and powerful Python library to generate SVG files
It is though, if you add a type checker to your pipeline and use it without any escape hatches such as `Any` or `type: ignore`, you are essentially making the promise that your code is statically typed. But I say it is a matter of perspective because in my opinion runtime type checking should be avoided if we can get away with statically typed code, but there are type checkers that perform runtime type checking via annotations such as [Beartype](https://github.com/beartype/beartype) (with some trickery like assuming homogenous data structures as to not have to check every element of every structure). Anyway the definition of "type safe" is not 100% even in compiled languages.
- Python’s “Type Hints” are a bit of a disappointment to me
-
What's the best practice to validate parameter types at runtime in Python, with and without a third-party module?
There is the beartype project.
-
Statically typed Python
Personally I find working around mypy's quirks to be more effort than it's worth, so to offer another option: typeguard or beartype can be used to perform run-time type checking.
- Beartype: Unbearably fast runtime type checking in Python
What are some alternatives?
jaxtyping - Type annotations and runtime checking for shape and dtype of JAX/NumPy/PyTorch/etc. arrays. https://docs.kidger.site/jaxtyping/
typeguard - Run-time type checker for Python
progen - Official release of the ProGen models
pydantic - Data validation using Python type hints
mypy - Optional static typing for Python
pyright - Static Type Checker for Python
mypyc - Compile type annotated Python to fast C extensions
RFdiffusion - Code for running RFdiffusion
toit - Program your microcontrollers in a fast and robust high-level language.
jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
benchmarks - Some benchmarks of different languages