BERT-pytorch VS scibert

Compare BERT-pytorch vs scibert and see what are their differences.

BERT-pytorch

Google AI 2018 BERT pytorch implementation (by codertimo)
Bert Transformer Pytorch NLP language-model
Source Code
Suggest alternative
Edit details

scibert

A BERT model for scientific text. (by allenai)
Bert NLP scientific-papers
Source Code
arxiv.org
Suggest alternative
Edit details
InfluxDB
Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
Our great sponsors
BERT-pytorch scibert
1 2
5,995 1,402
- 0.0%
0.0 0.0
8 months ago about 2 years ago
Python Python
Apache License 2.0 Apache License 2.0
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.

BERT-pytorch

Posts with mentions or reviews of BERT-pytorch. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2021-05-20.
  • Lack of activation in transformer feedforward layer?
    2 projects | /r/learnmachinelearning | 20 May 2021
    I'm curious as to why the second matrix multiplication is not followed by an activation unlike the first one. Is there any particular reason why a non-linearity would be trivial or even avoided in the second operation? For reference, variations of this can be witnessed in a number of different implementations, including BERT-pytorch and attention-is-all-you-need-pytorch.

scibert

Posts with mentions or reviews of scibert. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2022-11-15.

What are some alternatives?

When comparing BERT-pytorch and scibert you can also consider the following projects:

haystack - :mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

bertviz - BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)

transformers - 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

paperai - 📄 🤖 Semantic search and workflows for medical/scientific papers

transformer-pytorch - Transformer: PyTorch Implementation of "Attention Is All You Need"

clip-as-service - 🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP

Transformers4Rec - Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation and works with PyTorch.

cuad - CUAD (NeurIPS 2021)

galai - Model API for GALACTICA

BERT-pytorch vs haystack scibert vs haystack BERT-pytorch vs bertviz scibert vs transformers BERT-pytorch vs transformers scibert vs paperai BERT-pytorch vs transformer-pytorch scibert vs clip-as-service BERT-pytorch vs Transformers4Rec scibert vs bertviz BERT-pytorch vs cuad scibert vs galai