ModuleFormer

ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward experts. We released a collection of ModuleFormer-based Language Models (MoLM) ranging in scale from 4 billion to 8 billion parameters. (by IBM)

ModuleFormer Alternatives

Similar projects and alternatives to ModuleFormer based on common topics and language

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better ModuleFormer alternative or higher similarity.

ModuleFormer reviews and mentions

Posts with mentions or reviews of ModuleFormer. We have used some of these posts to build our list of alternatives and similar projects.

Stats

Basic ModuleFormer repo stats
1
216
5.7
22 days ago

IBM/ModuleFormer is an open source project licensed under Apache License 2.0 which is an OSI approved license.

The primary programming language of ModuleFormer is Python.


Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com