inference
ChatGLM2-6B
inference | ChatGLM2-6B | |
---|---|---|
2 | 4 | |
2,871 | 15,547 | |
26.5% | 1.1% | |
9.8 | 6.6 | |
5 days ago | about 2 months ago | |
Python | Python | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
inference
-
GreptimeAI + Xinference - Efficient Deployment and Monitoring of Your LLM Applications
Xorbits Inference (Xinference) is an open-source platform to streamline the operation and integration of a wide array of AI models. With Xinference, you’re empowered to run inference using any open-source LLMs, embedding models, and multimodal models either in the cloud or on your own premises, and create robust AI-driven applications. It provides a RESTful API compatible with OpenAI API, Python SDK, CLI, and WebUI. Furthermore, it integrates third-party developer tools like LangChain, LlamaIndex, and Dify, facilitating model integration and development.
-
🤖 AI Podcast - Voice Conversations🎙 with Local LLMs on M2 Max
Code: https://github.com/xorbitsai/inference/blob/main/examples/AI_podcast.py
ChatGLM2-6B
- Are We Overlooking China's Progress in AI?
-
A new open-source language model claims to have surpassed GPT-4 right now. This needs to be fact-checked
If its benchmark results, eg on MMLU few-shot, hold and are indicative of its actual performance (which, mind you, isn't a given, for 6B nor for 130B), this 6B should be competitive with decent 30Bs. Plus natively long context and MQA. This is genuinely interesting, unlike boomer noises about CCP poisoning our checkpoints or whatever.
What are some alternatives?
truss - The simplest way to serve AI/ML models in production
PentestGPT - A GPT-empowered penetration testing tool
agentchain - Chain together LLMs for reasoning & orchestrate multiple large models for accomplishing complex tasks
Chinese-LLaMA-Alpaca - 中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)
h2o-wizardlm - Open-Source Implementation of WizardLM to turn documents into Q:A pairs for LLM fine-tuning
gpt_academic - 为GPT/GLM等LLM大语言模型提供实用化交互接口,特别优化论文阅读/润色/写作体验,模块化设计,支持自定义快捷按钮&函数插件,支持Python和C++等项目剖析&自译解功能,PDF/LaTex论文翻译&总结功能,支持并行问询多种LLM模型,支持chatglm3等本地模型。接入通义千问, deepseekcoder, 讯飞星火, 文心一言, llama2, rwkv, claude2, moss等。
mpt-30B-inference - Run inference on MPT-30B using CPU
LLMSurvey - The official GitHub page for the survey paper "A Survey of Large Language Models".
aihandler - A simple engine to help run diffusers and transformers models
llm-gateway - Gateway for secure & reliable communications with OpenAI and other LLM providers
rwkv.cpp - INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
DoctorGLM - 基于ChatGLM-6B的中文问诊模型