SaaSHub helps you find the best software and product alternatives Learn more →
Top 6 Python model-converter Projects
-
MMdnn
MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.
-
coremltools
Core ML tools contain supporting tools for Core ML model conversion, editing, and validation.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
Ckpt2Diff
This user-friendly wizard is used to convert a Stable Diffusion Model from CKPT format to Diffusers format.
Project mention: CoreML commit from Apple mentions iOS17 exclusive features | /r/u_Standard-Sundae-6011 | 2023-06-03
Project mention: [D] Object detection models that can be easily converted to CoreML | /r/MachineLearning | 2023-07-25
Project mention: I made an app that runs Mistral 7B 0.2 LLM locally on iPhone Pros | news.ycombinator.com | 2024-01-07Conceptually, to the best of my understanding, nothing too serious; perhaps the inefficiency of processing a larger input than necessary?
Practically, a few things:
If you want to have your cake & eat it too, they recommend Enumerated Shapes[1] in their coremltools docs, where CoreML precompiles up to 128 (!) variants of input shapes, but again this is fairly limiting (1 tok, 2 tok, 3 tok... up to 128 token prompts.. maybe you enforce a minimum, say 80 tokens to account for a system prompt, so up to 200 tokens, but... still pretty short). But this is only compatible with CPU inference, so that reduces its appeal.
It seems like its current state was designed for text embedding models, where you normalize input length by chunking (often 128 or 256 tokens) and operate on the chunks — and indeed, that’s the only text-based CoreML model that Apple ships today, a Bert embedding model tuned for Q&A[2], not an LLM.
You could used a fixed input length that’s fairly large; I haven’t experimented with it once I grasped the memory requirements, but from what I gather from HuggingFace’s announcement blog post[3], it seems that is what they do with swift-transformers & their CoreML conversions, handling the details for you[4][5]. I haven’t carefully investigated the implementation, but I’m curious to learn more!
You can be sure that no one is more aware of all this than Apple — they published "Deploying Transformers on the Apple Neural Engine" in June 2022[6]. I look forward to seeing what they cook up for developers at WWDC this year!
---
[1] "Use `EnumeratedShapes` for best performance. During compilation the model can be optimized on the device for the finite set of input shapes. You can provide up to 128 different shapes." https://apple.github.io/coremltools/docs-guides/source/flexi...
[2] BertSQUAD.mlmodel (fp16) https://developer.apple.com/machine-learning/models/#text
[3] https://huggingface.co/blog/swift-coreml-llm#optimization
[4] `use_fixed_shapes` "Retrieve the max sequence length from the model configuration, or use a hardcoded value (currently 128). This can be subclassed to support custom lengths." https://github.com/huggingface/exporters/pull/37/files#diff-...
[5] `use_flexible_shapes` "When True, inputs are allowed to use sequence lengths of `1` up to `maxSequenceLength`. Unfortunately, this currently prevents the model from running on GPU or the Neural Engine. We default to `False`, but this can be overridden in custom configurations." https://github.com/huggingface/exporters/pull/37/files#diff-...
[6] https://machinelearning.apple.com/research/neural-engine-tra...
Project mention: Introducing Nobuco: PyTorch to Tensorflow converter. Intuitive, flexible, efficient. | /r/pytorch | 2023-07-02Hence, [Nobuco](https://github.com/AlexanderLutsenko/nobuco). It's designed with simplicity and hackability in mind and automatically solves mismatching channel orders offering near-optimal performance. Try it, and spread the word if you like it. If not, feel free to open an issue or feature request. Any forms of contribution are highly welcome!
Python model-converter related posts
- [D] Object detection models that can be easily converted to CoreML
- Orange Pi 5 Plus Koboldcpp Demo (MPT, Falcon, Mini-Orca, Openllama)
- [R] Importing TensorFlow neural networks stored as *.h5
- [D] Tools for converting TF code to Pytorch
- Apple’s New M1 Chip is a Machine Learning Beast
-
A note from our sponsor - SaaSHub
www.saashub.com | 19 Apr 2024
Index
What are some of the best open-source model-converter projects in Python? This list will help you:
Project | Stars | |
---|---|---|
1 | MMdnn | 5,782 |
2 | coremltools | 4,037 |
3 | mmdeploy | 2,481 |
4 | exporters | 525 |
5 | nobuco | 177 |
6 | Ckpt2Diff | 12 |