Machine Learning Compilation for Large Language Models (MLC LLM)
Documentation:
Machine Learning Compilation for Large Language Models (MLC LLM) is a high-performance universal deployment solution that allows native deployment of any large language models with native APIs with compiler acceleration. The mission of this project is to enable everyone to develop, optimize and deploy AI models natively on everyone’s devices with ML compilation techniques.
Installation
MLC LLM is available via . It is always recommended to install it in an isolated conda virtual environment.
To verify the installation, activate your virtual environment, run
python -c "import mlc_llm; print(mlc_llm.__path__)"
You are expected to see the installation path of MLC LLM Python package.
Quick Start
Please check out our documentation for the .
Introduction
Please check out our documentation for the .
Links
- You might want to check out our online public for a systematic walkthrough of our approaches.
- is a companion project using MLC LLM’s WebGPU and WebAssembly backend.
- is a companion project for diffusion models with the WebGPU backend.
Disclaimer
The pre-packaged demos are subject to the model License.