ModelTC
★ 4,289LightLLM
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
HYSEN LABS DIRECTORY
Verified repositories, classifications and analysis from the ModelTC organization.
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Project brief: Lightweight Image Video Action Generation Inference Framework. February 27, 2026: We now support FP8 and NVFP4 quantization for autoregressive video generation models (Self Forcing)!
[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.