vllm-project
★ 96,874vllm
A high-throughput and memory-efficient inference and serving engine for LLMs.
HYSEN LABS DIRECTORY
vllm-project
Verified repositories, classifications and analysis from the vllm-project organization.
11 curated open-source projects
vllm-project
★ 7,742vllm-omni
A framework for efficient model inference with omni-modality models.
vllm-project
★ 6,471semantic-router
A programmable Mixture-of-Models router for heterogeneous LLM inference
vllm-project
★ 5,186aibrix
Cost-efficient and pluggable Infrastructure components for GenAI inference.
vllm-project
★ 3,966llm-compressor
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
vllm-project
★ 3,081vllm-ascend
Community maintained hardware plugin for vLLM on Ascend
vllm-project
★ 2,730production-stack
vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization
vllm-project
★ 1,996vllm-metal
Community maintained hardware plugin for vLLM on Apple Silicon
vllm-project
★ 1,789guidellm
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
vllm-project
★ 957speculators
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
vllm-project
★ 504vime
An LLM post-training framework with vLLM for RL Scaling