vllm-project
★ 91,844vllm
A high-throughput and memory-efficient inference and serving engine for LLMs.
HYSEN LABS DIRECTORY
vllm-project
Verified repositories, classifications and analysis from the vllm-project organization.
10 curated open-source projects
vllm-project
★ 6,792vllm-omni
A framework for efficient model inference with omni-modality models.
vllm-project
★ 5,826semantic-router
A programmable Mixture-of-Models router for heterogeneous LLM inference
vllm-project
★ 5,090aibrix
Cost-efficient and pluggable Infrastructure components for GenAI inference.
vllm-project
★ 3,785llm-compressor
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
vllm-project
★ 2,829vllm-ascend
Community maintained hardware plugin for vLLM on Ascend
vllm-project
★ 1,739vllm-metal
Community maintained hardware plugin for vLLM on Apple Silicon
vllm-project
★ 1,609guidellm
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
vllm-project
★ 833speculators
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
vllm-project
★ 467vime
An LLM post-training framework with vLLM for RL Scaling