vllm-project

vllm-project

vllm

★ 91.844

Eine speichereffiziente Inferenz- und Serving-Engine mit hohem Durchsatz für LLMs.

vllm-project

vllm-omni

★ 6.792

Ein Framework für effiziente Modellinferenz mit Omnimodalitätsmodellen.

vllm-project

semantic-router

★ 5.826

A programmable Mixture-of-Models router for heterogeneous LLM inference

vllm-project

aibrix

★ 5.090

Kosteneffiziente und steckbare Infrastrukturkomponenten für GenAI-Inferenz.

vllm-project

llm-compressor

★ 3.785

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

vllm-project

vllm-ascend

★ 2.829

Community maintained hardware plugin for vLLM on Ascend

vllm-project

vllm-metal

★ 1.739

Community maintained hardware plugin for vLLM on Apple Silicon

vllm-project

guidellm

★ 1.609

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

vllm-project

speculators

★ 833

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

vllm-project

vime

★ 467

An LLM post-training framework with vLLM for RL Scaling