vllm

项目描述

vLLM is a high-throughput, memory-efficient inference and serving engine designed for Large Language Models (LLMs), offering optimized execution, multimodal support, and ease of use for model serving.