Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 76k 15.4k

2. horovod/horovod horovod/horovod Public

   Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

   Python 14.7k 2.2k

3. kserve/kserve kserve/kserve Public

   Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

   Go 5.3k 1.4k

4. IsaacRe/vllm-kvcompress IsaacRe/vllm-kvcompress Public

   KV cache compression for high-throughput LLM inference

   Python 155 7

5. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python