Pinned Loading

1. fastertransformer_backend fastertransformer_backend Public

   Forked from triton-inference-server/fastertransformer_backend

   Python

2. onnxruntime onnxruntime Public

   Forked from microsoft/onnxruntime

   ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

   C++

3. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 74.7k 15k

4. huggingface/text-embeddings-inference huggingface/text-embeddings-inference Public

   A blazing fast inference solution for text embeddings models

   Rust 4.6k 377

5. vllm-project/llm-compressor vllm-project/llm-compressor Public

   Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

   Python 3k 457