Popular repositories Loading

1. control-layer control-layer Public

   The world’s fastest AI model gateway (450x less overhead than LiteLLM). Unified access to LLMs across endpoints (openAI, self-hosted, etc.) behind a single authentication layer - with API key gener…

   Rust 61 7

2. autobatcher autobatcher Public

   Drop-in AsyncOpenAI replacement that transparently batches requests

   Python 13 1

3. deepseek-reddit-agent deepseek-reddit-agent Public

   An example notebook which shows how you can build a LLM agent that scrapes information from Reddit and summarize key bullets using a self-hosted DeepSeek-R1-Distill-Llama-8B deployed with Titan Tak…

   Jupyter Notebook 11 2

4. inference-stack inference-stack Public

   The Doubleword Inference Stack is the easiest & most performant way to run genAI infrastructure in your private environment.

   Go Template 4

5. zerodp zerodp Public

   ZeroDP implements an efficient zero-copy data parallel approach for serving Mixture-of-Experts (MoE) models, where expert weights are shared across data parallel ranks via CUDA IPC (Inter-Process C…

   Python 4 1

6. arxiv-daily-digest arxiv-daily-digest Public

   A sample project that uses the Doubleword Batch API to sift through the new papers published on Arxiv and summarize them as a daily slack message.

   Python 3