smg-project/smg

★689Rust🇺🇸 North AmericaApache-2.0⑂ 201Issue 75Updated Oct 29, 2026

View on GitHub ↗Website ↗

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

A Rust gateway that unifies vLLM, TRT-LLM, SGLang and more — SMG is a Rust-built gateway offering full OpenAI and Anthropic API compatibility across vLLM, TRT-LLM, SGLang, TokenSpeed, Gemini and others. It adds KV cache-aware routing, a gRPC pipeline, tokenization caching, WASM plugins and multi-tenant auth, aimed at teams running large-scale, multi-engine LLM deployments.

Metrics

Retention can be computed 30 days after a repository first trends (it isn't shown until then). See About for how each metric is defined.

Star growth and Trending appearances (last 30 days)

Created Nov 22, 2025 · First on trending Oct 20, 2026

Other notable repositories in the same category (LLMs & ML)

• pst2154/nemotron-deciffusionStructured decisions from an 8B diffusion model in a single GPU pass
• todesking/slopsamplerCataloging the telltale phrases of AI-generated Japanese text
• datalab-to/indraA 1M-parameter OCR model that beats PaddleOCR and Tesseract on CPU
• fudan-generative-vision/RemorySoft memory that survives context compaction for LLM agents
• google-pytorch/torch_tpuA PyTorch Backend That Runs Models Directly on Google TPUs

See all LLMs & ML repositories →

Rank within anthropic

See all anthropic repositories →