llama

ollama/ollama

Ollama adds direct launchers for Claude Code, Codex, and coding agents — Ollama now ships one-command integrations for coding tools like Claude Code, Codex, Copilot CLI, Droid and OpenCode, plus OpenClaw for a chat-based AI assistant across WhatsApp, Telegram, Slack and Discord. Useful for developers who want local or cloud model access wired straight into their existing workflows.

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

vllm-project/vllm

vLLM keeps growing as a broad, hardware-agnostic LLM serving engine — vLLM is a high-throughput inference engine using PagedAttention, continuous batching, and quantization (FP8, INT4, GGUF, and more) to serve LLMs cheaply. Backed by over 2000 contributors, it now supports CUDA, AMD, TPU and Blackwell hardware plus models like DeepSeek-V3, Qwen3, and GPT-OSS, making it suited for teams deploying large models at scale.

A high-throughput and memory-efficient inference and serving engine for LLMs

unslothai/unsloth

Unsloth ships a desktop app to run and fine-tune LLMs locally — Unsloth now offers a native desktop app for Windows, macOS and Linux that lets users run and train LLMs and diffusion models locally, supporting GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4 and FLUX. It targets self-hosted, privacy-conscious users who want fine-tuning and inference without cloud dependency.

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

mudler/LocalAI

A self-hosted AI engine that swaps backends like plugins — LocalAI wraps llama.cpp, vLLM, whisper.cpp, stable-diffusion and MLX each in its own image instead of bundling everything, keeping the core small. It runs LLMs, vision, voice, image and video models via an OpenAI-compatible API on CPU or GPU, with agents, rerank and distributed/libp2p setups for self-hosting without a GPU requirement.

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

AstrBotDevs/AstrBot

AstrBot: one bot core for Telegram, Discord, QQ and LLM agents — AstrBot bundles chat platform integrations, LLM access, plugins and agent features into a single Python framework, positioned as a self-hosted alternative for building AI assistants across IM apps. It supports MCP and runs via Docker, aimed at developers who want one codebase instead of separate bots per platform.

AI Agent Assistant & development framework that integrates lots of IM platforms, LLMs, plugins and AI feature, and can be your openclaw alternative. ✨

lyogavin/airllm

Run massive LLMs like 2.8T-param Kimi K3 on a single 4GB GPU — AirLLM streams model weights layer by layer so huge LLMs (even a 2.8T-param model or DeepSeek-V3) run inference without quantization or pruning on consumer GPUs. New training support lets you fine-tune 125B models under 6GB VRAM, useful for researchers and hobbyists lacking datacenter hardware.

AirLLM 2.8T LLM inference with single 4GB GPU

sgl-project/sglang

Unified inference engine for LLMs, VLMs, and diffusion models — SGLang now bundles SGLang Diffusion, a built-in image/video generation engine, alongside its existing LLM and vision-language serving stack. It targets agentic workloads, large-scale serving, and RL rollouts, with broad hardware and model support including Blackwell GPUs, MoE architectures, and models like DeepSeek, Qwen, and GLM.

SGLang is a high-performance serving framework for large language models and multimodal models.

multimodal-art-projection/YuE

YuE2 merges symbolic and audio music generation into one model — YuE2 moves beyond raw audio-only generation by unifying symbolic composition with audio synthesis, adding zero-shot style covers and agentic editing skills. It targets researchers and musicians wanting fine-grained control over composition plus free online demos, benchmarks, and Hugging Face checkpoints.

YuE2: frontier music generation with symbolic planning, zero-shot covers, and agentic music editing.

AAswordman/Operit

The most powerful AI agent and AI chat software on Android/Operit是一款Android上能力最为强大、发展最久的AI Agent

higgsfield-ai/higgsfield

Multi-node LLM training with a GitHub-Actions-style workflow — Higgsfield is an open-source GPU orchestration layer and ML framework for training billion- to trillion-parameter models like LLaMA. It handles node allocation, ZeRO-3/FSDP sharding, experiment queuing, and CI-style integration with GitHub Actions, letting developers launch distributed training with minimal boilerplate code.

Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters

mostlygeek/llama-swap

A lightweight proxy that hot-swaps local AI models on the fly — llama-swap is a single Go binary that sits in front of llama.cpp, vllm, stable-diffusion.cpp and similar servers, switching models on demand via one config file. It supports both OpenAI and Anthropic API endpoints, including chat, embeddings, images, audio and rerank, making it useful for anyone running multiple local models without juggling separate server instances.

Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc