transformers

10 repositories

rohitg00/ai-engineering-from-scratch

A 542-lesson curriculum for building AI systems from the ground up — A free, MIT-licensed course with 542 lessons across 20 phases covering model internals, retrieval pipelines, and agent runtimes in Python, TypeScript, Rust, and Julia. Learners write code, run evals, and inspect failures, with labs and project challenges for self-paced or agent-assisted study.

Python🇪🇺 EuropeAttention 44
★79,622
labmlai/annotated_deep_learning_paper_implementations

A literate-programming library of 60+ annotated deep learning papers — This repo pairs PyTorch code with side-by-side explanatory notes for transformers, GANs, RL algorithms, optimizers and more, rendered on a companion website. It's actively updated weekly and now points to a separate 'Annotated Implementation Skill' for using coding agents to implement new papers in the same style. Useful for learners and engineers wanting to read algorithms as documented code rather than just papers.

PythonAttention 17
★67,774
arc53/DocsGPT

Self-hosted AI agents that answer strictly from your own documents — DocsGPT is a private, self-hosted platform for building AI agents and assistants grounded in your own docs, with a visual Agent Builder, deep research, document analysis, multi-model support and API connectors. A one-line install script makes it quick to run locally, aimed at teams wanting enterprise search without sending data to third parties.

Python🇪🇺 EuropeAttention 7
★18,374
neuml/txtai

A Single Framework Combining Vector Search, Graphs and LLM Agents — txtai bundles vector indexes, graph networks and relational databases into one embeddings database that powers semantic search, RAG and multi-model workflows. Rather than stitching together separate vector DB, graph and LLM tools, developers get a unified foundation for building autonomous agents and search-driven AI applications in Python.

Python🇺🇸 North AmericaAttention 47
★13,656
FareedKhan-dev/train-llm-from-scratch

A from-scratch LLM pipeline covering pretraining through RLHF — A full PyTorch-only path from raw text to an aligned, reasoning-capable model, with no trl, peft or transformers dependencies. Covers both a classic transformer and a modern architecture (RoPE, RMSNorm, MoE, KV cache), plus hand-written SFT, DPO, GRPO, LoRA and speculative decoding. Aimed at learners wanting to train LLMs from scratch on a single GPU or even CPU.

Python🌐 ElsewhereAttention 38
★13,278
openvinotoolkit/openvino

Intel's inference toolkit for deploying models from PyTorch, TensorFlow, ONNX and more — OpenVINO optimizes and runs deep learning and generative AI models (LLMs, diffusion, vision, speech) trained in PyTorch, TensorFlow, ONNX, Keras or JAX/Flax, with Hugging Face integration via Optimum Intel. It targets CPU, integrated/discrete GPU and NPU across edge-to-cloud deployments, useful for developers needing framework flexibility without original training frameworks at inference time.

C++Attention 76
★11,595
huggingface/tokenizers

tokenizers heads toward a v1.0.0 release candidate — The Rust-based tokenization engine behind many transformer models is moving from 0.23 to a v1.0.0 release candidate, with opt-in training and Hub-download features to trim binary size. Useful for NLP engineers who need fast, production-grade tokenization across languages and hardware.

Rust🇪🇺 EuropeAttention 4
★11,259
MakazhanAlpamys/Soup

One-command LLM fine-tuning that fits an 8B model into 4GB VRAM — Soup is a CLI that fine-tunes and post-trains LLMs from a single YAML config, using layer streaming to run 8B-parameter training on a consumer laptop GPU with just 4GB VRAM. It targets developers who want LoRA/QLoRA/DPO/SFT workflows without SSH setup or config complexity, with exports to GGUF and Ollama.

Python🇺🇸 North AmericaAttention 39
★10,659
Rizzo-AI-Academy/rizzo-pii

A small local model to anonymize Italian legal text before LLM use — rizzo-pii runs a 0.3B-parameter model fully offline (no API key, ~0.5GB RAM) to detect and reversibly mask 22 PII categories in Italian legal documents, reporting 0.989 micro-F1. It targets lawyers and legal-tech teams who need GDPR/EU AI Act-aligned anonymization before sending text to frontier LLMs, with ready installers for Windows, macOS and Linux.

PythonAttention 40
★1,283
SheepSeb/tinyworldformer

A Looped Transformer World Model That Dreams Games on a $6 Pico 2W — Two tiny neural nets (26k-131k params) run entirely on a Raspberry Pi Pico 2W: one generates game frames via a looped, weight-reused transformer, the other plays using the generated frames via streaming RL. Interesting for anyone exploring microcontroller-scale generative world models or looped transformer architectures.

Rust🇪🇺 EuropeAttention 34
★375