nlp

huggingface/transformers

A unified model-definition framework spanning text, vision and audio — Transformers acts as the shared definition layer for models like DeepSeek, Gemma, GLM and Qwen, supporting both inference and training across text, vision, audio and multimodal tasks. It suits researchers and engineers who want one consistent API and access to pretrained checkpoints from the Hub instead of per-architecture code.

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

rohitg00/ai-engineering-from-scratch

A 542-lesson curriculum for building AI systems from the ground up — A free, MIT-licensed course with 542 lessons across 20 phases covering model internals, retrieval pipelines, and agent runtimes in Python, TypeScript, Rust, and Julia. Learners write code, run evals, and inspect failures, with labs and project challenges for self-paced or agent-assisted study.

Learn AI Engineering! Learn it. Build it. Ship it for others.

google/langextract

Google's library for grounding LLM extractions in source text — LangExtract uses LLMs like Gemini or OpenAI models to pull structured data from unstructured text, mapping every extraction back to its exact character span in the source. It supports cloud APIs, local Ollama models, and custom providers, with interactive HTML visualization for verifying results on documents like clinical notes or radiology reports.

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

NandhaKishorM/laya

A 33ms non-autoregressive decider for typed yes/no calls — Laya runs typed choice, score, and yes/no decisions over text in a single forward pass (33ms), across 100+ languages, trained with RL against strictly proper scoring rules. It ships a router that auto-selects the right checkpoint, handles documents up to 8,192 tokens, and integrates with LangChain, LlamaIndex, CrewAI, and MCP — useful for agent pipelines needing fast, calibrated System-1 decisions instead of LLM calls.

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

huggingface/tokenizers

tokenizers heads toward a v1.0.0 release candidate — The Rust-based tokenization engine behind many transformer models is moving from 0.23 to a v1.0.0 release candidate, with opt-in training and Hub-download features to trim binary size. Useful for NLP engineers who need fast, production-grade tokenization across languages and hardware.

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

epoko77-ai/im-not-ai

AI가 쓴 한글을 사람 글처럼 윤문하는 Claude 스킬 — Korean AI-text humanizer: detects and rewrites translationese, mechanical parallelism, and 71 other AI tells

nanaism/yomiyasu

An Agent Skill that cleans up AI-sounding Japanese prose — A Claude Code/Codex/Cursor skill that fixes the structural tics of AI-generated Japanese—vague metaphors, unclear subject-predicate links, inanimate subjects—rather than just banning buzzwords. Aimed at writers of technical docs, specs, PR descriptions, and internal reports who need naturally readable Japanese.

AI生成の日本語を自然な日本語へ推敲するAgent Skill / Agent Skill for Refining AI-Generated Japanese into Natural Japanese

coji/natural-japanese

An Agent Skill that strips AI stiffness out of Japanese work writing — This Claude/Codex Agent Skill rewrites AI-generated Japanese reports, minutes, and PR descriptions into natural phrasing without adding facts or changing certainty levels. Tests across 25 documents show bold-text overuse dropping from 551 to 13 instances, aimed at writers polishing AI drafts.

仕事の日本語を、読みやすくわかりやすく書く・直すための Agent Skill です。

pulse-energy-eu/slopshape

Verification package for detecting AI-written B2B blog posts — This release accompanies a paper replicating StoryScope's structural-feature detection method on commercial B2B blog content instead of fiction. It bundles code, prompts, and verification artifacts to reproduce 97.0 macro-F1 results on 2,250 human posts paired with AI mirrors from five generator models, useful for NLP researchers auditing AI-text detection claims.

Release package for SlopShape: Identifying AI-Generated Commercial Web Content (verification artifacts, instrument, prompts, code)

0227lia/data-science-portfolio

A student portfolio that keeps data science claims honest — This Astro/TypeScript static site presents six reproducible public-data case studies spanning data engineering, geospatial analysis, forecasting, operations research, NLP and ML. It explicitly separates observed results from modeled scenarios and sources all displayed metrics from committed runs in linked repos, useful for anyone evaluating a data science student's work critically.

Six reproducible public-data case studies in data engineering, geospatial analytics, forecasting, operations research, NLP and ML.