GitHub Python Monthly Trending Repositories

HKUDS/DeepTutor

DeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.

DeepTutor: Lifelong Personalized Tutoring

Features · Get Started · Explore · CLI · Ecosystem · Community

🤝 We welcome any kinds of contributing! Vote on roadmap items or propose new ones at Roadmap, and see our Contributing Guide for branching strategy, coding standards, and how to get started.

📦 Releases

[2026.8.10] v1.5.11 — Prose around a DSML tool call stops vanishing, a truncated reply continues instead of ending, live memory usage in Settings, and LightRAG indexing off the event loop.

[2026.8.7] v1.5.10 — Every account signs in to its own Codex, model output language becomes its own setting, empty tool calls are rejected instead of retried, and uploads stop blocking the loop.

[2026.8.4] v1.5.9 — Gemini Embedding 2 on its native endpoint, a per-model reasoning effort control, a Novita AI gateway, retrieval roles for queries, and Compose deployments that keep all of data/.

Past releases (more than 1 week ago)

[2026.8.2] v1.5.8 — Memory: a real heap ceiling for the dev server, source installs serve a production build, bounded LLM client and index caches, and a keep-alive fix for stray 500s.

[2026.7.31] v1.5.7 — A per-account MCP Services store, 101 CLI Apps the tutor can run, credentials moved out of the sandbox's reach, and a mobile layout.

[2026.7.29] v1.5.6 — Remote Codex sign-in completes behind an SSH tunnel, generated files get their own card in Activity, non-English languages stop collapsing to Chinese, and book creation no longer times out.

[2026.7.26] v1.5.5 — Sign in with your ChatGPT plan via OpenAI Codex OAuth, an Eden AI provider, knowledge bases that report what they hold, traceable rag citations, and GraphRAG indexing without a workaround.

[2026.7.24] v1.5.4 — Maintenance sweep: the post-answer "generating" stall is gone, IM partners render Markdown tables faithfully, LLM JSON parsing is sturdier, plus quiz, create-KB form, and Math Animator fixes.

[2026.7.24] v1.5.3 — Themeable code blocks, four more coding CLIs in My Agents (Gemini, Kimi, opencode, MiMo), an Atlas Cloud LLM provider, and a broad chat, memory, embedding, and parsing reliability sweep.

[2026.7.19] v1.5.2 — Configurable chat attachment limits, PageIndex retrieval that reasons across your documents via agentic tool calls, broader Anthropic/OpenAI model support, and steadier Book, Knowledge Base, and chat UI.

[2026.7.9] v1.5.1 — Remove a single failed document from a knowledge base — even one stuck in an error state — instead of deleting and rebuilding the whole base.

[2026.7.4] v1.5.0 — LlamaIndex ingestion now honors your Document Parsing engine with multimodal image extraction, Partner & Soul ids stay URL-safe for non-Latin names, and optional RAG extras install cleanly on Python 3.14+.

[2026.6.30] v1.4.15 — A native Mattermost channel for Partners, plus fixes so Guided Learning multiple-choice questions grade correctly and a configured zero chunk overlap is honored.

[2026.6.29] v1.4.14 — Click an assigned partner to chat in one step, Deep Research flags partial reports, LightRAG indexes without MinerU, FAISS handles non-ASCII paths, and PocketBase sessions are isolated per user.

[2026.6.27] v1.4.13 — Partners support non-Latin names and become assignable to users, logos render after login (#599), tiny knowledge bases retrieve reliably, and containers start cleanly under rootless Podman.

[2026.6.24] v1.4.12 — A new LightRAG Server retrieval engine, a lightweight PyMuPDF4LLM parsing engine, and a FAISS vector backend that makes large knowledge-base retrieval dramatically faster.

[2026.6.23] v1.4.11 — Native tool calling on every cloud OpenAI-compatible provider, a redesigned admin Users page, LaTeX in quiz options, an honest session-loading spinner, and configurable container host binding.

[2026.6.21] v1.4.10 — A self-service Profile page with avatars, a rootless-ready container guide with a single-port request-time proxy, and deny-by-default MCP tools for non-admin users.

[2026.6.19] v1.4.9 — Settings polish: Search shows only the fields your provider needs, connection profiles can be renamed and auto-named by provider, and graded Mastery Path questions flow into your Question Bank.

[2026.6.18] v1.4.8 — Connect your own Partners under My Agents and consult them live in chat — answering through their own persona, library and skills — each with its own private memory.

[2026.6.18] v1.4.7 — Connect your local Claude Code / Codex and consult it live mid-turn, My Agents graduates to a top-level /agents, and Partner conversations gain branch / resume / delete with a replayable trace.

[2026.6.17] v1.4.6 — Four-surface consolidation: a Space learning dashboard with importable My Agents and top-level Memory, a Knowledge Center with GraphRAG / PageIndex / LightRAG / linked-KB / Obsidian, opened-up Settings, and per-model capability gating.

[2026.6.14] v1.4.5 — Guided Learning rebuilt on the chat agent loop with a hard per-type mastery gate and a /learning dashboard, a new loop-plugin framework, plus Markdown export / save-to-notebook for Partner conversations.

[2026.6.13] v1.4.4 — Install community skills from ClawHub with deeptutor skill install behind a security gate, plus real in-browser DOCX/XLSX previews for knowledge-base files.

[2026.6.12] v1.4.3 — TutorBot becomes Partners on a production-grade IM pipeline (15 channels, live streaming), Chat moves to a single agent loop, real per-user isolation, and a rebuilt Visualize.

[2026.5.28] v1.4.2 — Stability + polish: Gemini 2.5+ unblocked across Visualize and Chat, auth-routing fix (#485), smooth-streaming chat UX, a Recents sidebar, and Lemonade local-provider support.

[2026.5.27] v1.4.1 — Security + stability: TutorBot tool sandbox locked down, per-user resource isolation, multimodal image fallback, an HTTP/SSE API for TutorBots, and a v1.4.0 chat regression fix.

[2026.5.22] v1.4.0 — GA cut of v1.4: Auto Mode, three-layer Memory, agentic Deep Research / Solve / Question, LlamaIndex RAG refactor, Visualize/Animator merge, and restart-safe turn runtime.

[2026.5.21] v1.4.0-beta — Three-layer Memory workbench (L1/L2/L3), every chat capability rebuilt on a single agentic engine, LlamaIndex-only RAG, and a unified Settings + Capabilities surface.

[2026.5.10] v1.3.10 — Remote Docker CORS recovery, DISABLE_SSL_VERIFY across SDK providers, safer code-block citations, and optional Matrix E2EE add-on.

[2026.5.9] v1.3.9 — TutorBot Zulip and NVIDIA NIM support, safer thinking-model routing, deeptutor start, sidebar tooltips, and session-store parity.

[2026.5.8] v1.3.8 — Optional multi-user deployments with isolated user workspaces, admin grants, auth routes, and scoped runtime access.

[2026.5.4] v1.3.7 — Thinking-model/provider fixes, visible Knowledge index history, and safer Co-Writer clear/template editing.

[2026.5.3] v1.3.6 — Catalog-based model selection for chat and TutorBot, safer RAG re-indexing, OpenAI Responses token-limit fixes, and Skills editor validation.

[2026.5.2] v1.3.5 — Smoother local launch settings, safer RAG queries, cleaner local embedding auth, and Settings dark-mode polish.

[2026.5.1] v1.3.4 — Book page chat persistence and rebuild flows, chat-to-book references, stronger language/reasoning handling, RAG document extraction hardening.

[2026.4.30] v1.3.3 — NVIDIA NIM + Gemini embedding support, unified Space context for chat history/skills/memory, session snapshots, RAG re-index resilience.

[2026.4.29] v1.3.2 — Transparent embedding endpoint URLs, RAG re-index resilience for invalid persisted vectors, memory cleanup for thinking-model output, Deep Solve runtime fix.

[2026.4.28] v1.3.1 — Stability: safer RAG routing & embedding validation, Docker persistence, IME-safe input, Windows/GBK robustness.

[2026.4.27] v1.3.0 — Versioned KB indexes with re-index workflow, rebuilt Knowledge workspace, embedding auto-discovery with new adapters, Space hub.

[2026.4.25] v1.2.5 — Persistent chat attachments with file-preview drawer, attachment-aware capability pipelines, TutorBot Markdown export.

[2026.4.25] v1.2.4 — Text/code/SVG attachments, one-command Setup Tour, Markdown chat export, compact KB management UI.

[2026.4.24] v1.2.3 — Document attachments (PDF/DOCX/XLSX/PPTX), reasoning thinking-block display, Soul template editor, Co-Writer save-to-notebook.

[2026.4.22] v1.2.2 — User-authored Skills system, chat input performance overhaul, TutorBot auto-start, Book Library UI, visualization fullscreen.

[2026.4.21] v1.2.1 — Per-stage token limits, Regenerate response across all entry points, RAG & Gemma compatibility fixes.

[2026.4.20] v1.2.0 — Book Engine "living book" compiler, multi-document Co-Writer, interactive HTML visualizations, Question Bank @-mention.

[2026.4.18] v1.1.2 — Schema-driven Channels tab, RAG single-pipeline consolidation, externalized chat prompts.

[2026.4.17] v1.1.1 — Universal "Answer now", Co-Writer scroll sync, unified settings panel, streaming Stop button.

[2026.4.15] v1.1.0 — LaTeX block math overhaul, LLM diagnostic probe, Docker + local LLM guidance.

[2026.4.14] v1.1.0-beta — Bookmarkable sessions, Snow theme, WebSocket heartbeat & auto-reconnect, embedding registry overhaul.

[2026.4.13] v1.0.3 — Question Notebook with bookmarks & categories, Mermaid in Visualize, embedding mismatch detection, Qwen/vLLM compatibility, LM Studio & llama.cpp support, and Glass theme.

[2026.4.11] v1.0.2 — Search consolidation with SearXNG fallback, provider switch fix, and frontend resource leak fixes.

[2026.4.10] v1.0.1 — Visualize capability (Chart.js/SVG), quiz duplicate prevention, and o4-mini model support.

[2026.4.10] v1.0.0-beta.4 — Embedding progress tracking with rate-limit retry, cross-platform dependency fixes, and MIME validation fix.

[2026.4.8] v1.0.0-beta.3 — Native OpenAI/Anthropic SDK (drop litellm), Windows Math Animator support, robust JSON parsing, and full Chinese i18n.

[2026.4.7] v1.0.0-beta.2 — Hot settings reload, MinerU nested output, WebSocket fix, and Python 3.11+ minimum.

[2026.4.4] v1.0.0-beta.1 — Agent-native architecture rewrite (~200k lines): Tools + Capabilities plugin model, CLI & SDK, TutorBot, Co-Writer, Guided Learning, and persistent memory.

[2026.1.23] v0.6.0 — Session persistence, incremental document upload, flexible RAG pipeline import, and full Chinese localization.

[2026.1.18] v0.5.2 — Docling support for RAG-Anything, logging system optimization, and bug fixes.

[2026.1.15] v0.5.0 — Unified service configuration, RAG pipeline selection per knowledge base, question generation overhaul, and sidebar customization.

[2026.1.9] v0.4.0 — Multi-provider LLM & embedding support, new home page, RAG module decoupling, and environment variable refactor.

[2026.1.5] v0.3.0 — Unified PromptManager architecture, GitHub Actions CI/CD, and pre-built Docker images on GHCR.

[2026.1.2] v0.2.0 — Docker deployment, Next.js 16 & React 19 upgrade, WebSocket security hardening, and critical vulnerability fixes.

📰 News

2026-05-22 🌐 Official docs site live at deeptutor.info — guides, references, and capability tours in one place.
2026-04-19 🎉 20k stars in 111 days! Thank you for the support toward truly personalized, intelligent tutoring.
2026-04-10 📄 Our paper is live on arXiv — read the preprint for the design and ideas behind DeepTutor.
2026-02-06 🚀 10k stars in just 39 days! A huge thank you to our incredible community.
2026-01-01 🎊 Happy New Year! Join our Discord, WeChat, or Discussions — let's shape DeepTutor together.
2025-12-29 🎓 DeepTutor is officially released!

✨ Key Features

DeepTutor is an agent-native learning workspace that connects tutoring, problem solving, quiz generation, research, visualization, and mastery practice in one extensible system.

One runtime for every mode — Chat, Quiz, Research, Visualize, Solve, and Mastery Path run on the same agent loop, so you switch the objective, not the engine, and context moves with the learner.
Connected learning context — Knowledge bases, books, Co-Writer drafts, notebooks, question banks, personas, and Memory stay available across every workflow instead of living in isolated tools.
Subagents and Partners — consult a live coding CLI (Claude Code, Codex, Gemini, Kimi, opencode, or MiMo) or a Partner from any turn (or import their past conversations), and run persistent IM companions on the same brain.
Multi-engine knowledge — versioned RAG libraries across LlamaIndex, PageIndex, GraphRAG, LightRAG, or a linked Obsidian vault, with pluggable document parsing.
Extensible tools and skills — built-in tools, MCP servers, CLI apps, image / video / voice generation models, and installable community skills from EduHub.
Inspectable memory — L1 traces, L2 surface summaries, and L3 synthesis make personalization visible and editable, with a Memory Graph that traces every claim back to its evidence.

🚀 Get Started

DeepTutor ships four installation paths. They all share one workspace layout: settings live in data/user/settings/ under the directory you launch from (or under DEEPTUTOR_HOME / deeptutor start --home if you set one explicitly). For the full app, the recommended flow is pick a workspace directory → install → deeptutor init → deeptutor start.

Option 1 — Install From PyPI · full local Web app + CLI, no clone required

Full local Web app + CLI, no clone required. Needs Python 3.11–3.13 and a Node.js 20+ runtime on PATH (the packaged Next.js standalone server is spawned by deeptutor start).

mkdir -p my-deeptutor && cd my-deeptutor
pip install -U deeptutor
deeptutor init     # prompts for ports + LLM provider + optional embedding
deeptutor start    # starts backend + frontend; keep the terminal open

deeptutor init prompts for backend port (default 8001), frontend port (default 3782), LLM provider / base URL / API key / model, and an optional embedding provider for Knowledge Base / RAG.

After deeptutor start, open the frontend URL printed in the terminal — by default http://127.0.0.1:3782. Press Ctrl+C in that terminal to stop both backend and frontend. Skipping deeptutor init is fine for a quick trial; the app boots with default ports and empty model settings, configure them later in Settings → Models.

Option 2 — Install From Source · develop against a checkout

For development against a checkout. Use Python 3.11–3.13 and Node.js 22 LTS to match CI and Docker.

git clone https://github.com/HKUDS/DeepTutor.git
cd DeepTutor

# Create a venv (macOS/Linux). Windows PowerShell:
#   py -3.11 -m venv .venv ; .\.venv\Scripts\Activate.ps1
python3 -m venv .venv && source .venv/bin/activate
python -m pip install --upgrade pip

# Install backend + frontend deps
python -m pip install -e .
( cd web && npm ci --legacy-peer-deps )

deeptutor init
deeptutor start --dev

deeptutor start builds the local web/ frontend for production once and reuses it; --dev runs Next.js with HMR. Config layout, ports, and Ctrl+C match Option 1.

Conda environment (instead of venv)

conda create -n deeptutor python=3.11
conda activate deeptutor
python -m pip install --upgrade pip

Optional install extras — dev / partners / matrix / math-animator

pip install -e ".[dev]"             # tests/lint tools
pip install -e ".[partners]"        # Partner IM channel SDKs + MCP client
pip install -e ".[matrix]"          # Matrix channel without E2EE/libolm
pip install -e ".[matrix-e2e]"      # Matrix E2EE; requires libolm
pip install -e ".[math-animator]"   # Manim addon; requires LaTeX/ffmpeg/system libs

Frontend dependency tweaks & dev-server troubleshooting

Changing frontend dependencies: run npm install --legacy-peer-deps to refresh web/package-lock.json, then commit both web/package.json and web/package-lock.json.

Stuck dev server: if deeptutor start --dev reports an existing frontend that isn't responding, stop the PID it prints. If no Next.js process is actually running, the lock files are stale — remove them and retry:

rm -f web/.next/dev/lock web/.next/lock
deeptutor start --dev

Option 3 — Docker · one self-contained container

One container for the full Web app. Images on GitHub Container Registry:

ghcr.io/hkuds/deeptutor:latest — stable release
ghcr.io/hkuds/deeptutor:pre — pre-release, when available

See CONTAINERIZATION.md for podman/rootless/read-only-rootfs deployments and the full per-installation guide.

docker run --rm --name deeptutor \
  -p 127.0.0.1:3782:3782 \
  -v deeptutor-data:/app/data \
  ghcr.io/hkuds/deeptutor:latest

Only 3782 needs to be published. The browser talks exclusively to the frontend origin; the Next.js middleware (web/proxy.ts) forwards /api/* and /ws/* to the FastAPI backend inside the container. Publishing 8001 (-p 127.0.0.1:8001:8001) is optional — handy only for hitting the API directly with curl or scripts.

Open http://127.0.0.1:3782. The container creates /app/data/user/settings/*.json on first boot; configure model providers from the Web Settings page. Config, API keys, logs, workspace files, memory, and knowledge bases persist in the deeptutor-data volume.

Different host ports: change the left side of each -p host:container mapping (e.g. -p 127.0.0.1:8088:3782). If you change container-side ports in /app/data/user/settings/system.json, restart and update the right side of each mapping to match.
Detached: add -d, then docker logs -f deeptutor to follow, docker stop deeptutor to stop, docker rm deeptutor before reusing the name. The deeptutor-data volume keeps your settings and workspace across restarts.

Remote Docker / reverse proxy: the browser only talks to the frontend origin (:3782); the in-container Next.js middleware forwards /api/* and /ws/* to the backend server-side. For the common single-container case you don't configure an API base at all — just point your reverse proxy / TLS terminator at :3782. You only need an API base for a split deployment (backend in a separate container/host): set next_public_api_base in data/user/settings/system.json to the in-network address the frontend server uses to reach the backend (it's read server-side, never sent to the browser).

{
  "next_public_api_base": "http://backend:8001"
}

next_public_api_base_external (and its alias public_api_base) are accepted as lower-precedence fallbacks. CORS uses frontend origins, not API URLs. With auth disabled, DeepTutor permits normal HTTP/HTTPS browser origins by default. With auth enabled, add exact frontend origins:

{
  "cors_origins": ["https://deeptutor.example.com"]
}

Connecting to Ollama / LM Studio / llama.cpp / vLLM / Lemonade on the host

Inside Docker, localhost is the container itself, not your host machine. To reach a model service running on the host, use the host gateway (recommended):

docker run --rm --name deeptutor \
  -p 127.0.0.1:3782:3782 -p 127.0.0.1:8001:8001 \
  --add-host=host.docker.internal:host-gateway \
  -v deeptutor-data:/app/data \
  ghcr.io/hkuds/deeptutor:latest

Then in Settings → Models, point the provider Base URL at host.docker.internal:

Ollama LLM: http://host.docker.internal:11434/v1
Ollama embedding: http://host.docker.internal:11434/api/embed
LM Studio: http://host.docker.internal:1234/v1
llama.cpp: http://host.docker.internal:8080/v1
Lemonade: http://host.docker.internal:13305/api/v1

Docker Desktop (macOS/Windows) usually resolves host.docker.internal without --add-host. On Linux, the flag is the portable way to create that hostname on modern Docker Engine.

Linux alternative — host networking: add --network=host and drop the -p flags. The container shares the host network directly, so open http://127.0.0.1:3782 (or the frontend_port in system.json), and host services can be reached with normal localhost URLs like http://127.0.0.1:11434/v1. Note that host networking exposes container ports directly on the host and may conflict with existing services — to keep them on loopback, set BACKEND_HOST=127.0.0.1 and FRONTEND_HOST=127.0.0.1 (see CONTAINERIZATION.md).

Option 4 — CLI Only · no Web UI, from a source checkout

When you don't need the Web UI. The CLI-only package is installed from a source checkout, not from PyPI.

git clone https://github.com/HKUDS/DeepTutor.git
cd DeepTutor

# Create a venv (macOS/Linux). Windows PowerShell:
#   py -3.11 -m venv .venv-cli ; .\.venv-cli\Scripts\Activate.ps1
python3 -m venv .venv-cli && source .venv-cli/bin/activate
python -m pip install --upgrade pip

python -m pip install -e ./packaging/deeptutor-cli
deeptutor init --cli
deeptutor chat

deeptutor init --cli shares the same data/user/settings/ layout as the full app but skips the backend/frontend port prompts and defaults embeddings to off (choose Yes if you plan to use deeptutor kb … or RAG tools). It still writes a complete runtime layout (system.json, auth.json, integrations.json, model_catalog.json, main.yaml, agents.yaml) and still prompts for the active LLM provider and model.

Common commands

deeptutor chat                                          # interactive REPL
deeptutor chat --capability deep_solve --tool rag --kb my-kb
deeptutor run chat "Explain Fourier transform"
deeptutor run deep_solve "Solve x^2 = 4" --tool rag --kb my-kb
deeptutor kb create my-kb --doc textbook.pdf
deeptutor memory show
deeptutor config show

The local deeptutor-cli install ships no Web assets or server dependencies. Keep the source checkout around — the editable install points to it. To add the Web app later, install the PyPI package (Option 1) and run deeptutor init + deeptutor start from the same workspace.

Code Execution Sandbox (office skills) · running model-generated code for docx / pdf / pptx / xlsx

The built-in office skills — docx / pdf / pptx / xlsx — work by having the model write a short Python script (python-docx, reportlab, openpyxl, …), run it through the exec / code_execution tools, and hand back a download URL. Those tools mount whenever a sandbox backend is active, which it is by default in every deployment shape:

Local (Option 1 / 2) and Docker (Option 3, single container): a restricted subprocess sandbox runs the model's code (on the host locally, or inside the container under Docker — the container being its own isolation boundary).
docker-compose: routed instead to a hardened, least-privileged runner sidecar (Dockerfile.runner) via DEEPTUTOR_SANDBOX_RUNNER_URL — the strongest posture, and preferred automatically when present.

The subprocess sandbox is controlled by the sandbox_allow_subprocess setting in data/user/settings/system.json (default true). Running model-generated code on your host is a real trust decision — set it to false (or export DEEPTUTOR_SANDBOX_ALLOW_SUBPROCESS=0) to disable host-side execution, at the cost of the office skills no longer being able to produce files.

Configuration reference — config files under data/user/settings/ (JSON/YAML)

Everything under data/user/settings/ is plain JSON/YAML. The Settings page in the browser is the recommended editor.

File	Purpose
`model_catalog.json`	LLM, embedding, and search provider profiles; API keys; active models
`system.json`	Backend/frontend ports, public API base, CORS, SSL verification, attachment directory and upload/extraction limits
`auth.json`	Optional auth toggle, username, password hash, token/cookie settings
`integrations.json`	Optional PocketBase and sidecar integration settings
`interface.json`	UI and model output language / theme / sidebar preferences
`main.yaml`	Runtime behavior defaults and path injection
`agents.yaml`	Capability/tool temperature and token settings

Project-root .env is not read as an application config file. For a minimal model setup, open Settings → Models, add an LLM profile (Base URL / API key / model name), and save. Add an embedding profile only if you plan to use Knowledge Base / RAG features.

📖 Explore DeepTutor

Start with the main surfaces you will use day to day: Chat, Partners, My Agents, Co-Writer, Book, Knowledge Center, Learning Space, Memory, and Settings. The tour then covers Multi-User deployments for shared, isolated workspaces.

🏗️ System architecture

💬 Chat — The Agent Loop You Actually Use

Chat is the default capability and where most work begins. A single thread can talk normally, call tools, ground itself in selected knowledge bases, read attachments, generate images, consult subagents, write notebook records, and continue with the same context across turns.

The loop is deliberately simple: the model thinks in rounds, calls tools when useful, observes the results, and finishes with a tool-free message. ask_user is special — instead of guessing, the agent can pause the turn, ask a structured clarifying question, and resume once you answer.

User-toggleable tools are brainstorm, web_search, paper_search, reason, and geogebra_analysis — plus imagegen and videogen once you configure the matching generation model. Contextual tools such as rag, kb_files, read_source, read_memory, write_memory, read_skill, load_tools, exec, web_fetch, ask_user, list_notebook, write_note, github, and consult_subagent mount automatically when the turn has the right context.

Context comes in two kinds: sticky session context (subagent, knowledge bases, persona, model, voice) lives on the composer toolbar and persists across turns; one-time references (files, chat history, books, notebooks, question bank, imported agents) come from the + menu for a single turn.

Chat is also the launch point for deeper capabilities: Quiz for question generation, Research for cited reports, Visualize for charts / diagrams / animations, and — under More Capabilities — Solve for worked reasoning and Mastery Path for learning-plan flows.

🤝 Partner — Persistent Companions on the Same Brain

Partners are persistent companions with their own soul, model policy, library, memory, and channels. They are not a separate bot engine: every inbound web or IM message becomes a normal ChatOrchestrator turn inside a partner-scoped workspace. A partner is "a chat that has a personality and a phone number."

Each partner has a SOUL.md, model selection, channels, tool policy, and assigned library. Knowledge bases, skills, and notebooks are copied into data/partners/<id>/workspace/, so the same RAG, skill, notebook, and memory tools work without special cases. A partner reads its owner's memory but writes only its own.

The channel layer is schema-driven and can connect to IM platforms such as Feishu, Telegram, Slack, Discord, DingTalk, QQ/NapCat, WeCom, WhatsApp, Zulip, Mattermost, Matrix, Mochat, and Microsoft Teams depending on installed extras and configured credentials. A partner can also be connected as a subagent and consulted from a normal chat turn — see My Agents below.

🧑‍🚀 My Agents — Consult & Import Other Agents

My Agents turns other agents into context for DeepTutor, and does two distinct things. Connect a live agent — a Claude Code, Codex, Gemini, Kimi, opencode, or MiMo Code CLI on your machine, or one of your Partners — and consult it from inside a chat turn: DeepTutor actually runs the other agent and streams its work into the Activity panel via the consult_subagent tool. Select it with the Agent chip (or type @), and set how many rounds the consult may take.

Import past conversations — bring in your existing Claude Code and Codex history as named, searchable, resumable agents. Pick which days to import; refreshing re-syncs them. Reference an imported conversation from any chat turn via + → My Agents, and DeepTutor reads it as a third-party transcript — it stays their conversation, not DeepTutor's own voice.

✍️ Co-Writer — Selection-Aware Markdown Drafting

Co-Writer is a split-view Markdown workspace for reports, tutorials, notes, and long-form learning artifacts. Documents autosave and render a live preview (KaTeX math, diagram fences), and can be saved back into notebooks when a draft becomes reusable context.

Its defining idea is surgical editing: select a span and ask DeepTutor to rewrite, expand, or shorten it. The edit agent can ground the change in a knowledge base or web evidence, keeps a trace of its tool calls, and shows every change as an accept/reject diff — so nothing lands until you approve it.

📖 Book — Living Books from Your Materials

Book turns selected sources into an interactive living book — not a static PDF, but a reading environment built from typed blocks. A book can start from knowledge bases, notebooks, question banks, or chat history; the creation flow proposes a chapter outline before content is generated, so you review the shape instead of accepting a blind one-shot output.

Each chapter compiles into typed blocks — text, callouts, quizzes, flash cards, timelines, code, figures, interactive HTML, animations, concept graphs, deep dives, and user notes — and every page has its own Page Chat. Blocks are editable: insert, move, regenerate, or switch a block's type without rewriting the chapter. Maintenance commands such as deeptutor book health and deeptutor book refresh-fingerprints help detect when source knowledge has drifted from compiled pages.

📚 Knowledge Center — Multi-Engine RAG Libraries

Knowledge bases are the document collections behind RAG — they ground Chat turns, Co-Writer edits, Book generation, and Partner conversations. What's distinctive is a choice of retrieval engines: LlamaIndex (the default, local vector + BM25), PageIndex (hosted, reasoning retrieval with page-level citations), GraphRAG and LightRAG (knowledge-graph retrieval), LightRAG Server (retrieval offloaded to an external LightRAG instance you connect over HTTP), Tencent IMA (a library you curate in IMA, searched over its OpenAPI), or a linked Obsidian vault the tutor reads and writes in place. Each KB is bound to one engine.

Creating a KB, you either create new (upload documents and build a fresh index) or link existing (reuse an index built elsewhere, read in place with no re-index). Re-indexing writes a new flat version-N directory and keeps prior ones, so a working index is never destroyed mid-rebuild. A single document can be removed even from an error-state base — dropping a file that failed to parse without a full delete-and-rebuild. Document parsing — Text-only, MinerU, Docling, markitdown, or PyMuPDF4LLM — is chosen in Settings → Knowledge Base, with local model downloads off by default. The CLI mirrors the lifecycle with deeptutor kb list, info, create, add, search, set-default, and delete.

🌐 Learning Space — Skills, Personas, and Reusable Context

Learning Space is the library and personalization layer — where the things that persist live. Conversations & Materials holds your chat history, notebooks, and a question bank (each saved question keeps your answer, the reference answer, and an explanation). Personalization holds mastery paths, personas (behavior presets such as peer, research-assistant, teacher), skills (SKILL.md playbooks the model reads on demand), MCP Services — a curated store of hosted MCP servers you install for yourself in one click, plus any remote server you configure by URL — and CLI Apps, command-line tools from the CLI-Anything catalog that the chat agent calls directly, with each app's own usage guide loaded on demand. Everything here can be reused from Chat, Partners, Co-Writer, and Book.

You don't have to write every skill yourself — Import from EduHub browses the community catalog and downloads a skill straight into your library through a security gate (see Ecosystem).

🧠 Memory — Inspectable Personalization

Memory is a file-backed, three-layer system you can read, curate, and audit — deliberately not a hidden vector store. L1 is the workspace mirror plus an append-only event trace (trace/<surface>/<date>.jsonl); L2 is per-surface curated facts (L2/<surface>.md); L3 is cross-surface synthesis (L3/<profile|recent|scope|preferences>.md). Because L2 cites L1 and L3 cites L2, nothing in your profile is unaccountable.

The Memory Graph shows the whole pyramid — L3 synthesis at the centre, L2 in the middle ring, L1 traces on the outside — so you can trace any synthesized claim back to the exact raw event behind it. Memory is tracked across chat, notebook, quiz, kb, book, partner, and cowriter surfaces; the consolidator's Update / Audit / Dedup budgets are tuned in Settings → Memory.

⚙️ Settings — One Control Plane

Settings is the operational control plane, with a live status strip (backend health and resident memory across the process tree) and one card per area: Appearance (theme, interface and model output language, code-block styling), Network (API base, ports, CORS), Models (LLM, Embedding, Search, Text-to-Speech, Speech-to-Text, Image Generation, Video Generation), Knowledge Base (document parsing engine), Chat (tools, per-capability parameters, attachment caps), Partners & Agents (the subagents you can consult from a turn), and Memory (the consolidator's budgets).

Most sections use a draft-and-apply flow, so you can test a provider before committing it. Four themes ship in the box — Default, Cream, Dark, and Glass. Project-root .env files are intentionally ignored; runtime configuration lives under data/user/settings/*.json unless DEEPTUTOR_HOME or deeptutor start --home points the app elsewhere.

OpenAI Codex OAuth (experimental). Picking OpenAI Codex under Models → LLM replaces the API-key fields with a browser sign-in that runs against your own ChatGPT plan, so no OPENAI_API_KEY is needed. Tokens live only in data/system/user-secrets/<owner>/private/openai-codex/ — in the multi-container Compose deployment, outside every tree the exec sandbox can reach — and DeepTutor never reads or modifies your ~/.codex CLI login. The model list comes from that account's live catalog; signing in publishes the profile but only becomes the active model when no LLM is configured yet. Because a token authorizes one person's plan, the profile is not shareable through user grants — each account signs in for itself, ordinary users included: their card sits under Models → LLM, and the resulting models, catalog, and sign-out stay private to that account.

Default local Docker and Podman deployments use separate loopback networks and need a temporary bridge during sign-in. Follow the temporary local Codex OAuth bridge guide for the exact Docker, Compose, Podman, and teardown commands.

For a remote deployment, the browser's localhost and the server's localhost are different machines, so an ordinary reverse proxy alone cannot carry the browser's localhost callback to the server. Use an SSH tunnel as the callback bridge. The tunnel reaches the already-published Web port; Next.js rewrites only the exact callback path to the public callback broker, and the broker validates state before routing to the original OAuth operation. The callback listener remains on the backend loopback, ports 1455 and 1457 are not published, and this path supports the default Docker bridge network.

ssh -N -L 1455:127.0.0.1:3782 <ssh-user>@<server-host>

If DeepTutor reports fallback callback port 1457, use:

ssh -N -L 1457:127.0.0.1:3782 <ssh-user>@<server-host>

Run only the one command that matches the actual callback port; never run both. 3782 is only the example Web port: it is the configured frontend/container port reported as callback_forward_port. That value does not guarantee that the same port is listening on the SSH host's 127.0.0.1. If Docker or Podman publishes a different host port, or a reverse proxy listens on a different port, replace only the right-hand target port (3782 above) with the Web port actually listening on the SSH host's 127.0.0.1; keep the left-hand callback port as 1455 or 1457. <server-host> is the SSH host whose loopback owns that listening port. If the browser URL names a reverse proxy or load balancer, replace it with the correct SSH frontend host.

The CLI prints the tunnel command and then immediately tries to open the browser. On a remote deployment, keep the authorization page open without completing it, establish the printed tunnel in another terminal, and only then continue authorization.

Remote-topology detection has a localhost boundary. If Web itself is reached through an SSH or IDE localhost forward, the browser cannot tell that the server is remote. For the current Web operation, leave its authorization page unfinished, read redirect_uri in that operation's authorize URL to identify callback port 1455 or 1457, and create the second tunnel from that local port to the actual Web port. Alternatively, cancel that Web operation and start a new one with the CLI; the CLI output belongs to the new operation and must not be used for the existing Web operation. Quota errors and catalog failures are reported as-is and never fall back to a paid provider. This compatibility path is experimental: the upstream interface may change.

👥 Multi-User — Shared Deployments · optional auth, isolated per-user workspaces

Authentication is off by default — DeepTutor runs single-user. Turn it on and one data/ tree hosts an admin workspace, isolated per-user workspaces, and partner workspaces side by side:

data/
├── user/                    # Admin workspace + global settings
├── users/<uid>/             # Per-user scope: chat history, memory, notebooks, KBs
├── partners/<id>/workspace/ # Partner (synthetic-user) scope
├── cli-apps/                # Installed CLI apps, mounted read-only into the sandbox
└── system/                  # auth · grants · audit · user-secrets/<owner> (OAuth tokens)

The first registered user becomes admin and owns model catalogs, provider credentials, shared knowledge bases, skills, and per-user grants. Everyone else gets an isolated workspace and a redacted Settings page — admin-assigned models, KBs, and skills show up as scoped, read-only options, never as raw API keys.

Enable it: turn auth on in data/user/settings/auth.json, restart deeptutor start, register the first admin at /register, then add users from /admin/users and assign models, KBs, skills, partners, tool/MCP/CLI-app policy, and code-execution access through grants.

PocketBase stays a single-user integration — keep integrations.pocketbase_url blank for multi-user deployments unless you've wired up an external user store.

⌨️ DeepTutor CLI — Agent-Native Interface

One deeptutor binary, two ways in: an interactive REPL for people who live in the terminal, and structured JSON for other agents that drive DeepTutor as a tool. Same capabilities, tools, and knowledge bases either way.

Drive it yourself

deeptutor chat opens an interactive REPL; deeptutor run <capability> "<message>" fires a single turn and exits. Both speak the same --capability, --tool, --kb, and --config flags.

deeptutor chat                                              # interactive REPL
deeptutor chat --capability deep_solve --kb my-kb --tool rag
deeptutor run chat "Explain the Fourier transform" --tool rag --kb textbook
deeptutor run deep_research "Survey 2026 papers on RAG" \
  --config mode=report --config depth=standard

Everything the Web app does is here too — knowledge bases (kb), sessions (session), partners (partner), skills (skill), notebooks, memory, and config. Full list below.

Let an agent drive it

DeepTutor is built to be operated by another agent. Add --format json to any run and each turn streams NDJSON — one event per line (content, tool_call, tool_result, done, …), every line tagged with its session_id. Runs are headless-safe: an ask_user pause with no TTY auto-resolves with an empty reply instead of hanging.

# One shot, machine-readable
deeptutor run deep_solve "Find d/dx[sin(x^2)]" --tool reason --format json

# Chain turns in one stateful session — capture the id, reuse it
SID=$(deeptutor run deep_research "Survey 2026 papers on RAG" \
  --config mode=report --config depth=standard --format json \
  | jq -r 'select(.type=="done").session_id')
deeptutor run deep_question "Quiz me on that survey" --session "$SID" --format json

The repo ships a root SKILL.md — a ~150-line handover doc that teaches any tool-using LLM the whole surface in one read. Hand it to Claude Code, Codex, or OpenCode (they pick up SKILL.md automatically), or wrap deeptutor run as a tool in a LangChain / AutoGen loop. Full recipes: Agent Handoff.

Command reference

Command	Description
`deeptutor init`	Create or update `data/user/settings` for the current workspace
`deeptutor start [--home PATH] [--dev]`	Launch backend + frontend together; `--dev` enables frontend HMR
`deeptutor serve [--port PORT]`	Start only the FastAPI backend
`deeptutor run <capability> <message>`	Run a single capability turn (`chat`, `deep_solve`, `deep_question`, `deep_research`, `visualize`, `math_animator`, `mastery_path`); add `--format json` for NDJSON output
`deeptutor chat`	Interactive REPL with capability, tool, KB, notebook, and history controls
`deeptutor partner list/create/start/stop`	Manage IM-connected partners
`deeptutor kb list/info/create/add/search/set-default/delete`	Manage LlamaIndex knowledge bases
`deeptutor skill search/install/list/remove/login/logout/publish/update`	Manage skills, install from hubs, and publish your own (`eduhub:<slug>` by default, see Ecosystem)
`deeptutor memory show/clear`	Inspect L2/L3 memory docs or clear L1/all memory
`deeptutor session list/show/open/rename/delete`	Manage shared sessions
`deeptutor notebook list/create/show/add-md/replace-md/remove-record`	Manage notebooks from Markdown files
`deeptutor book list/health/refresh-fingerprints`	Inspect books and refresh source fingerprints
`deeptutor plugin list/info`	Inspect registered tools and capabilities
`deeptutor config show`	Print configuration summary
`deeptutor provider login <provider>`	Provider auth (`openai-codex` OAuth login; `github-copilot` validates an existing Copilot auth session)

CLI-only distribution

The CLI-only package lives in packaging/deeptutor-cli. In this checkout, install it from source:

python -m pip install -e ./packaging/deeptutor-cli

It isn't published to PyPI yet, so the main Get Started section keeps the source-install path.

🧩 Ecosystem — EduHub & the Skills Community

DeepTutor skills use the open Agent-Skills format — a folder with a SKILL.md playbook (YAML frontmatter + Markdown) and optional reference files. Nothing about it is DeepTutor-specific, so any registry that speaks the format becomes a source for your library. DeepTutor ships with EduHub — our own education-focused skill registry — wired in as the default hub.

EduHub — DeepTutor's skill ecosystem

EduHub is the community hub DeepTutor launched for sharing teaching-oriented agent skills — Socratic tutors, flashcard builders, essay feedback, exam blueprints, concept explainers, and more. It is built into DeepTutor, so there's nothing to configure: a bare slug or an eduhub: prefix resolves to it.

Find and install — in the browser, open Learning Space → Skills → Import from EduHub to browse the catalog and download a skill straight into your library. From the terminal:

deeptutor skill search "socratic tutor"               # search EduHub (the default hub)
deeptutor skill install socratic-tutor                # fetch → verify → register
deeptutor skill install eduhub:socratic-tutor@1.2.0   # pin a hub and a version
deeptutor skill list                                  # local skills with their hub provenance

Publish your own — package a SKILL.md and share it back to the community:

deeptutor skill login                                 # browser sign-in to EduHub
deeptutor skill publish ./my-skill                    # interactive: pick a track + tags, then upload
deeptutor skill update                                # roll back or release a new version

EduHub is also a standalone, ClawHub-compatible registry, so agents that aren't DeepTutor (Claude Code, Codex, …) can use it directly through the eduhub CLI — npx eduhub install socratic-tutor.

The import safety gate

Whatever the source, every import passes the same safety gate before anything touches your workspace:

the registry's security verdict is checked first — flagged packages are refused unless you pass --allow-unverified;
archives are extracted defensively (zip-slip / zip-bomb guards) behind a text/script suffix whitelist, so binaries never land in the workspace;
frontmatter is normalized to DeepTutor's schema and always: is stripped, so a downloaded skill can never force itself into every system prompt;
provenance — hub, version, verdict, and install time — is written to .hub-lock.json for audits and updates.

In multi-user deployments, installing is admin-only: a new skill lands in the admin catalog and stays invisible to other users until a grant assigns it, so an admin can vet it before rolling it out.

Also compatible with ClawHub

Because DeepTutor speaks the open Agent-Skills format, ClawHub works as a first-class source too — it's built in alongside EduHub. Pick it with the hub prefix:

deeptutor skill search "git release notes" --hub clawhub
deeptutor skill install clawhub:git-release-notes@1.0.1

Add more registries in settings/skill_hubs.json: a type: "clawhub" entry points at any compatible HTTP API (EduHub and ClawHub both speak it), type: "command" wraps whatever fetch CLI a registry ships, and "default" chooses the hub used for bare slugs. All of them feed the same import gate.

🌐 Community

📮 Contact

DeepTutor is an open-source project led by Bingxi Zhao within the HKUDS Group, and it iterates in a fully open-source form, built together with the community. So far, we DO NOT have paid online products of any form. Feel free to reach out at bingxizhao39@gmail.com for discussions, ideas, or collaboration.

🙏 Appreciation

Heartfelt thanks to Chao Huang, director of the Data Intelligence Lab @ HKU, and to our HKUDS labmates for their warm support — especially Jiahao Zhang, Zirui Guo, and Xubin Ren. We're also deeply grateful to the open-source community: your stars, issues, pull requests, and discussions shape DeepTutor every single day.

DeepTutor also stands on the shoulders of outstanding open-source projects that gave us both tools and inspiration:

Project	Role / Inspiration
LlamaIndex	RAG pipeline and document-indexing backbone
nanobot	Ultra-lightweight agent engine that powered the original TutorBot (HKUDS)
LightRAG	Simple & fast RAG (HKUDS)
AutoAgent	Zero-code agent framework (HKUDS)
AI-Researcher	Automated research pipeline (HKUDS)
OpenClaw	Open agent gateway and skill ecosystem behind ClawHub
Codex	Agent-native coding CLI that inspired our CLI workflow
Claude Code	Agentic coding CLI that inspired the DeepTutor agent loop
ManimCat	AI-driven math animation generation for Math Animator

🗺️ Roadmap & Contribute

We want DeepTutor to keep iterating and improving — and ultimately to become a gift we give back to the open-source community. Our roadmap is updated continuously; vote on items there or propose new ones. If you'd like to contribute, see the Contributing Guide for branching strategy, coding standards, and how to get started.

We hope DeepTutor becomes a gift for the community. 🎁

Licensed under the Apache License 2.0.

Shubhamsaboo/awesome-llm-apps

100+ AI Agents, Agent Skills and RAG Apps - Free and Open Source.

Awesome LLM Apps

100+ open-source AI agents, agent skills, and RAG apps. Hand-built, tested end-to-end, Apache-2.0.

Clone it, ship it, sell it - 100% free and open-source

Works with Claude, Gemini, GPT, DeepSeek, Llama, Qwen and other open-source models.

Step-by-step tutorials on Unwind AI · Quick start · Browse all templates

_{Project Graveyard}	_{Insurance Claim Live Agent Team}	_{AI Fraud Investigation Agent}
_{Self-Improving Agent Skills}	_{AI Home Renovation Agent}	_{Always-on HN Briefing Agent}

🚀 Run one now

Give your coding agent a new skill in 10 seconds:

npx skills add https://github.com/Shubhamsaboo/awesome-llm-apps/tree/main/agent_skills/project-graveyard

Then ask it: "why do I never finish my side projects?"

Or clone and run any agent in 30 seconds:

git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git
cd awesome-llm-apps/starter_ai_agents/ai_travel_agent
pip install -r requirements.txt
streamlit run travel_agent.py

📬 New templates drop weekly. Get them in your inbox on Unwind AI.

📂 Browse all templates

🧩 Agent Skills

Give your coding agent new abilities. One command to install, plain English to use. Every skill ships real code and passes a security + eval CI gate. Works with Claude Code, Codex, Cursor, and other coding agents. Browse all skills →

⚰️ Project Graveyard - Finds every side project you abandoned, tells you why each one died, and helps you finish the one worth going back to
🔭 Scope Creep Detector - Checks whether a diff grew beyond its stated intent and recommends what to keep, split, or justify
🏺 Commit Archaeologist - Reconstructs why a file or code region exists from its introducing commit, later edits, co-changes, and intent clues
🩺 Dependency Doctor - Checks a dependency manifest for standard-library pins, obsolete backports, unpinned entries, duplicate constraints, and yanked releases
🧠 Advisor Orchestrator Worker - Meta Loop with Claude Fable 5 as advisor, GPT-5.6 as orchestrator, and Gemini 3.5 Flash as worker
♾️ Self-Improving Agent Skills - Automatically optimize agent skills using Gemini and ADK

🌱 Starter AI Agents

Single-file agents that run with just an API key - a great place to start.

🎙️ AI Blog to Podcast Agent - Turn any blog URL into a narrated podcast episode
❤️‍🩹 AI Breakup Recovery Agent - An agent team that talks you through the post-breakup spiral
📊 AI Data Analysis Agent - Ask questions of any CSV or Excel file in plain English
🩻 AI Medical Imaging Agent - Diagnostic analysis of X-rays and scans with Gemini
😂 AI Meme Generator Agent (Browser) - Makes memes by driving a real browser, not an image API
🎵 AI Music Generator Agent - Prompt in, MP3 track out
🛫 AI Travel Agent (Local & Cloud) - Personalized day-by-day travel itineraries
✨ Gemini Multimodal Agent - Video analysis plus web search in one agent
🔄 Mixture of Agents - Multiple LLMs answer, one aggregates the best response
📊 xAI Finance Agent - Real-time stock analysis powered by Grok
🔍 OpenAI Research Agent - Multi-agent topic research with the OpenAI Agents SDK
🕸️ Web Scraping AI Agent - Describe what to extract and the agent scrapes it

🚀 Advanced AI Agents

Production-style agents with tools, memory, and multi-step reasoning.

🏚️ 🍌 AI Home Renovation Agent with Nano Banana Pro - Photos of your space in, renovation plan and photorealistic renders out
🧠 DevPulse AI - Multi-Agent Signal Intelligence - Aggregates and scores technical signals into a daily intelligence digest
🔍 AI Deep Research Agent - Comprehensive web research with the OpenAI Agents SDK and Firecrawl
📊 AI VC Due Diligence Agent Team - Multi-agent startup investment analysis with Gemini 3
🔬 AI Research Planner & Executor (Google Interactions API) - Multi-phase research with stateful conversations and auto-generated infographics
🤝 AI Consultant Agent - Market analysis and strategy recommendations with live web research
🏗️ AI System Architect Agent - Architecture reviews using DeepSeek R1 reasoning plus Claude
💰 AI Financial Coach Agent - Personalized budget, debt, and savings analysis
🎬 AI Movie Production Agent - Script drafts and casting ideas from a one-line movie concept
📈 AI Investment Agent - Stock comparison reports built on Yahoo Finance data
📡 Earnings Call Analyst Agent - Turns YouTube earnings calls into a playback-synced analyst workspace
🏋️‍♂️ AI Health & Fitness Agent - Tailored diet and workout plans from your goals
🚀 AI Product Launch Intelligence Agent - Go-to-market intelligence on competitor launches
🔍 AI Fraud Investigation Agent - Cross-references public records to flag facilities that don't add up
🗞️ AI Journalist Agent - Researches, writes, and edits articles on any topic
🧠 AI Mental Wellbeing Agent - A coordinated agent team for mental health support plans
📑 AI Meeting Agent - Context, industry insights, and strategy briefs before you walk in
🧬 AI Self-Evolving Agent - Agents that rewrite their own workflows with EvoAgentX
👨🏻‍💼 AI Sales Intelligence Agent Team - Generates competitive sales battle cards in real time
🎧 AI Social Media News and Podcast Agent - Curates your trusted sources into briefs and generated podcasts
🌐 Openwork - Open Browser Automation Agent _{↗ external} - Open-source agent that operates a real browser
🛡️ Trust-Gated Multi-Agent Research Team - Every agent verified, every action in a hash-chained audit trail

🛰️ Always-on Agents

Background agents that run on schedules or events, monitor changing context, decide what needs attention, and proactively deliver updates, artifacts, or actions.

📰 Always-on Hacker News Briefing Agent - A scheduled scout that ships a ranked daily brief to Slack or email
📡 Release Radar Agent - Watches dependency releases and briefs you on breaking, deprecated, security, and major-version changes

🤝 Multi-agent Teams

Multiple agents collaborating to accomplish complex, cross-domain tasks.

🧲 AI Competitor Intelligence Agent Team - Structured competitor teardowns built from their own websites
💲 AI Finance Agent Team - A financial analyst team in 20 lines of Python
🎨 AI Game Design Agent Team - Full game concepts from a swarm of design specialists
🧭 AG2 Adaptive Research Team - Agent teamwork with routing and fallback, built on AG2
👨‍⚖️ AI Legal Agent Team (Cloud & Local) - Research, contract analysis, and strategy from a full legal bench
💼 AI Recruitment Agent Team - Resume screening to interview scheduling, end to end
🏠 AI Real Estate Agent Team - Property search, market analysis, and recommendations
👨‍💼 AI Services Agency (CrewAI) - A digital agency that scopes and plans your software project
👨‍🏫 AI Teaching Agent Team - A faculty of agents that builds your complete learning path
💻 Multimodal Coding Agent Team - Snap a photo of a coding problem, get a sandboxed solution
✨ Multimodal Design Agent Team - Design critiques from a Gemini-powered expert panel
🎨 🍌 Multimodal UI/UX Feedback Agent Team - Landing page feedback plus an auto-generated improved version
🌏 AI Travel Planner Agent Team - A complete trip itinerary, crafted by a team

🗣️ Voice AI Agents

Speech-in, speech-out agents using real-time voice APIs.

🗣️ AI Audio Tour Agent - Self-guided audio tours from your location, interests, and pace
📞 Customer Support Voice Agent - Voice answers grounded in your own docs
🛡️ Insurance Claim Live Agent Team - Real-time voice claim intake with Gemini Live
🔊 Voice RAG Agent (OpenAI SDK) - Ask your PDFs questions, hear the answers
🎙️ OpenSource Voice Dictation Agent (Wispr Flow clone) _{↗ external} - Open-source dictation that types where you talk

🖼️ Generative UI and Agentic Frontends

Agents that render interactive UI components, not just text: forms, cards, charts, editable plans.

🗂️ Generative UI Starter Project - A chat-driven kanban board you and the agent work together
🪙 AI Financial Coach Agent - Budget, savings, and debt plans rendered as interactive cards
📊 AI Dashboard Canvas Agent - Describe a dashboard in chat, charts assemble on a live canvas
🛠️ AI MCP App Builder - Describe an MCP app, get a live sandboxed instance back
✈️ MCP Apps Generative UI Showcase - MCP apps that render real interactive UI, flight search included
🎛️ AI Shadcn Component Generator - Chat your way to production-ready shadcn components
🔍 AI Deep Research Agent - Research where every tool call renders as a live workspace card

🎮 Autonomous Game-Playing Agents

Agents that play games end-to-end: reasoning, strategy, and action.

🎮 AI 3D Pygame Agent - DeepSeek R1 writes PyGame code, browser agents run it live
♜ AI Chess Agent - Agent White vs Agent Black with validated moves
🎲 AI Tic-Tac-Toe Agent - Two different LLMs battle it out, move by move

♾️ MCP AI Agents

Agents that connect to external tools and data via Model Context Protocol.

♾️ Browser MCP Agent - Drive a real browser with natural language over MCP
🐙 GitHub MCP Agent - Explore and analyze any repo in plain English
📑 Notion MCP Agent - Talk to your Notion pages from the terminal
🌍 AI Travel Planner MCP Agent - Itineraries built on live Airbnb and Google Maps data
🔀 Multi-MCP Agent Router - Specialist agents, each wired to its own MCP server

📀 RAG (Retrieval Augmented Generation)

Retrieval pipelines, from simple chains to agentic and multi-source.

🔥 Agentic RAG with Embedding Gemma - Fully local agentic RAG with EmbeddingGemma and Llama 3.2
🧐 Agentic RAG with Reasoning - Watch the agent's step-by-step reasoning as it retrieves
📰 AI Blog Search (RAG) - Agentic search over blog content, built on LangGraph
🔍 Autonomous RAG - GPT-4o answers from your PDFs, falls back to web search
🔄 Contextual AI RAG Agent - Managed RAG: datastore to grounded chat in minutes
🔄 Corrective RAG (CRAG) - Retrieval that grades itself and retries before answering
📎 Typed Agentic RAG with Pydantic AI - Validated answers with exact citations, or a refusal when evidence is weak
🐋 Deepseek Local RAG Agent - Local DeepSeek reasoning over your own documents
🤔 Gemini Agentic RAG - Query rewriting and web fallback with Gemini Flash Thinking
👀 Hybrid Search RAG (Cloud) - Keyword plus vector search feeding Claude
🔄 Llama 3.1 Local RAG - Chat with any webpage, fully offline
🖥️ Local Hybrid Search RAG - Hybrid search with everything running on your machine
🧬 Multimodal Agentic RAG - Text, PDFs, images, audio, and video, answered with citations
🦙 Local RAG Agent - Llama 3.2 and Qdrant, no API keys required
🧩 RAG-as-a-Service - A production RAG service in under 50 lines
✨ RAG Agent with Cohere - Command R7B retrieval with web-search fallback
⛓️ Basic RAG Chain - The minimal retrieval pipeline, applied to pharma research
📠 RAG with Database Routing - Routes each question to the right database automatically
🖼️ Vision RAG - Ask questions about images and PDF pages with Embed-4
🩺 RAG Failure Diagnostics Clinic - Find out why your RAG pipeline is wrong, systematically
🕸️ Knowledge Graph RAG with Citations - Multi-hop answers with verifiable source attribution

💾 LLM Apps with Memory

Agents and chatbots that remember conversations and user state across sessions.

💾 AI ArXiv Agent with Memory - Paper search that remembers your research interests
🛩️ AI Travel Agent with Memory - A travel assistant that remembers your preferences
💬 Llama3 Stateful Chat - Session-persistent chat with Llama 3
📝 LLM App with Personalized Memory - A chatbot that keeps context across conversations
🗄️ Local ChatGPT Clone with Memory - Fully local, with a personal memory per user
🧠 Multi-LLM Application with Shared Memory - Different models, one shared conversation memory

💬 Chat with X

Turn any data source into a chat interface.

💬 Chat with GitHub (GPT & Llama3) - Any repo, answered in 30 lines of RAG
📨 Chat with Gmail - Ask your inbox questions
📄 Chat with PDF (GPT & Llama3) - The classic, in 30 lines of Python
📚 Chat with Research Papers (ArXiv) (GPT & Llama3) - Explore arXiv conversationally with GPT-4o
📝 Chat with Substack - Chat with any newsletter's archive
📽️ Chat with YouTube Videos - Ask videos questions via their transcripts

🎯 LLM Optimization Tools

Reduce token usage, context size, and API cost without losing quality.

🎯 Toonify Token Optimization - Reduce LLM API costs by 30-60% using TOON format
🧠 Headroom Context Optimization - Reduce LLM API costs by 50-90%

🔧 LLM Fine-tuning

End-to-end fine-tuning recipes for open-source models.

🦥 Gemma 3 Fine-tuning - 4-bit LoRA with Unsloth, small and readable
🦙 Llama 3.2 Fine-tuning - Fine-tune in 30 lines, free on Colab

🧑‍🏫 AI Agent Framework Crash Courses

Deep-dive tutorials on the major agent frameworks.

Google ADK Crash Course - Starter agent, structured outputs, tools (built-in, function, third-party, MCP), memory, callbacks, plugins, and multi-agent patterns. Model-agnostic.
OpenAI Agents SDK Crash Course - Starter agent, function calling, structured outputs, tools, memory, evaluation, handoffs, swarm orchestration, and routing logic.

⭐ Star the repo to get notified when new templates drop.

_{Deutsch · Español · français · 日本語 · 한국어 · Português · Русский · 中文}

_{Apache-2.0 · See LICENSE · Fork it, ship it, sell it.}

tirth8205/code-review-graph

Local-first code intelligence graph for MCP and CLI. Builds a persistent map of your codebase so AI coding tools read only what matters, with benchmarked context reductions on reviews and large-repo workflows.

code-review-graph

Stop burning tokens. Start reviewing smarter.

English | 简体中文 | 日本語 | 한국어 | हिन्दी

Usage · Commands · FAQ · Troubleshooting · GitHub Action · Reproducing the benchmarks · Roadmap

AI coding tools can end up re-reading large parts of your codebase on review tasks. code-review-graph fixes that. It builds a structural map of your code with Tree-sitter, tracks changes incrementally, and gives your AI assistant precise context via MCP so it reads only what matters.

Quick Start

pip install code-review-graph                     # or: pipx install code-review-graph
code-review-graph install          # auto-detects and configures all supported platforms
code-review-graph build            # parse your codebase

One command sets up everything. install detects which AI coding tools you have, writes the correct MCP configuration for each one, installs platform-native hooks/skills where supported, and injects graph-aware instructions into your platform rules. It auto-detects whether you installed via uvx or pip/pipx and generates the right config. Restart your editor/tool after installing.

To target a specific platform:

code-review-graph install --platform codex       # configure only Codex
code-review-graph install --platform cursor      # configure only Cursor
code-review-graph install --platform claude-code  # configure only Claude Code
code-review-graph install --platform gemini-cli   # configure only Gemini CLI
code-review-graph install --platform antigravity   # configure only Antigravity
code-review-graph install --platform windsurf     # configure only Windsurf
code-review-graph install --platform zed          # configure only Zed
code-review-graph install --platform continue     # configure only Continue
code-review-graph install --platform opencode     # configure only OpenCode
code-review-graph install --platform qwen         # configure only Qwen
code-review-graph install --platform qoder        # configure only Qoder
code-review-graph install --platform kiro         # configure only Kiro
code-review-graph install --platform copilot      # configure only GitHub Copilot (VS Code)
code-review-graph install --platform copilot-cli  # configure only GitHub Copilot CLI
code-review-graph install --platform codebuddy    # configure only CodeBuddy Code

Requires Python 3.10+. For the best experience, install uv (the MCP config will use uvx if available, otherwise falls back to the code-review-graph command directly).

To remove CRG from a Git or SVN project, use the symmetric uninstall command from anywhere inside its working tree. The target is normalized to the working tree root, and non-repository directories are refused. It removes only CRG-owned files and entries; unrelated MCP servers, hooks, skills, and JSONC comments remain untouched. Shared configuration changes use atomic replacement so a failed write leaves the original file intact.

code-review-graph uninstall --dry-run    # preview every action; write nothing
code-review-graph uninstall              # preview, ask for confirmation, then apply
code-review-graph uninstall --yes        # apply without prompting
code-review-graph uninstall --all-repos  # also clean every registered repository
code-review-graph uninstall --keep-data  # remove integrations but keep graph databases
code-review-graph uninstall --keep-user-configs --repo .  # clean this project only

Then open your project and ask your AI assistant:

Build the code review graph for this project

The initial build takes ~10 seconds for a 500-file project. After that, watch mode and supported hooks can keep the graph updated automatically.

How It Works

Your repository is parsed into an AST with Tree-sitter, stored as a graph of nodes (functions, classes, imports) and edges (calls, inheritance, test coverage), then queried at review time to compute the minimal set of files your AI assistant needs to read.

Blast-radius analysis

When a file changes, the graph traces every caller, dependent, and test that could be affected. This is the "blast radius" of the change. Your AI reads only these files instead of scanning the whole project.

Incremental updates in seconds

When hooks or watch mode are enabled, file saves and supported commit hooks trigger incremental updates. The graph diffs changed files, finds their dependents through the graph's own import and call edges, and re-parses only the files whose SHA-256 hash actually changed. On a ~3,000-file project (django) a two-file edit re-indexes in about 2.5 seconds on the path the hooks use, of which ~1.4 s is process start-up; a no-op update costs only that start-up. See Incremental update latency for the full measurement.

Whole codebase or targeted answer?

The bigger the repository, the more token waste hurts. Instead of feeding a whole corpus to the model, the graph returns an answer-shaped slice of it: on this repository, 208,821 source tokens become ~3,190 tokens per question.

Broad language coverage + Jupyter notebooks

Parser support covers functions, classes, imports, call sites, inheritance, and test detection across the current parser surface, using Tree-sitter where available and targeted fallbacks where needed. Current support includes Python, JavaScript/TypeScript/TSX, Go, Rust, Java, C/C++, C#, VB.NET, Ruby, Kotlin, Swift, PHP, Scala, Solidity, Dart, R, Perl, Lua/Luau, Objective-C, shell scripts, Elixir, Zig, PowerShell, Julia, ReScript, GDScript, Nix, Verilog/SystemVerilog, SQL, Terraform/OpenTofu structure (.tf; generic .hcl files are recognized as file nodes), Ansible playbooks/roles/tasks, Vue/Svelte SFCs, Astro files parsed through the TypeScript parser, Jupyter/Databricks notebooks (.ipynb), and Perl XS files (.xs). Generic YAML is not treated as source code.

PHP projects additionally get repository-bounded Composer PSR-4 resolution, Blade template references, and Laravel Route/Eloquent semantic edges when the source includes explicit framework imports, model inheritance, and receiver evidence.

Add your own language (no fork needed)

If your repo uses a language the parser does not cover yet, drop a languages.toml into .code-review-graph/ mapping file extensions to any grammar bundled in tree_sitter_language_pack, plus the tree-sitter node types for functions, classes, imports, and calls:

[languages.erlang]
extensions = [".erl"]
grammar = "erlang"
function_node_types = ["function_clause"]
class_node_types = ["record_decl"]
import_node_types = ["import_attribute"]
call_node_types = ["call"]

The generic tree-sitter walker handles extraction from there — no code changes, and built-in languages can never be overridden. See docs/CUSTOM_LANGUAGES.md for the schema reference, validation rules, and a worked end-to-end example.

Risk-scored PR reviews in CI (GitHub Action)

The same analysis runs as a composite GitHub Action — and it stays local-first: the knowledge graph is built and queried entirely on your CI runner, with no source code sent to any external service. On each pull request the action posts a single sticky comment with risk-scored functions, affected execution flows, and test gaps, updated in place on every push. An optional fail-on-risk input turns the review into a merge gate.

# .github/workflows/code-review-graph.yml
on:
  pull_request:

permissions:
  contents: read
  pull-requests: write

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: tirth8205/code-review-graph@v2.3.6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

See docs/GITHUB_ACTION.md for inputs, risk levels, and caching details, or the dogfood workflow this repo runs on itself in .github/workflows/pr-review.yml.

Benchmarks

Headline number: the median per-question token reduction across the 6 repos is ~65x (whole-corpus baseline vs graph query). The 376x maximum is a single best-case repo (fastapi, the largest corpus) — not the typical result.

All numbers come from the automated evaluation runner against 6 real open-source repositories (13 commits total). Every config pins an upstream SHA, the Leiden community detector runs with a fixed seed, and embeddings are deterministic on CPU — so two runs on different machines produce identical numbers. The full reproduction recipe with expected outputs is in docs/REPRODUCING.md. A weekly report-only run on the two smallest configs lives in .github/workflows/eval.yml.

Token efficiency: ~65x median per-question reduction (range 36x – 376x; whole-corpus vs graph query)

For a typical agent question ("how does authentication work", "what is the main entry point", etc.), the graph returns ~2,000–3,500 tokens of targeted search hits + neighbor edges instead of forcing the agent to read every source file. The table below averages over the 5 sample questions defined in code_review_graph/token_benchmark.py.

Repo	Snapshot SHA	naive_corpus_tokens	avg graph_tokens	Reduction
fastapi	`22381558`	948,793	2,653	375.6x
flask	`a29f88ce`	143,594	2,196	71.0x
code-review-graph	`84bde354`	208,821	3,190	68.1x
gin	`5c00df8a`	166,868	2,766	61.9x
httpx	`b55d4635`	142,356	2,661	60.6x
express	`b4ab7d65`	136,052	3,936	36.0x

Median per-question reduction across the 6 repos: ~65x. The range is 36x – 376x, where 376x is the best case (fastapi, the largest corpus), not the headline.

Re-captured 2026-08-02 from clean clones at the pinned SHAs (crg 2.3.7, local all-MiniLM-L6-v2 embeddings). These numbers are lower than the 2026-05-25 capture they replace: the graph response grew as node embedding text became richer, so avg graph_tokens rose across every repo. fastapi is now measured at its current pin 22381558 rather than the retired 0227991a.

The whole-corpus baseline above is an upper bound no real agent pays: a competent agent greps for identifiers and reads only the best-matching files. The agent_baseline eval benchmark measures that realistic baseline — a pure-python grep over the corpus, top-3 files by match count, token-counted and compared to the graph query cost (evaluate/results/<repo>_agent_baseline_*.csv).

The formal eval/benchmarks/token_efficiency.py benchmark measures a different scenario — full get_review_context() JSON versus just the changed-file content of a commit — and reports ratios below 1 for small commits, because the review-context response carries impact-radius edges plus source snippets that exceed a tiny single-file diff. That is not a bug; the two benchmarks answer different questions. See docs/REPRODUCING.md for the full methodology.

Since v2.3.4, review and impact tools attach a compact context_savings estimate so MCP clients can see the approximate context saved per call. In v2.3.5 the CLI surfaces this as the boxed Token Savings panel shown above (see "Token Savings panel" in the Usage section) and adds --verify to cross-check against OpenAI's cl100k_base tokenizer. Calibration data in docs/REPRODUCING.md shows the estimate is within ~1% of real GPT-4 tokens in aggregate across 222 sample files.

Impact accuracy: 0.69 average F1 against graph-derived ground truth (recall 1.0 is a circular upper bound, not "100% recall")

Blast-radius analysis recovers every file in the ground truth on all 13 evaluation commits — but read that as an upper bound, not as "100% recall": in this mode the ground truth (changed files + files with call/import edges into them) is derived from the same graph the predictor traverses, so it is circular by construction. The over-prediction visible in the precision column is a deliberate trade-off: better to flag too many files than miss a broken dependency.

Repo	Commits	Avg F1	Avg Precision	Recall (graph-derived upper bound)
httpx	2	0.863	0.785	1.0
code-review-graph	2	0.734	0.584	1.0
fastapi	2	0.697	0.539	1.0
express	2	0.667	0.500	1.0
flask	2	0.633	0.485	1.0
gin	3	0.609	0.439	1.0
Average	13	0.693	0.546	1.000

The benchmark also runs an honest co-change mode: the predictor is seeded with a single changed file and graded against the other files the author actually touched in the same commit — independent-ish evidence from git history, not from the graph. Both modes appear side by side in the result CSVs (ground_truth_mode column). As of the 2026-08-02 capture that mode returns predicted_files = 0 on every graded commit, so it is not yet a usable measurement and no co-change number is quoted here — the harness needs fixing before the mode says anything about accuracy.

Build performance

Repo	Files	Nodes	Edges	Flow Detection	Search Latency
express	141	1,910	17,553	106ms	0.7ms
fastapi	1,122	6,285	27,117	128ms	1.5ms
flask	83	1,446	7,974	95ms	0.7ms
gin	99	1,286	16,762	111ms	0.5ms
httpx	60	1,253	7,896	96ms	0.4ms

Limitations and known weaknesses

Impact "recall 1.0" is graph-derived and circular: the historical ground truth comes from the same graph edges the predictor walks, so it is an upper bound by construction. The honest co-change mode (grade against files actually co-changed in the same commit) is measured alongside it; expect those numbers to be substantially lower.
Small single-file changes: Graph context can exceed naive file reads for trivial edits (see express results above). The overhead is the structural metadata that enables multi-file analysis.
Search quality (MRR 0.35): Keyword search finds the right result in the top-4 for most queries, but ranking needs improvement. Express queries return 0 hits due to module-pattern naming.
Flow detection (33% recall): Framework and conventional entry patterns are strongest for Python and PHP/Laravel. JavaScript and Go flow detection needs work.
Precision vs recall trade-off: Impact analysis is deliberately conservative. It flags files that might be affected, which means some false positives in large dependency graphs.

Features

Feature	Details
Incremental updates	Re-parses only the files whose hash changed. On a ~3,000-file repo a two-file edit takes ~2.5s on the hook path (measured).
Broad language + notebook support	Python, JavaScript/TypeScript/TSX, Go, Rust, Java, C/C++, C#, VB.NET, Ruby, Kotlin, Swift, PHP, Scala, Solidity, Dart, R, Perl, Lua/Luau, Objective-C, shell scripts, Elixir, Zig, PowerShell, Julia, ReScript, GDScript, Nix, Verilog/SystemVerilog, SQL, Terraform/OpenTofu structure (`.tf`; generic `.hcl` files are file-only), Ansible playbooks/roles/tasks, Vue/Svelte SFCs, Astro files parsed through the TypeScript parser, Jupyter/Databricks (.ipynb), and Perl XS (.xs)
Framework-aware PHP parsing	Repository-bounded Composer PSR-4 imports, Blade template references, and evidence-gated Laravel Route-to-controller and Eloquent relationship edges
Blast-radius analysis	Shows which functions, classes, and files are likely affected by a change
Auto-update hooks	Hooks and watch mode can update the graph on file saves and supported commit hooks
Semantic search	Optional vector embeddings via sentence-transformers, Google Gemini, MiniMax, or any OpenAI-compatible endpoint (real OpenAI, Azure, new-api, LiteLLM, vLLM, LocalAI)
Interactive visualisation	D3.js force-directed graph with search, community legend toggles, and degree-scaled nodes
Hub & bridge detection	Find most-connected nodes and architectural chokepoints via betweenness centrality
Surprise scoring	Detect unexpected coupling: cross-community, cross-language, peripheral-to-hub edges
Knowledge gap analysis	Identify isolated nodes, untested hotspots, thin communities, and structural weaknesses
Suggested questions	Auto-generated review questions from graph analysis (bridges, hubs, surprises)
Edge confidence	Three-tier confidence scoring (EXTRACTED/INFERRED/AMBIGUOUS) with float scores on edges
Graph traversal	Free-form BFS/DFS exploration from any node with configurable depth and token budget
Export formats	GraphML (Gephi/yEd), Neo4j Cypher, Obsidian vault with wikilinks, SVG static graph
Graph diff	Compare graph snapshots over time: new/removed nodes, edges, community changes
Token benchmarking	Measure naive full-corpus tokens vs graph query tokens with per-question ratios
Estimated context savings	Compact `context_savings` metadata on relevant MCP/CLI review outputs, labelled as estimated and kept to three small fields
Memory loop	Persist Q&A results as markdown for re-ingestion, so the graph grows from queries
Community auto-split	Oversized communities (>25% of graph) are recursively split via Leiden
Execution flows	Trace call chains from entry points, sorted by weighted criticality
Community detection	Cluster related code via Leiden algorithm with resolution scaling for large graphs
Architecture overview	Auto-generated architecture map with coupling warnings
Risk-scored reviews	`detect_changes` maps diffs to affected functions, flows, and test gaps
Custom languages	Add new languages via `.code-review-graph/languages.toml` — no fork or code changes needed
GitHub Action	Sticky risk-scored PR review comments in CI, with an optional `fail-on-risk` merge gate
Refactoring tools	Rename preview, framework-aware dead code detection, community-driven suggestions
Wiki generation	Auto-generate markdown wiki from community structure
Multi-repo registry	Register multiple repos, search across all of them
Multi-repo daemon	`crg-daemon` watches multiple repos as child processes, with health checks and auto-restart
MCP prompts	5 workflow templates: review, architecture, debug, onboard, pre-merge
Full-text search	FTS5-powered hybrid search combining keyword and vector similarity
Local storage	SQLite file in `.code-review-graph/`. Core graph storage needs no external database or cloud service.
Watch mode	Continuous graph updates as you work

Usage

Slash commands

Command	Description
`/code-review-graph:build-graph`	Build or rebuild the code graph
`/code-review-graph:review-delta`	Review changes since last commit
`/code-review-graph:review-pr`	Full PR review with blast-radius analysis

CLI reference

code-review-graph install          # Auto-detect and configure all platforms
code-review-graph install --platform <name>  # Target a specific platform
code-review-graph uninstall --dry-run  # Preview safe removal of installed artifacts
code-review-graph build            # Parse entire codebase
code-review-graph update           # Incremental update (changed files only)
code-review-graph status           # Graph statistics
code-review-graph watch            # Auto-update on file changes
code-review-graph visualize        # Generate interactive HTML graph
code-review-graph visualize --format json      # Export local graph data as JSON
code-review-graph visualize --format graphml   # Export as GraphML
code-review-graph visualize --format svg       # Export as SVG
code-review-graph visualize --format obsidian  # Export as Obsidian vault
code-review-graph visualize --format cypher    # Export as Neo4j Cypher
code-review-graph wiki             # Generate markdown wiki from communities
code-review-graph detect-changes --brief         # Risk panel + token savings (read-only)
code-review-graph update --brief                 # Refresh graph + same panel
code-review-graph detect-changes --brief --verify  # Cross-check vs tiktoken
code-review-graph register <path>  # Register repo in multi-repo registry
code-review-graph unregister <id>  # Remove repo from registry
code-review-graph repos            # List registered repositories
code-review-graph daemon start     # Start multi-repo watch daemon
code-review-graph daemon stop      # Stop the daemon
code-review-graph daemon status    # Show daemon status and repos
code-review-graph eval             # Run evaluation benchmarks
code-review-graph serve            # Start MCP server

JSON exports stay inside the local graph data directory, which Git ignores by default. They can contain absolute paths and code-structure metadata, so inspect and sanitize an export before publishing it outside your machine.

Token Savings panel: detect-changes --brief vs update --brief

Both commands print the same compact panel showing how many tokens the graph saved you compared to handing the changed files to an agent raw. They differ in one thing: whether the graph gets refreshed first.

┌─────────────────────── Token Savings ────────────────────────┐
│ Full context would be:     12,921 tokens                     │
│ Graph context used:           762 tokens                     │
│ Saved:                     12,159 tokens (~94%)              │
│ Breakdown: Functions 244 · Tests 191 · Risk 244 · Other 83   │
└──────────────────────────────────────────────────────────────┘

Command	What it does	When to use
`detect-changes --brief`	Read-only. Looks at your current changes, queries the existing graph, prints the panel. ~1 sec.	Most of the time — the hooks (or `crg-daemon`) keep the graph fresh in the background, so this is enough.
`update --brief`	Re-parses your changed files into the graph first, then prints the same panel. ~5 sec.	After a rebase, a large change set, or any time you suspect the graph is stale.

Both end with the same panel because both call the same analyze_changes() step at the end. The difference is whether the graph itself got refreshed before that analysis ran.

Add --verify to either command to cross-check the displayed numbers against OpenAI's cl100k_base tokenizer (the GPT-4 family). Requires pip install tiktoken. The estimate stays within ~1% of real tokens on a typical change set — see docs/REPRODUCING.md for the calibration data.

The same context_savings metadata is also attached automatically to the JSON responses of get_impact_radius, get_review_context, detect_changes, and get_architecture_overview MCP tools, so AI agents can surface the savings to humans in chat without any extra prompting.

Multi-repo daemon

If your editor doesn't support hooks (e.g. Cursor, OpenCode), or you just want your graph to stay fresh in the background without any editor integration, the daemon is for you. It watches your repos for file changes and automatically rebuilds the graph — no manual build or update commands needed.

The daemon is included with code-review-graph — no separate install required.

Quick setup:

# 1. Register the repos you want to watch
crg-daemon add ~/project-a --alias proj-a
crg-daemon add ~/project-b

# 2. Start the daemon (runs in the background)
crg-daemon start

# 3. That's it — graphs stay up to date automatically
crg-daemon status                 # check daemon and per-repo watcher status
crg-daemon logs --repo proj-a -f  # tail logs for a specific repo
crg-daemon stop                   # stop daemon and all watcher processes

Also available as code-review-graph daemon start|stop|status|....

Under the hood, crg-daemon add writes to a TOML config file at ~/.code-review-graph/watch.toml. You can also edit this file directly:

[[repos]]
path = "/home/user/project-a"
alias = "proj-a"

[[repos]]
path = "/home/user/project-b"
alias = "project-b"

The daemon monitors this config file for changes and automatically starts/stops watcher processes as repos are added or removed. Health checks every 30 seconds restart dead watchers. No external dependencies required.

See docs/COMMANDS.md for the full config reference and all available options.

30 MCP tools

Your AI assistant uses these automatically once the graph is built.

Tool	Description
`build_or_update_graph_tool`	Build or incrementally update the graph
`run_postprocess_tool`	Re-run flow detection, community detection, and FTS indexing
`get_minimal_context_tool`	Ultra-compact context (~100 tokens) — call this first
`get_impact_radius_tool`	Blast radius of changed files
`get_review_context_tool`	Token-optimised review context with structural summary
`query_graph_tool`	Callers, callees, tests, imports, inheritance queries
`traverse_graph_tool`	BFS/DFS traversal from any node with token budget
`semantic_search_nodes_tool`	Search code entities by name or meaning
`embed_graph_tool`	Compute vector embeddings for semantic search
`list_graph_stats_tool`	Graph size and health
`get_docs_section_tool`	Retrieve documentation sections
`find_large_functions_tool`	Find functions/classes exceeding a line-count threshold
`list_flows_tool`	List execution flows sorted by criticality
`get_flow_tool`	Get details of a single execution flow
`get_affected_flows_tool`	Find flows affected by changed files
`list_communities_tool`	List detected code communities
`get_community_tool`	Get details of a single community
`get_architecture_overview_tool`	Architecture overview from community structure
`detect_changes_tool`	Risk-scored change impact analysis for code review
`get_hub_nodes_tool`	Find most-connected nodes (architectural hotspots)
`get_bridge_nodes_tool`	Find chokepoints via betweenness centrality
`get_knowledge_gaps_tool`	Identify structural weaknesses and untested hotspots
`get_surprising_connections_tool`	Detect unexpected cross-community coupling
`get_suggested_questions_tool`	Auto-generated review questions from analysis
`refactor_tool`	Rename preview, dead code detection, suggestions
`apply_refactor_tool`	Apply a previously previewed refactoring
`generate_wiki_tool`	Generate markdown wiki from communities
`get_wiki_page_tool`	Retrieve a specific wiki page
`list_repos_tool`	List registered repositories
`cross_repo_search_tool`	Search across all registered repositories

MCP Prompts (5 workflow templates): review_changes, architecture_map, debug_issue, onboard_developer, pre_merge_check

Configuration

To exclude paths from indexing, create a .code-review-graphignore file in your repository root:

generated/**
*.generated.ts
vendor/**
node_modules/**

Note: in git repos, only tracked files are indexed (git ls-files), so gitignored files are skipped automatically. Use .code-review-graphignore to exclude tracked files or when git isn't available.

Optional dependency groups:

pip install "code-review-graph[embeddings]"          # Local vector embeddings (sentence-transformers)
pip install "code-review-graph[google-embeddings]"   # Google Gemini embeddings
pip install "code-review-graph[communities]"         # Community detection (igraph)
pip install "code-review-graph[enrichment]"          # Python call-resolution enrichment (Jedi)
pip install "code-review-graph[eval]"                # Evaluation benchmarks (matplotlib)
pip install "code-review-graph[wiki]"                # Wiki generation with LLM summaries (ollama)
pip install "code-review-graph[all]"                 # All optional dependencies

Environment Variables

Variable	Description	Default
`CRG_GIT_TIMEOUT`	Timeout in seconds for Git operations	`30`
`CRG_DATA_DIR`	Override directory for graph databases and generated graph artefacts	-
`CRG_EMBEDDING_MODEL`	Default model for vector embeddings	`all-MiniLM-L6-v2`
`CRG_ACCEPT_CLOUD_EMBEDDINGS`	Suppress the cloud embedding egress warning after explicit acknowledgement	-
`CRG_ALLOW_REMOTE_CODE`	Allow HuggingFace models that require `trust_remote_code=True`	`0`
`CRG_MAX_IMPACT_NODES`	Maximum nodes to include in impact analysis	`500`
`CRG_MAX_IMPACT_DEPTH`	Search depth for blast-radius analysis	`2`
`CRG_MAX_BFS_DEPTH`	Maximum depth for graph traversal	`15`
`CRG_MAX_CHANGED_FUNCS`	Maximum changed functions analysed in one change report	`500`
`CRG_MAX_TRANSITIVE_FRONTIER`	Maximum frontier size for transitive caller/callee expansion	`50`
`CRG_TOOL_TIMEOUT`	Optional timeout in seconds for bounded MCP tools (`0` disables timeout)	`0`
`CRG_RECURSE_SUBMODULES`	Include git submodules in file collection when set to `1`, `true`, or `yes`	-
`CRG_TOOLS`	Comma-separated allowlist of MCP tools to expose when serving	-
`GOOGLE_API_KEY`	API key for Google Gemini embeddings	-
`MINIMAX_API_KEY`	API key for MiniMax embeddings	-
`VOYAGE_API_KEY`	API key for Voyage embeddings	-
`CRG_VOYAGE_MODEL`	Model name for Voyage embeddings	`voyage-code-3`
`CRG_VOYAGE_OUTPUT_DIMENSION`	Output dimension for Voyage embeddings	`1024`
`CRG_VOYAGE_OUTPUT_DTYPE`	Output dtype for Voyage embeddings	`float`
`CRG_VOYAGE_BASE_URL`	Voyage embeddings endpoint	`https://api.voyageai.com/v1`
`CRG_VOYAGE_BATCH_SIZE`	Batch size for Voyage embedding requests	`100`
`CRG_VOYAGE_MIN_INTERVAL_SEC`	Minimum delay between Voyage requests	`0`
`CRG_OPENAI_BASE_URL`	OpenAI-compatible embeddings endpoint	-
`CRG_OPENAI_API_KEY`	API key for OpenAI-compatible embeddings	-
`CRG_OPENAI_MODEL`	Model name for OpenAI-compatible embeddings	-
`CRG_OPENAI_DIMENSION`	Pin embedding dimension (v3 models support reduction)	-
`NO_COLOR`	If set, disables ANSI colors in terminal	-
`CRG_SERIAL_PARSE`	If `1`, disables parallel parsing (use for debugging)	-

OpenAI-compatible embeddings (real OpenAI, Azure, or any self-hosted gateway like new-api / LiteLLM / vLLM / LocalAI / Ollama in openai mode) need no extra install — just set the environment variables and pass provider="openai" to embed_graph:

export CRG_OPENAI_BASE_URL=http://127.0.0.1:3000/v1     # or https://api.openai.com/v1
export CRG_OPENAI_API_KEY=sk-...
export CRG_OPENAI_MODEL=text-embedding-3-small          # whatever your gateway serves
# optional:
export CRG_OPENAI_DIMENSION=1536                        # pin dim (v3 models support reduction)
export CRG_OPENAI_BATCH_SIZE=100                        # lower for gateways with tight limits
                                                        # (e.g. Qwen text-embedding-v4 caps at 10)

The cloud-egress warning is auto-skipped when the base URL points to localhost (127.0.0.1, localhost, 0.0.0.0, ::1).

Voyage embeddings need no extra install. Set VOYAGE_API_KEY and pass provider="voyage" to embed_graph; the default model is voyage-code-3:

export VOYAGE_API_KEY=pa-...
export CRG_ACCEPT_CLOUD_EMBEDDINGS=1
code-review-graph embed --provider voyage --model voyage-code-3

Model selection tip. Avoid -preview / -beta / -exp model IDs (e.g. google/gemini-embedding-2-preview) for anything you plan to keep long-term — preview models can change weights (different dimension → full re-embed required) or be deprecated without notice. Prefer stable GA releases such as text-embedding-3-small / text-embedding-3-large (OpenAI), Qwen/Qwen3-Embedding-8B (via self-hosted vLLM / LocalAI), or gemini-embedding-001 (via the native Gemini provider, which requires GOOGLE_API_KEY instead of the OpenAI-compatible path).

code-review-graph embeds identifiers, signatures, structural context, and a bounded first-paragraph docstring/doc-comment summary. It does not transmit function bodies. Graphs created before documentation extraction was added need one full code-review-graph build before re-embedding so every file is reparsed. Routine builds never refresh embeddings by default. To refresh an existing index after a build, explicitly pass both --embedding-provider and --embedding-model; cloud choices may transmit this source-derived text and incur API cost.

Tool Filtering

CRG exposes 30 MCP tools by default. In token-constrained environments, you can limit the server to a subset of tools using --tools or the CRG_TOOLS environment variable:

# Via CLI flag
code-review-graph serve --tools query_graph_tool,semantic_search_nodes_tool,detect_changes_tool

# Via environment variable
CRG_TOOLS=query_graph_tool,semantic_search_nodes_tool code-review-graph serve

The CLI flag takes precedence over the environment variable. When neither is set, all tools are available. This is especially useful for MCP client configurations:

{
  "mcpServers": {
    "code-review-graph": {
      "command": "code-review-graph",
      "args": ["serve", "--tools", "query_graph_tool,semantic_search_nodes_tool,detect_changes_tool,get_review_context_tool"]
    }
  }
}

FAQ & how it compares

Short, honest answers in docs/FAQ.md:

vs LSP / language servers — one persistent cross-language graph instead of per-language daemons; LSP stays more precise per symbol.
vs RAG / embeddings — structural edges parsed from the AST, not similarity chunks; embeddings are optional and only assist search.
vs grep / agentic search — grep wins on one-hop lookups; the graph wins on multi-hop questions (impact radius, callers-of-callers, tests-for, affected flows).
vs Serena, codegraph, claude-context, repomix — factual comparison table.
When NOT to use it — small repos, trivial single-file diffs, one-off questions.
Does it phone home? — no; zero telemetry, cloud embeddings are opt-in.
How do I verify it is working? — status, detect-changes --brief, /mcp.

Troubleshooting

`pip` / `pipx` cannot download `hatchling` (or `Errno 9` / `Bad file descriptor` to PyPI)

Installing from a source tree (for example pipx install .) needs build dependencies from PyPI (for example hatchling). If you see Could not find a version that satisfies the requirement hatchling after connection warnings, the Python/pip in that terminal may not be able to open an HTTPS client to pypi.org (sometimes seen in an integrated editor terminal; less often system-wide with VPN, firewall, or proxy).

Options:

Run the same command from macOS Terminal.app (or iTerm) instead of the IDE’s terminal, then retry pipx install . or pipx install "git+https://..." .
Use uv to install the CLI from a checkout (uses different download machinery than pip in many cases):
```
cd /path/to/code-review-graph
uv tool install . --force
```
For development in a clone without a global install, use uv sync and uv run code-review-graph … (or activate .venv after uv sync).

Diagnose (optional): python3 scripts/diagnose_pypi_connectivity.py — if it prints FAILED, the issue is environment/network, not a wrong package name in this repo.

Windows Configuration Issues (Invalid JSON / Connection Closed)

If you are using Windows and encounter Invalid JSON: EOF while parsing or MCP error -32000: Connection closed when connecting via Claude Code, do not use the cmd /c wrapper in your config.

Ensure fastmcp is updated to at least 3.2.4+. Then, configure your ~/.claude.json to execute the .exe directly and pass the UTF-8 environment variable via the config:

"code-review-graph": {
  "command": "C:\\path\\to\\your\\venv\\Scripts\\code-review-graph.exe",
  "args": ["serve", "--repo", "C:\\path\\to\\your\\project"],
  "env": { "PYTHONUTF8": "1" }
}

Contributing

git clone https://github.com/tirth8205/code-review-graph.git
cd code-review-graph
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytest

Adding a new language

Edit code_review_graph/parser.py and add your extension to EXTENSION_TO_LANGUAGE along with node type mappings in _CLASS_TYPES, _FUNCTION_TYPES, _IMPORT_TYPES, and _CALL_TYPES. Include a test fixture and open a PR.

Licence

MIT. See LICENSE.

code-review-graph.com

pip install code-review-graph && code-review-graph install
_{Works with Codex, Claude Code, CodeBuddy Code, Cursor, Windsurf, Zed, Continue, OpenCode, Antigravity, Gemini CLI, Qwen, Qoder, Kiro, GitHub Copilot, and GitHub Copilot CLI}

virgiliojr94/book-to-skill

Turn any technical book PDF into a Claude Code skill — ready to study, reference, and use while you work.

book-to-skill

Turn any technical book, document folder, or collection of sources into a unified agent skill — ready to study, reference, and use while you work in GitHub Copilot CLI, Amp, or Claude Code.

Why · What it generates · Beyond books · How it works · Usage · Install · FAQ · Performance · Architecture · Changelog

24×–51× fewer tokens than dumping the book into context to answer one question, measured on real books (how it's measured).

How it works, in 3 steps:

Point it at a file, folder, or glob — /book-to-skill ./my-book.pdf
It distills the book into a skill — frameworks, decision rules, anti-patterns, and per-chapter files. Structure, not a summary.
Your agent loads it on demand — ask /my-book replication and it reads the right chapter and answers from the real content, no hallucination.

🤔 Why

You buy a great technical book. You read it once. Three months later you can't remember chapter 7 existed.

The usual workarounds don't help:

📄 "Let me just search the PDF" → you get a list of pages, not answers
🧠 "I'll ask the agent about this book" → it either hallucinates or says it doesn't have the content
📝 "I'll take notes as I read" → you end up with a 200-line doc you never open again

book-to-skill solves this by turning the book into a structured skill your agent loads on demand.

Once installed, you just type /your-book-slug replication and the agent reads the right chapter and answers from the actual content. No hallucination. No digging through PDFs. The book becomes part of your workflow.

Works with any host that supports the open Agent Skills standard — GitHub Copilot CLI, Amp, and Claude Code all read the same SKILL.md format.

📦 What it generates

Running /book-to-skill your-book.pdf (or a folder, glob, or list of files) creates a full skill in your agent's skills directory (~/.copilot/skills/<slug>/ for Copilot CLI, ~/.agents/skills/<slug>/ for Amp or cross-agent, ~/.claude/skills/<slug>/ for Claude Code):

File	Purpose	Size
`SKILL.md`	Core mental models + chapter index	~4,000 tokens
`chapters/ch01-*.md` …	One file per chapter, loaded on-demand	~1,000 tokens each
`glossary.md`	Every key term, alphabetically sorted with chapter refs	~1,500 tokens
`patterns.md`	All techniques, algorithms, and design patterns	~2,000 tokens
`cheatsheet.md`	Decision tables and quick-reference rules	~1,000 tokens

Chapter files are loaded on-demand — they don't count against the skill budget until you ask about that topic.

🏢 Beyond books

The name says "book", but the input is any structured prose. The same extraction works on knowledge you own and re-read constantly:

Internal documentation — architecture decision records, runbooks, onboarding guides. Fold a whole docs/ folder into one skill and ask it while you code.
Brand & design systems — voice guidelines, tone-of-voice docs, component principles. Turn a brand book into a skill your team queries instead of skimming a 60-page PDF.
Research clusters — a stack of papers plus your own notes, merged into a single unified skill and updated as new material lands (see Update / fold-in).
Specs & standards — RFCs, API contracts, compliance docs you reference but never memorize.

If you re-open a document often enough to wish you'd memorized it, it's a candidate.

🧾 The Discovery Loop Tax

A PDF-reading agent doesn't just read — it navigates: it re-fetches the ToC, backtracks, and re-processes all of it on every turn. book-to-skill pays that structuring cost once, at conversion, so queries stay proportional to the answer — 24×–51× fewer tokens than dumping the book into context, measured on real books.

📊 Full methodology, numbers, and per-book tables → docs/performance.md

⚙️ How it works

Two halves: a deterministic Python extractor (document → clean text + metadata) and a spec-driven generator (your agent follows SKILL.md to turn that into a structured skill). On-demand chapter files keep the loaded skill small.

🔧 Full walkthrough (Steps 0–10, extraction modes, token budgets) → docs/how-it-works.md

🚀 Usage

/book-to-skill <path|folder|glob> [skill-name] — plus analyze-only, generate-from-analysis, and update/fold-in modes.

▶️ All modes and examples → docs/usage.md

📥 Install

# Agent skill (registers /book-to-skill) — clone into your skills folder:
git clone https://github.com/virgiliojr94/book-to-skill.git ~/.claude/skills/book-to-skill
# (Copilot CLI: ~/.copilot/skills/ · Amp/cross-agent: ~/.agents/skills/)

📥 All hosts, optional extractors, and the standalone CLI → docs/install.md

❓ FAQ

Common questions — "why not just dump the PDF?", cost, privacy, non-book inputs, multi-file books.

❓ Answers → docs/faq.md

🔧 Requirements

The extractor tries tools in order per format and uses the first available. If nothing is installed, it tells you which command to run. Plain text, Markdown, reStructuredText and AsciiDoc need no extra deps.

Check your setup in one command: python3 scripts/extract.py --check prints which extractors are installed for every format and the exact command to install anything missing — no file needed.

PDF — choose by book type:

Book type	Tool	Install	Speed
Text-heavy (prose, few tables)	`pdftotext` (poppler)	`sudo apt install poppler-utils`	⚡ instant
Text-heavy fallback	`pypdf`	`pip3 install pypdf`	⚡ instant
Text-heavy fallback	`pdfminer.six`	`pip3 install pdfminer.six`	⚡ instant
Technical (code, tables, formulas)	`docling`	`pip3 install docling`	~1.5s/page

Before extraction begins, the skill asks you whether the book is technical or text-heavy and picks the right tool automatically. Docling preserves markdown tables and code blocks; pdftotext is faster for prose-only books.

EPUB:

Tool	Install	Quality
`ebooklib` + `beautifulsoup4`	`pip3 install ebooklib beautifulsoup4`	⭐⭐⭐ Best
stdlib `zipfile`	built-in — no install needed	⭐⭐ Always available

Other formats:

Format	Tool	Install
DOCX	`python-docx` (fallback: stdlib ZIP/XML)	`pip3 install python-docx`
HTML	`beautifulsoup4` (fallback: stdlib `html.parser`)	`pip3 install beautifulsoup4`
RTF	`striprtf` (fallback: regex)	`pip3 install striprtf`
MOBI / AZW / AZW3	Calibre `ebook-convert` (external app, not pip)	https://calibre-ebook.com/download
TXT / Markdown / reStructuredText / AsciiDoc	built-in	—

📁 Repository structure

book-to-skill/
├── SKILL.md              # Skill definition + step-by-step instructions (the generator spec)
├── scripts/
│   ├── extract.py        # Thin entrypoint wrapper
│   └── extractor/        # Modular extraction package
│       ├── config.py     # Extensions, paths, dependency constants
│       ├── dependencies.py  # optional-dep probing + --check
│       ├── exceptions.py # ExtractionError (per-source failures, batch-safe)
│       ├── utils.py      # CLI parsing, multi-source resolution, chapter detection, runner
│       └── parsers/      # Format-specific parsers (pdf, epub, docx, html, rtf, calibre, text)
├── tools/
│   ├── discovery_tax.py  # measures token cost vs context-dump / discovery loop
│   └── validate_skill.py # checks a generated SKILL.md against host rules (--lens claude|copilot|amp)
├── tests/                # pytest suite (extraction, detection, discovery tax)
├── docs/
│   ├── performance.md    # measured benchmarks, discovery tax, cost
│   └── architecture.md   # pipeline + component map
├── CHANGELOG.md          # release history (semver)
├── CONTRIBUTING.md       # dev setup, PR conventions, release process
├── SECURITY.md           # vulnerability reporting
└── README.md             # This file

⚖️ Copyright & fair use

book-to-skill ships no book content — not a single page. It's a converter you point at files you already own.

Processing is local. Extraction and analysis run on your machine. Your files are never uploaded by this tool. (If your agent's model runs in the cloud, the text you feed it follows that provider's normal data terms — same as any prompt.)
You use your own copy. Bring a book you bought, docs your company owns, or papers you have the right to read.
The output is your notes. A generated skill is a structured, synthesized derivative — framework names, definitions, takeaways — not a reproduction of the text. The skill explicitly never copies raw passages (see Quality Rule #7). Treat it like handwritten study notes: yours, for personal use.
Don't redistribute. Publishing or sharing a generated skill of a copyrighted work can infringe the rights holder. Keep skills of third-party books private. Internal docs, your own writing, and openly-licensed material are fine to share within the bounds of their license.

When in doubt, follow the license or terms of the source document. This project is a tool; how you use it is on you.

💖 Sponsors

book-to-skill is free and MIT-licensed, maintained on personal time. If it saves you tokens or study hours, consider sponsoring its upkeep: PR reviews, multilingual fixes, releases, and docs.

Become a sponsor → github.com/sponsors/virgiliojr94

Every sponsor is listed in BACKERS.md. Thank you for keeping open, privacy-first tooling alive. ✨

License

MIT — applies to the converter (code + skill definition) in this repository, not to any book or document you process with it.

Star History

huggingface/speech-to-speech

Build local voice agents with open-source models

Speech To Speech: Build voice agents with open-source models

A low-latency, fully modular voice-agent pipeline: VAD -> STT -> LLM -> TTS, exposed through an OpenAI Realtime-compatible WebSocket API. Every component is swappable. The LLM slot speaks OpenAI-compatible protocols, so you can point it at a hosted provider, at HF Inference Providers, or at a vLLM or llama.cpp server on your own hardware for a fully local, fully open stack.

This pipeline runs in production as the conversation backend for thousands of Reachy Mini robots.

Quickstart

pip install speech-to-speech
export OPENAI_API_KEY=...
speech-to-speech serve

This starts an OpenAI Realtime-compatible server at ws://localhost:8765/v1/realtime using Parakeet TDT for local STT, an OpenAI-compatible LLM, and Qwen3-TTS for local speech output.

Talk to it from a second terminal:

speech-to-speech talk --url ws://127.0.0.1:8765/v1/realtime

To start the server and packaged microphone/speaker client in one command:

speech-to-speech local

Prefer to keep the LLM on your own machine? Serve Gemma 4 with llama.cpp:

llama-server -hf ggml-org/gemma-4-E4B-it-GGUF -np 2 -c 65536 -fa on --swa-full

Then point the OpenAI-compatible LLM backend at it:

speech-to-speech serve \
    --model_name "ggml-org/gemma-4-E4B-it-GGUF" \
    --responses_api_base_url "http://127.0.0.1:8080/v1" \
    --responses_api_api_key ""

Any OpenAI Realtime-compatible client can connect. See Realtime API for the protocol and LLM backends for provider and local-server options.

Index

How it works

The pipeline is a cascade of four components, each running in its own thread and connected by queues:

Voice Activity Detection (VAD): Silero VAD v5 detects speech boundaries and turn-taking.
Speech to Text (STT): transcribes the user's turn, with optional live partial transcripts.
Language Model (LLM): generates the response, streaming text and tool calls.
Text to Speech (TTS): synthesizes audio and streams it back to the client.

Every stage has multiple interchangeable backends, selected via CLI flags. The code is designed for easy modification, with a focus on models available through Transformers and the Hugging Face Hub.

Installation

Requires Python 3.10+.

pip install speech-to-speech

The default install covers the standard realtime path:

Parakeet TDT for STT
OpenAI-compatible API for the language model
Qwen3-TTS for speech output, using the GGML backend by default on non-macOS platforms and mlx-audio on Apple Silicon
local audio and realtime server modes

macOS and non-macOS dependencies are resolved automatically via platform markers in pyproject.toml.

CUDA Note for Qwen3-TTS

On Linux, the Qwen3-TTS GGML backend comes from faster-qwen3-tts[ggml]. Its default qwentts-cpp-python wheel on PyPI targets CUDA 12.8. If your machine does not have the CUDA 12 runtime that wheel expects, install the matching wheel from the Hugging Face wheelhouse before installing speech-to-speech:

# CUDA 13.x
pip install "qwentts-cpp-python==0.3.1+cu130" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu130

# CUDA 12.4
pip install "qwentts-cpp-python==0.3.1+cu124" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu124

# CPU-only fallback
pip install "qwentts-cpp-python==0.3.1+cpu" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cpu

pip install speech-to-speech

To use the previous CUDA-graphs implementation instead of GGML, pass --qwen3_tts_backend torch.

Optional Components

Optional components are installed with pip extras:

pip install "speech-to-speech[kokoro]"          # Kokoro-82M TTS on non-macOS
pip install "speech-to-speech[pocket]"          # Pocket TTS
pip install "speech-to-speech[chattts]"         # ChatTTS
pip install "speech-to-speech[faster-whisper]"  # Faster Whisper STT
pip install "speech-to-speech[whisper-mlx]"     # Lightning Whisper MLX STT on macOS
pip install "speech-to-speech[paraformer]"      # Paraformer STT through FunASR
pip install "speech-to-speech[mlx-lm]"          # mlx-vlm support for vision models on macOS

Deprecated implementations, including MeloTTS, live in archive/ and are no longer wired into the CLI.

Note on DeepFilterNet: DeepFilterNet, used for optional audio enhancement in VAD, requires numpy<2 and conflicts with Pocket TTS, which requires numpy>=2. Install it manually only in environments where you are not using Pocket TTS.

From Source

git clone https://github.com/huggingface/speech-to-speech.git
cd speech-to-speech
uv sync

This installs the package in editable mode and makes the speech-to-speech CLI available.

Supported Components

Component	Backend	Platforms	Install
VAD	Silero VAD v5	all	built-in
STT	Parakeet TDT (default)	CUDA / CPU through nano-parakeet, Apple Silicon through MLX	built-in
STT	Whisper through Transformers	CUDA / CPU	built-in
STT	Faster Whisper	CUDA / CPU	`faster-whisper`
STT	Lightning Whisper MLX	Apple Silicon	`whisper-mlx`
STT	MLX Audio Whisper	Apple Silicon	built-in on macOS
STT	Paraformer	CUDA / CPU	`paraformer`
LLM	OpenAI-compatible API (`responses-api`, `chat-completions`)	hosted providers or self-hosted servers	built-in
LLM	Transformers	CUDA / CPU	built-in
LLM	mlx-lm	Apple Silicon	built-in on macOS
TTS	Qwen3-TTS (default)	GGML / CUDA on Linux, mlx-audio on macOS	built-in
TTS	Kokoro-82M	CUDA / CPU, Apple Silicon	`kokoro` on non-macOS; built-in on macOS
TTS	Pocket TTS	CPU / CUDA	`pocket`
TTS	ChatTTS	CUDA / CPU	`chattts`
TTS	MMS TTS	CUDA / CPU	built-in

Select implementations with --stt, --llm_backend, and --tts. The CLI constructs configuration only for the selected backends; known options for inactive backends remain accepted for compatibility but are ignored with a warning. JSON configuration may likewise include extra inactive-backend keys, which are ignored. Run speech-to-speech serve -h for the defaults, or pass selectors before -h to see another combination's backend-specific flags (for example, speech-to-speech serve --stt mlx-audio-whisper -h).

Commands

Command	Behavior	Use it when
`serve`	Runs the pipeline server over OpenAI Realtime WebSocket and WebRTC.	You are building an app or device against the API.
`talk --url <full-realtime-url>`	Runs the packaged microphone/speaker client.	You want to talk to an existing Realtime server.
`local`	Composes `serve` and `talk` in-process over loopback.	You want to run the server and talk to it from one command.

serve binds to 127.0.0.1 by default; pass --host 0.0.0.0 explicitly for network exposure. local always binds to loopback and connects the same packaged client at ws://127.0.0.1:<port>/v1/realtime.

Migrating from `--mode`

--mode is deprecated and will stop working soon. During this migration window, speech-to-speech --mode realtime runs speech-to-speech serve, and speech-to-speech --mode local runs speech-to-speech local; both print a warning. All other mode values have been removed and exit with guidance to use the new commands.

Realtime Server

export OPENAI_API_KEY=...
speech-to-speech serve

This is equivalent to:

speech-to-speech serve \
    --thresh 0.6 \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
    --qwen3_tts_speaker Aiden \
    --qwen3_tts_language auto \
    --qwen3_tts_backend ggml \
    --qwen3_tts_non_streaming_mode True \
    --qwen3_tts_mlx_quantization 6bit \
    --model_name gpt-5.4-mini \
    --chat_size 30 \
    --responses_api_stream \
    --enable_live_transcription

The default model is gpt-5.4-mini through the OpenAI Responses API. Override it with --model_name, and set --responses_api_base_url for another OpenAI-compatible provider or server.

Local Mac

speech-to-speech local --mac-optimal-settings

Optionally with a specific LLM:

speech-to-speech local \
    --mac-optimal-settings \
    --model_name mlx-community/Qwen3-4B-Instruct-2507-bf16

This setting:

Uses MPS defaults for supported model components.
Sets Parakeet TDT for STT.
Sets MLX LM as the LLM backend.
Sets Qwen3-TTS for TTS, using mlx-audio with the 6bit MLX variant by default.

The preset supplies these as defaults only: explicit --device, component-device flags such as --qwen3_tts_device, and --stt, --llm_backend, --model_name, and --tts all win. Use it with serve instead of local when you want to expose the server without starting the microphone/speaker client.

--tts pocket and --tts kokoro are also valid on macOS.

To compare the MLX quantization variants locally:

python scripts/benchmark_tts.py \
    --handlers qwen3 \
    --iterations 3 \
    --qwen3_mlx_quantizations bf16 4bit 6bit 8bit

Docker

Install the NVIDIA Container Toolkit, then:

docker compose up

The compose file starts a llama.cpp server with Gemma 4 and the Realtime server, exposing ports 8080 and 8765.

Realtime API

Realtime mode supports the OpenAI Realtime protocol over WebSocket and WebRTC, with live transcription and low-latency turn-taking. WebSocket clients connect at /v1/realtime:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8765/v1",
    websocket_base_url="ws://localhost:8765/v1",
    api_key="not-needed",
)

with client.realtime.connect(model="local") as conn:
    conn.send(
        {
            "type": "session.update",
            "session": {
                "type": "realtime",
                "instructions": "You are a helpful assistant.",
                "audio": {
                    "input": {
                        "turn_detection": {
                            "type": "server_vad",
                            "interrupt_response": True,
                        }
                    }
                },
            },
        }
    )

    for event in conn:
        print(event.type)

The server implements the core Realtime event set: input_audio_buffer.append, session.update, conversation.item.create, response.create, and response.cancel inbound; speech start/stop, streaming transcription, audio deltas, tool calls, and response.done outbound. The full event reference, architecture, and design details live in the Realtime Engine README.

LLM Proxy

With --enable_llm_proxy, the realtime server also exposes the remote LLM it is configured with as a plain OpenAI compatible endpoint, so a client can run side tasks (summaries, titles, background agents) with tools and streaming, fully concurrent with the voice conversation and never interrupted by new speech:

POST /v1/chat/completions when running --llm_backend chat-completions
POST /v1/responses when running --llm_backend responses-api

The server performs no authentication and no throttling of its own. Enable the proxy only on a trusted network, or deploy the server behind a gateway that owns access control. The s2s-endpoint compute replica is such a gateway: it opens these paths only to clients that created their session with an HF token, checks the API key against that token, and applies a rate limit per user. Point the stock OpenAI SDK at whichever host you talk to; this server ignores the API key (a gateway in front decides what it must be):

from openai import OpenAI

llm = OpenAI(base_url="http://localhost:8765/v1", api_key="unused")
completion = llm.chat.completions.create(
    model="anything",  # ignored: the server forces its configured --model_name
    messages=[{"role": "user", "content": "Summarize the conversation so far: ..."}],
)

Requests are stateless (send the full message list each time) and are proxied to the configured upstream with the key held by the server, which never reaches clients. The model field is always overwritten with the server configured --model_name. The proxy is off by default, requires a remote backend (chat-completions or responses-api), and answers 501 with the reason otherwise.

LLM Backends

The LLM is the most compute-intensive and highest-latency component in the pipeline. A single forward pass through a large model can dominate end-to-end response time, so choosing the right backend for your hardware and latency budget matters. The pipeline supports:

Local inference: transformers on CUDA / CPU and mlx-lm on Apple Silicon.
Self-hosted servers: responses-api and chat-completions can point at a local vLLM or llama.cpp server.
Provider APIs: the same backends work with OpenAI, HF Inference Providers, OpenRouter, and other OpenAI-compatible providers.

Two API backends are available, sharing the same --responses_api_* connection flags:

--llm_backend responses-api (default) targets /v1/responses.
--llm_backend chat-completions targets /v1/chat/completions.

Direct Audio Input (No STT)

Use --stt none --llm_backend chat-completions to send each completed VAD audio segment directly to an audio-input model. Direct audio mode is not supported with --llm_backend responses-api: a model may accept audio through /v1/chat/completions without supporting /v1/responses, including OpenAI's gpt-audio-1.5.

You must explicitly set --model_name to a model that accepts audio: the default gpt-5.4-mini accepts text and image input, but not audio. Check the provider's model documentation and endpoint support before enabling this mode. For OpenAI, see the GPT-5.4 mini model card and audio-input guide.

speech-to-speech serve \
    --stt none \
    --llm_backend chat-completions \
    --model_name "YOUR_AUDIO_CAPABLE_MODEL" \
    --responses_api_base_url "https://provider.example/v1" \
    --responses_api_api_key "$PROVIDER_API_KEY"

OpenAI-compatible servers represent input audio differently. Use --responses_api_audio_content_type input_audio (the default) for embedded WAV base64, or --responses_api_audio_content_type audio_url for a base64 data URL.

The examples below pair Parakeet TDT for local STT and Qwen3-TTS for local TTS with different LLM backends.

Responses API Backend

Works with any provider or server that implements the OpenAI Responses API. Point --responses_api_base_url at the endpoint and set --model_name accordingly:

Provider / server	`--responses_api_base_url`	`--responses_api_api_key`
OpenAI	omit, uses OpenAI default	`$OPENAI_API_KEY`
HF Inference Providers	`https://router.huggingface.co/v1`	`$HF_TOKEN`
OpenRouter	`https://openrouter.ai/api/v1`	`$OPENROUTER_API_KEY`
vLLM	`http://localhost:8000/v1`	omit or any string
llama.cpp	`http://127.0.0.1:8080/v1`	empty string

# OpenAI
speech-to-speech local \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_mlx_quantization 6bit \
    --model_name "gpt-4o-mini" \
    --responses_api_api_key "$OPENAI_API_KEY" \
    --responses_api_stream \
    --enable_live_transcription

# HF Inference Providers: Qwen3.5-9B via Together
speech-to-speech local \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_mlx_quantization 6bit \
    --model_name "Qwen/Qwen3.5-9B:together" \
    --responses_api_base_url "https://router.huggingface.co/v1" \
    --responses_api_api_key "$HF_TOKEN" \
    --responses_api_stream \
    --enable_live_transcription

# HF Inference Providers: GPT-oss-20B via Groq
speech-to-speech serve \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --qwen3_tts_mlx_quantization 6bit \
    --model_name "openai/gpt-oss-20b:groq" \
    --responses_api_base_url "https://router.huggingface.co/v1" \
    --responses_api_api_key "$HF_TOKEN" \
    --responses_api_stream \
    --enable_live_transcription

Chat Completions Backend

Identical configuration to responses-api, reusing the same --responses_api_* connection flags, but talks to /v1/chat/completions instead of /v1/responses. Prefer it when:

the provider ignores chat_template_kwargs.enable_thinking on the Responses path and needs a reasoning_effort knob to suppress reasoning, or
the server's Responses streaming tool-call path is unreliable, while its Chat Completions tool-call streaming is solid. This is useful for some vLLM builds; see #312.

Add --responses_api_reasoning_effort none to disable reasoning on providers where the chat-template flag has no effect:

# vLLM serving a Qwen model with tool calling
speech-to-speech serve \
    --stt parakeet-tdt \
    --llm_backend chat-completions \
    --tts qwen3 \
    --model_name "Qwen/Qwen3-4B-Instruct-2507" \
    --responses_api_base_url "http://localhost:8000/v1" \
    --responses_api_stream

# Gemma 4 31B via the HF router on Cerebras, with reasoning disabled for low voice latency
speech-to-speech serve \
    --stt parakeet-tdt \
    --llm_backend chat-completions \
    --tts qwen3 \
    --model_name "google/gemma-4-31B-it:cerebras" \
    --responses_api_base_url "https://router.huggingface.co/v1" \
    --responses_api_api_key "$HF_TOKEN" \
    --responses_api_reasoning_effort none \
    --responses_api_stream

Fully Local

Run the LLM in a separate llama.cpp process for the lowest-friction fully local setup, as shown in the Reachy Mini local conversation guide:

For a fully local native-audio setup with the browser demo, Realtime turn revisions, and barge-in, see the tested Gemma 4 12B speech-to-speech example for Apple Silicon.

# Terminal 1: llama.cpp serving Gemma 4
llama-server -hf ggml-org/gemma-4-E4B-it-GGUF -np 2 -c 65536 -fa on --swa-full

# Terminal 2: speech-to-speech using that local LLM server
speech-to-speech serve \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --model_name "ggml-org/gemma-4-E4B-it-GGUF" \
    --responses_api_base_url "http://127.0.0.1:8080/v1" \
    --responses_api_api_key "" \
    --responses_api_stream \
    --enable_live_transcription

Use speech-to-speech local when you want to run the same server and talk through the machine hosting it. In-process local backends are available with --llm_backend mlx-lm on Apple Silicon or --llm_backend transformers on CUDA / CPU.

Offline Operation

The pipeline can run without internet access after the dependencies and model assets for the selected components are installed locally. Before disconnecting, start the exact configuration once while online so it can cache the STT, LLM, TTS, Silero VAD, NLTK, and Smart Turn resources it needs.

For the lowest-friction fully local LLM setup, run llama.cpp on the same machine as described in Fully Local. Once llama.cpp and the pipeline assets are available locally, set HF_HUB_OFFLINE=1 when starting speech-to-speech to prevent Hugging Face Hub requests:

HF_HUB_OFFLINE=1 speech-to-speech serve \
    --model_name "ggml-org/gemma-4-E4B-it-GGUF" \
    --responses_api_base_url "http://127.0.0.1:8080/v1" \
    --responses_api_api_key ""

Without the local base URL override, the default responses-api LLM backend calls a remote service. Alternatively, use an in-process local backend such as transformers or mlx-lm. Every selected model must already be cached or supplied through a local path supported by its backend.

Smart Turn uses a separate ONNX checkpoint. A cached checkpoint works with HF_HUB_OFFLINE=1; for an explicit, cache-independent setup, pass --smart_turn_model_path /path/to/smart-turn-v3.2-cpu.onnx. If the checkpoint is not available, pass --no_smart_turn to disable Smart Turn.

Multi-Language Support

Language coverage depends on the STT and TTS backends you pick, not on the pipeline itself:

Component	Backend	Languages
STT	Parakeet TDT (default)	25 European languages
STT	Whisper / Whisper MLX / Faster Whisper	Broad multilingual coverage, depending on the selected Whisper checkpoint
STT	Paraformer	Depends on the selected FunASR checkpoint; the default is Chinese-oriented
TTS	Qwen3-TTS (default)	Multilingual, with `--qwen3_tts_language auto` by default
TTS	Kokoro	Multiple language/voice mappings, depending on backend availability
TTS	ChatTTS	English and Chinese
TTS	MMS TTS	Broad multilingual coverage through MMS checkpoints

Make sure the STT, LLM, and TTS you pair all cover your target language(s). Two usage patterns:

Single language: set --language to the target language code. The default is en.
Language switching: set --language auto. The STT detects the language of each spoken prompt and forwards it to the LLM. Optionally add --enable_lang_prompt to append a "Please reply to my message in ..." instruction. It defaults to False; large LLMs usually infer the language from context, but the explicit instruction can help smaller models.

Automatic language detection:

speech-to-speech serve \
    --stt parakeet-tdt \
    --language auto \
    --llm_backend mlx-lm \
    --model_name "mlx-community/Qwen3-4B-Instruct-2507-bf16"

A single non-English language, Chinese in this example:

speech-to-speech serve \
    --stt whisper-mlx \
    --stt_model_name large-v3 \
    --language zh \
    --llm_backend mlx-lm \
    --model_name mlx-community/Qwen3-4B-Instruct-2507-bf16

Both commands also work with --mac-optimal-settings; explicit --stt flags override the defaults it sets.

Pocket TTS

Pocket TTS from Kyutai Labs provides streaming TTS with voice cloning:

speech-to-speech serve \
    --tts pocket \
    --pocket_tts_voice jean \
    --pocket_tts_device cpu

Available voice presets: alba, marius, javert, jean, fantine, cosette, eponine, azelma. Custom voice files and Hugging Face paths also work.

CLI Reference

References for pipeline CLI arguments live in the arguments classes and in speech-to-speech serve -h. Client arguments are listed by speech-to-speech talk -h.

Module-Level Parameters

See ModuleArguments. It allows setting:

a common --device, if every part should run on the same device
macOS model/device defaults (--mac-optimal-settings)
STT implementation (--stt)
LLM backend (--llm_backend: transformers, mlx-lm, responses-api, or chat-completions)
TTS implementation (--tts)
logging level
realtime pipeline pool size (--num_pipelines)

VAD Parameters

See VADHandlerArguments. Notable options:

--thresh: threshold value to trigger voice activity detection.
--min_speech_ms: minimum duration of detected voice activity to be considered speech.
--min_speech_continuation_ms: sustain-bar hysteresis threshold for speech that continues a reopenable soft-ended, uncommitted turn within the reopen window. The default and recommended pairing is --min_speech_ms 384 --min_speech_continuation_ms 192.
--min_silence_ms: minimum length of silence intervals for segmenting speech. Default is 64 ms.
--short_segment_merge_ms: optional merge window for stitching adjacent VAD segments that are each shorter than --min_speech_ms.
--speculative_reopen_ms: delay response commitment for 800 ms after a soft-ended turn so immediately resumed speech can reopen it.
--unanswered_reopen_ms: sanity cap on how long a soft-ended speculative turn that has not yet received any assistant output stays reopenable. With Smart Turn enabled, this is clamped to at least --smart_turn_max_wait_ms so a turn remains reopenable for its full grace.

Smart Turn endpointing

Smart Turn v3.2 can validate Silero's end-of-speech decisions using the content and prosody of the current turn. Silero finalizes the segment and STT/LLM work may begin speculatively. Complete turns start processing immediately and use --speculative_reopen_ms (800 ms by default) before committing output. Incomplete turns wait --smart_turn_incomplete_delay_ms (600 ms by default) before starting STT/LLM work, while their output remains gated by --smart_turn_max_wait_ms (2 seconds by default). If speech resumes during either delay, the existing turn is reopened as a newer revision, the accumulated audio is re-emitted, and work from the previous revision is discarded before it reaches the user.

The base package includes the quantized CPU runtime and enables Smart Turn by default:

pip install speech-to-speech
speech-to-speech serve

The latest supported v3.2 CPU checkpoint downloads from the Hugging Face Hub on first use. Pass --smart_turn_model_path /path/to/model.onnx to use a local model, or --no_smart_turn to disable Smart Turn. Smart Turn is enabled by default for server sessions and the packaged local client.

Tune the completion cutoff with --smart_turn_threshold (default 0.5). A higher threshold makes ambiguous pauses more likely to use the longer speculative response grace.

STT, LLM, and TTS Parameters

model_name, torch_dtype, and device are exposed for each STT, LLM, and TTS implementation. STT and TTS parameters use the handler prefix, for example --stt_model_name or --qwen3_tts_device. LLM model selection and chat settings are shared across backends via unprefixed flags, for example --model_name and --chat_size; backend-specific flags use the responses_api_ prefix for the responses-api and chat-completions backends and the llm_ prefix for local backends.

For example:

# Local transformers/mlx-lm backend
--model_name google/gemma-2b-it

# OpenAI-compatible backend
--llm_backend responses-api --model_name deepseek-chat --responses_api_base_url https://api.deepseek.com

Generation Parameters

Other generation parameters can be set using the handler prefix plus _gen_, for example --stt_gen_max_new_tokens 128 or --llm_gen_temperature 0.7. Parameters not yet exposed can be added to the relevant arguments class.

Contributing

Issues and PRs are welcome. Good starting points are the open issues. For larger changes, open an issue first to discuss the approach.

For local development:

uv sync
pytest
ruff check

Star History

Citations

If you use this pipeline, please also cite the component models you run. The defaults are:

Silero VAD

@misc{SileroVAD,
  author = {Silero Team},
  title = {Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/snakers4/silero-vad}},
  email = {hello@silero.ai}
}

Parakeet TDT

@misc{parakeet-tdt,
  author = {NVIDIA},
  title = {Parakeet TDT 0.6B v3},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3}}
}

Qwen3-TTS

@misc{qwen3-tts,
  author = {Qwen Team},
  title = {Qwen3-TTS},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice}}
}

Citations for optional backends such as Kokoro, Pocket TTS, ChatTTS, Whisper variants, Paraformer, and MMS live in the respective component READMEs.

bradautomates/claude-video

Give Claude the ability to watch any video. /watch downloads, extracts frames, transcribes, hands it all to Claude.

/watch

Give Claude the ability to watch any video.

Claude Code (recommended — auto-updates via marketplace):

/plugin marketplace add bradautomates/claude-video
/plugin install watch@claude-video

Codex, Cursor, Copilot, Gemini CLI, or any of 50+ Agent Skills hosts:

npx skills add bradautomates/claude-video -g

(-g installs globally for your user, available across all projects. Drop it to scope per-project.)

More install options (claude.ai web, manual) in the Install section below.

Zero config to start — yt-dlp and ffmpeg install on first run via brew on macOS (Linux/Windows print exact commands). Captions cover most public videos for free. Whisper API key is only needed when a video has no captions.

Claude can read a webpage, run a script, browse a repo. What it can't do, out of the box, is watch a video. You paste a YouTube link and it has to either guess from the title or pull a transcript that's missing 90% of what's on screen.

With Claude Video /watch you can paste a URL or a local path, ask a question, and Claude fetches captions first, downloads only what it needs, extracts frames (scene-aware, or fast keyframes at efficient detail), pulls a timestamped transcript (free captions when available, Whisper API as fallback), and Reads every frame as an image. By the time it answers, it has seen the video and heard the audio.

/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?

What people actually use it for

Analyze someone else's content. /watch https://youtu.be/<viral-video> what hook did they open with? Claude looks at the first frames, reads the opening transcript, breaks down the structure. Same for ad creative, competitor launches, podcast intros, anything where the how matters as much as the what.

Diagnose a bug from a video. Someone sends you a screen recording of something broken. /watch bug-repro.mov what's going wrong? Claude watches the recording, finds the frame where the issue appears, describes what's on screen, often catches the cause without you ever opening the file.

Summarize a video. /watch https://youtu.be/<long-thing> summarize this does the obvious thing — pulls the structure, the key moments, what was actually said and shown. Faster than watching at 2x.

Cut the hype out of an update video. /watch https://youtu.be/<launch-video> what's actually new — skip the hype Strip a "game-changer" feature drop down to the few things that matter, so you get the substance without ten minutes of intro and overselling.

Turn a playlist into notes. /watch https://youtu.be/<video> summarize this to a note Run it across a series and file a per-video summary, so a channel or course becomes a searchable set of notes instead of hours you have to sit through.

How it works

You paste a video and a question. URL (anything yt-dlp supports — YouTube, Loom, TikTok, X, Instagram, plus a few hundred more) or a local path (.mp4, .mov, .mkv, .webm).
yt-dlp checks captions first. At transcript detail, captioned URLs return without downloading video. Otherwise, or when Whisper needs audio, it downloads only what the run needs.
ffmpeg extracts frames at the chosen detail. efficient decodes keyframes only (near-instant); balanced/token-burner prefer scene-change frames and fall back to the duration-aware uniform sampler when they under-produce. JPEGs are 512px wide by default and clamped to 1998px tall for Claude Read compatibility.
The transcript comes from one of two places. First try: yt-dlp pulls native captions (manual or auto-generated) from the source. Free, instant, accurate-ish. Fallback: extract a mono 16 kHz 64 kbps mp3 audio clip (~480 kB/min) and ship it to Whisper — Groq's whisper-large-v3 (preferred — cheaper and faster) or OpenAI's whisper-1.
Frames + transcript are handed to Claude. The script prints frame paths with t=MM:SS markers and the transcript with timestamps. Claude Reads each frame in parallel — JPEGs render directly as images in its context.
Claude answers grounded in what's actually on screen and in the audio. Not "based on the description" or "according to the title." It saw the frames. It heard the transcript. It answers the way someone who watched the video would.
Cleanup. The script prints a working directory at the end. If you're not asking follow-ups, Claude removes it.

Frame budget — why it matters

Token cost is dominated by frames. Every frame is an image; image tokens add up fast. The script's auto-fps logic exists so you don't blow your context budget on a sparse scan of a 30-minute video that would have been better answered by a focused 30-second window.

Duration	Default frame budget	What you get
≤30 s	~30 frames	Dense — basically every key moment
30 s - 1 min	~40 frames	Still dense
1 - 3 min	~60 frames	Comfortable
3 - 10 min	~80 frames	Sparse but workable
> 10 min	100 frames (capped modes)	"Sparse scan" warning — re-run focused, or `--detail token-burner` for full uncapped coverage

When the user names a moment ("around 2:30", "the last 30 seconds", "from 0:45 to 1:00"), pass --start / --end. Focused mode gets denser per-second budgets, capped at 2 fps. Far more useful than a sparse pass over the whole thing.

Frame deduplication

Frame selection — keyframes (efficient), scene-change detection (balanced/token-burner), or the uniform sampler it falls back to — can still surface near-identical frames: a screen recording that holds one slide for 90 seconds produces a dozen, each billed as a separate image. A dedup pass drops them before frames reach Claude. It runs by default on every frame mode (--no-dedup turns it off):

One ffmpeg call scales each extracted JPEG to a 16×16 grayscale thumbnail. Everything after is pure-stdlib Python — no image libraries.
For each frame, compute the mean absolute difference against the last frame that was kept (average per-pixel brightness change, 0–255 scale).
If that difference is at or below the threshold (2.0), the frame is a near-duplicate and is dropped. Otherwise it's kept and becomes the new reference.
The frame-budget cap applies after dedup, so the budget is spent on distinct frames.

Comparing against the last kept frame (not the previous one) catches slow fades that never trip a frame-to-frame threshold. The threshold is deliberately low and measures absolute brightness rather than structure, so a one-line code diff, a terminal scrolling a row, or two differently-colored flat slides all survive.

The Frames line reports what was collapsed, e.g. 6 selected from 14 candidates (… 8 near-duplicates dropped …). On always-moving footage nothing is dropped and you pay what you would have anyway.

Detail modes — measured

The --detail dial trades speed and token cost for visual fidelity. Numbers below are from a real run against a 49:08 YouTube video (1280×720, English auto-captions) — a long, mostly-static screen recording, the case that stresses the caps hardest. Extraction times are local CPU against a pre-downloaded copy; the one-time download was ~37 s / 76 MB, shared by the three frame modes.

Mode	Engine	Frames	Cap	Extraction time	Temporal coverage	Est. image tokens
`transcript`	none (captions)	0	—	~4.5 s (one yt-dlp call, no download)	full (text)	0 (≈26.6k text tokens)
`efficient`	keyframe (`-skip_frame nokey`)	50	50	~0.5 s	0:00 → 49:04 (full)	~9.8k
`balanced`	scene-change	100	100	~20.9 s	0:00 → 48:38 (full)	~19.7k
`token-burner`	scene-change	116	uncapped	~21.0 s	0:00 → 48:38 (full)	~22.8k

Image tokens use Anthropic's (width × height) / 750 — at the default 512px width these 720p frames are 512×288, ≈197 tokens/frame; --resolution 1024 roughly 4×s that. The transcript is surfaced in every captioned mode and on long videos is often the larger cost.
One sampling rule across frame modes. Each detects all candidates across the full range, then even-samples (first + last always kept) down to its cap. The modes differ only in candidate source (keyframes vs. scene cuts) and cap, never in how coverage is spread — so the last frame always lands at the end, not partway through.
efficient is the speed tier (~0.5 s) — it only reconstructs keyframes, so it's ~40× faster than the scene modes, which decode every frame to find cuts. It can also return more frames than balanced on low-motion footage (keyframes outnumber scene cuts); "efficient" means fast extraction, not fewer frames.
token-burner only diverges from balanced past the cap. This clip had 116 cuts, so balanced sampled 100 and token-burner kept all 116. On high-motion video with hundreds of cuts, token-burner keeps everything (and trips the >250-frame token warning) while balanced thins to 100.

End-to-end from a cold URL, transcript is the cheapest mode by far; the frame modes add the shared ~37 s download on top of the extraction times above.

Install

Surface	Install
Claude Code	`/plugin marketplace add bradautomates/claude-video` then `/plugin install watch@claude-video`
Codex, Cursor, Copilot, Gemini CLI, +50 more	`npx skills add bradautomates/claude-video -g`
claude.ai (web)	Download `watch.skill` → Settings → Capabilities → Skills → `+`
Manual / dev	`git clone` then symlink `skills/watch` into your host's skills dir (see below)

Claude Code

/plugin marketplace add bradautomates/claude-video
/plugin install watch@claude-video

Update later with /plugin update watch@claude-video.

Codex, Cursor, Copilot, Gemini CLI, and 50+ other hosts

The Agent Skills CLI installs the skill into whatever agents it detects:

npx skills add bradautomates/claude-video -g

-g installs globally for your user (~/.codex/skills, ~/.cursor/skills, etc.); drop it to install into the current project instead. Useful flags:

-a, --agent <names…> — target specific hosts, e.g. -a codex -a cursor
-l, --list — list the skills in this repo without installing
--copy — copy files instead of symlinking (for filesystems without symlink support)

The CLI discovers the skill from skills/watch/SKILL.md and copies the whole folder — SKILL.md plus its scripts/ runtime — as a self-contained unit. SKILL.md resolves its own scripts relative to wherever it was installed, so it works the same on every host.

Update later with npx skills update watch -g.

claude.ai (web)

Download watch.skill from the latest release.
Go to Settings → Capabilities → Skills.
Click + and drop the file in.

Enable "Code execution and file creation" under Capabilities first — the skill shells out to ffmpeg and yt-dlp, so it won't run without it.

Manual (developer)

Clone the repo and symlink the self-contained skill folder into your host's skills directory — the symlink keeps the install in sync with your working tree as you edit:

git clone https://github.com/bradautomates/claude-video.git
ln -s "$(pwd)/claude-video/skills/watch" ~/.claude/skills/watch   # or ~/.codex/skills/watch

For claude.ai, build the .skill bundle from source: bash skills/watch/scripts/build-skill.sh produces dist/watch.skill.

First run

On the first /watch call, the skill runs scripts/setup.py --check. If ffmpeg / yt-dlp aren't on your PATH, or no Whisper API key is set, it walks you through fixing it:

macOS — auto-runs brew install ffmpeg yt-dlp.
Linux — prints the exact apt / dnf / pipx commands.
Windows — prints the winget / pip commands.
API key — scaffolds ~/.config/watch/.env (mode 0600) with commented placeholders for GROQ_API_KEY (preferred) and OPENAI_API_KEY.

After setup, preflight is silent and /watch just works. The check is a sub-100ms lookup, so it doesn't slow you down on subsequent runs.

Bring your own keys

Captions cover the majority of public videos for free. The Whisper fallback only kicks in when a video genuinely has no caption track — typically local files, TikToks, some Vimeos, and the occasional caption-less YouTube upload.

Capability	What you need	Cost
Download + native captions	`yt-dlp` + `ffmpeg`	Free
Whisper fallback (preferred)	Groq API key — `whisper-large-v3`	Cheap, fast
Whisper fallback (alt)	OpenAI API key — `whisper-1`	Standard pricing
Disable Whisper entirely	`--no-whisper`	Free, frames-only when no captions

Usage

/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?
/watch https://www.tiktok.com/@user/video/123 summarize this
/watch ~/Movies/screen-recording.mp4 when does the UI break?
/watch https://vimeo.com/123 what tools does she mention?

Focused on a specific section — denser frame budget, lower token cost:

/watch https://youtu.be/abc --start 2:15 --end 2:45
/watch video.mp4 --start 50 --end 60
/watch "$URL" --start 1:12:00            # from 1h12m to end

Other knobs (passed to scripts/watch.py):

--detail transcript|efficient|balanced|token-burner — fidelity/speed dial. transcript skips frames (transcript only); efficient uses fast keyframes (cap 50); balanced uses scene-aware frames (cap 100); token-burner is scene-aware and uncapped.
--timestamps T1,T2,… — grab a frame at each absolute timestamp (SS/MM:SS/HH:MM:SS). Claude reads the transcript first, then targets the moments the presenter flags ("look here", "as you can see"). Added on top of the detail frames (reserved against the cap); out-of-window cues are dropped in focus mode; with --detail transcript these become the only frames.
--max-frames N — lower the frame cap for a tighter token budget.
--resolution W — bump frame width to 1024 px when Claude needs to read on-screen text (slides, terminals, code).
--fps F — override the auto-fps calculation (still capped at 2 fps).
--whisper groq|openai — force a specific Whisper backend.
--no-whisper — disable transcription entirely; frames only.
--no-dedup — keep near-duplicate frames. By default a frame-delta pass drops frames that are visually near-identical to the one before them (held slides, static screen recordings, paused video), so the frame budget is spent on distinct content; this flag turns that off.
--out-dir DIR — keep working files somewhere specific (default: auto-generated tmp dir).

Limits

Long-video accuracy depends on the detail mode. On the capped modes (efficient, default balanced) coverage thins out past ~10 minutes — the frame cap spreads across the whole clip, so the script prints a "sparse scan" warning and you're better off re-running focused with --start/--end. token-burner lifts the cap and keeps every scene-change frame across the full video, so it stays complete on longer clips at the cost of more image tokens. The 10-minute mark is guidance for the capped modes, not a hard ceiling.
Detail is one dial. Defaults are balanced: scene-aware frames, 2 fps max, 100-frame cap. Use --detail efficient for a fast 50-frame keyframe pass, or --detail token-burner for uncapped scene candidates. Set WATCH_DETAIL in ~/.config/watch/.env to change the default.

Structure

.
├── skills/watch/                 # self-contained skill — copied as a unit by every installer
│   ├── SKILL.md                  # skill contract — the source of truth across all surfaces
│   └── scripts/
│       ├── watch.py              # entry point — orchestrates download → frames → transcript
│       ├── download.py           # yt-dlp wrapper
│       ├── frames.py             # ffmpeg frame extraction + auto-fps logic
│       ├── transcribe.py         # VTT parsing + dedupe + Whisper orchestration
│       ├── whisper.py            # Groq / OpenAI clients (pure stdlib)
│       ├── config.py             # shared config (~/.config/watch/.env)
│       ├── setup.py              # preflight + installer
│       └── build-skill.sh        # build dist/watch.skill for claude.ai upload (dev-only)
├── hooks/                        # SessionStart status hook (Claude Code only)
├── .claude-plugin/               # plugin.json + marketplace.json (Claude Code)
├── .codex-plugin/                # plugin.json — Codex/agents manifest ("skills": "./skills/")
├── .agents/plugins/              # marketplace.json — Agent Skills marketplace listing
├── AGENTS.md → CLAUDE.md         # generic-agent entry point
├── tests/                        # pytest suite (ffmpeg-synthesized clips, no network)
└── .github/workflows/            # release.yml — auto-builds watch.skill on tag push

Develop

# Run the test suite (stdlib + pytest; ffmpeg required for frame tests):
python3 -m pytest -q

# Build the claude.ai upload bundle:
bash skills/watch/scripts/build-skill.sh      # → dist/watch.skill

Releasing: tag vX.Y.Z, push the tag. The workflow builds dist/watch.skill and attaches it to the GitHub release. Keep the version in sync across skills/watch/SKILL.md, .claude-plugin/plugin.json, and .codex-plugin/plugin.json.

See CHANGELOG.md for version history.

Open source

MIT license.

Built on yt-dlp, ffmpeg, and Claude's multimodal Read tool. Whisper transcription via Groq or OpenAI.

Built by Brad Bonanno — I make content about building with AI on YouTube (@bradbonanno), and build AI operating systems for businesses at Solaris Automation. If /watch saves you from scrubbing through a video, come say hi on the channel.

Star History

github.com/bradautomates/claude-video · @bradbonanno · Solaris Automation · LICENSE

HKUDS/Vibe-Trading

"Vibe-Trading: Your Personal Trading Agent"

English | 中文 | 日本語 | 한국어 | العربية

Vibe-Trading: Your Personal Trading Agent

One Command to Empower Your Agent with Comprehensive Trading Capabilities

Website · Docs · News · Features · Shadow Account · Demo · Quick Start · Examples · API / MCP · Roadmap · Contributing

📰 News

⚠️ Security warning: The X account VibeTrading_HKU, Virtuals project 101845, and token contract 0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4 are not official Vibe-Trading assets. We have never launched or endorsed any token or memecoin. Do not buy, connect a wallet, or sign anything. Details.

2026-08-09 🪟 Secure Windows packaging, Canada markets, ModelScope, and Alpha Zoo over MCP: Windows desktop packaging now assembles a checksum-pinned embedded Python 3.12 runtime and x64 NSIS review/signing paths, plus Electron safeStorage for an allowlisted credential set. The renderer can set or clear secrets but never read them; plaintext configuration migrates once; decrypted values reach only the owned backend; and both unsigned review and signed builds fail closed on the wrong signature state. No installer artifact was published from this PR (#1015). Canadian equities now work end to end: .TO/.V symbols are classified in CAD, route through Yahoo → yfinance → local fallback, execute under Canada-specific GlobalEquity rules, benchmark against XIC.TO, and refuse mixed-currency aggregation. Strict USD-M historical backtests can also opt into position_adjustment=rebalance while preserving collateral, funding, fees, realized P&L, liquidation behavior, and immutable fill evidence across increases and reductions (#1024, #1019, closes #952). ModelScope joins the built-in providers through its official OpenAI-compatible hosted-inference endpoint, with Qwen/Qwen3.5-27B as the default (#1011); the new vibe-trading update distinguishes wheel installs from editable/source checkouts, installs the exact release it checked, and verifies fresh metadata without downgrading (#1020); and alpha_zoo plus bounded alpha_bench now reach MCP (64 tools), with horizon/result/output-path limits and safe report creation (#979). Verified Python and frontend lock refreshes also update grouped dependencies, postcss, and akshare (#1021, #1023, #1026, #1027). Thanks @QCYTSN, @wiliao, @honginp, @yrk111222, @zwrong, and @cgycorey.
2026-08-08 🧱 Desktop shell, eToro, atomic rebalancing, and a broad reliability pass: A source-first Electron host now owns the existing backend lifecycle — random loopback port, per-launch secret, five-locale startup recovery, and owned-process cleanup — while eToro joins with path-separated demo/real profiles; live risk-increasing actions remain mandate-gated and audited, and API capability surfaces are authenticated under enforced CSP (#923, #989, #961). Backtests gain opt-in atomic same-direction rebalancing with immutable fill evidence; Shadow splits mixed markets by settlement currency without invented FX aggregation and honors the configured runtime root; indicators use consecutive unsampled history; negative-equity drawdown and empty insolvent cross accounts are handled correctly (#951, #997, #1017, #1005, #958, #959). OpenAI Codex OAuth gets a separate synchronized credential store and one-shot 401 recovery; proxy opt-out covers sync and async clients; sandboxed runs retain their canonical root; scheduled research isolates malformed records and fixes interval-timezone validation; lowercase 4h requests return true four-hour bars (#1014, #995, #1012, #1003, #1004, #1013). QQ replies retain source message IDs, long model slugs remain readable, and the agent stops when evidence is sufficient (#1008, #1006, #1010). Thanks @QCYTSN, @Shizoqua, @ngoanpv, @hhj123123, @su322, @Robin1987China, @shadowinlife, @dineeshd, @honginp, @santhreal, @00EVA, @x-lambda, @ofeksh-tr.
2026-08-07 🛡️ Fewer false refusals, a closed sandbox gap, QVeris on MCP: The grounding gate stops rejecting well-formed answers over numbers that were never prices — confidence scores, indicator readings, moving-average windows, year-less dates like 8/5, percentage ranges, and a trading plan's own trigger levels (close ≥ 6.45 is a condition, not a quote) — while a quote outside recorded OHLC evidence is still refused, and a price table dated 08-05 now matches its evidence instead of every cell coming back unavailable (#1001, #983). Sandbox: generated strategy code can no longer import the broker layer, nor reach socket/subprocess/os.system/ctypes through a renamed binding — both were accepted before, and src.quantlib still imports. QVeris discovery/inspect/execute join the MCP surface (62 tools), with the cost quote read from the marketplace instead of trusted from the caller (#976, closes #964, thanks @shadowinlife). Plus HK market-data fallback routing repaired with a new Tencent HK source, yfinance crypto routed to the crypto engine, memory entries written and recovered with their .md suffix, MCP list/dict arguments tolerating JSON-string clients, and Portfolio Studio artifacts surfaced in run detail (#1000, #970, #984, #993, #980, #982, #966, #973, thanks @he-yufeng, @ngoanpv, @sambazhu).

Earlier news

2026-08-06 🧮 A tested finance-math layer + valuation engine + irregular cash flows + wired-in governance: src/quantlib replaces the formulas that lived as markdown inside skills with one tested implementation each — options, bonds, credit, econometrics, VaR/CVaR/EVT, attribution, event studies, multiple-testing control, purged cross-validation — ~250 functions, reachable from Web/API/MCP via the new read-only quantlib_call tool. A valuation engine (run_dcf / run_comps / three-statement) refuses to run on a missing input instead of silently defaulting it, and a new entity + cash-flow spine admits NAVs, capital calls, and coupons (XIRR/MOIC/DPI/TVPI and TWR/Modified Dietz via cashflow_performance; crypto L2 impact cost via orderbook_depth). Every run now writes a hash manifest, the audit ledger is hash-chained so tampering is detectable, and all 30 swarm presets were re-audited — a deliverable no granted tool can compute is now declared as such instead of invented.
2026-08-05 🔭 Institutional holdings, ETF look-through, prediction markets, research papers: Four read-only data tools, all on free public sources — SEC 13F books with quarter-over-quarter position diffs; ETF constituents across markets (a CSI-300 tracker resolves to 342 positions covering 98.7% of net assets, not the quarterly top ten); event contracts as labelled implied probability; and arXiv/OpenAlex search that marks what a source does not state instead of inferring it. Plus five scheduled-research templates, six institutional commands (/comps /dcf /attrib /memo /earnings /screen), investor lenses as a standalone skill, and an agent core that traces every number back to the tool that produced it.
2026-08-04 🔧 Correctness pass: fundamentals, A-share prices, oversized results: SEC reporting periods are now keyed on their (start, end) span — a 10-Q files the true quarter and the year-to-date frame under the same end date and fiscal period, so period="annual" had been returning a single quarter for AAPL FY2018–2020 (a 4.2× understatement) and every fiscal-Q4 slot in a quarterly series carried the full-year figure; get_fundamentals("AAPL.US") no longer answers ok:true with an all-null panel. Tushare A-share prices are now corporate-action adjusted in both the factor bench and backtests — a raw close-to-close return across an ex-date was off by up to 47 percentage points (300750.SZ, 2023-04-26) — and the CSI300 bench masks each date to its point-in-time index membership. Cross-market composite backtests refuse a mixed-currency code set instead of summing CNY, USD and KRW into one equity curve; option legs are marked at the volatility they were opened at, removing a fabricated day-zero P&L of up to +93% of premium; oversized tool results are paged by whole record with an explicit total instead of being cut mid-JSON; and calc_metrics reports tracking error and benchmark beta.
2026-08-03 ⏰ Timezone-aware scheduled research + unblocked stock screening: Scheduled jobs now take an optional IANA timezone and evaluate cron on that zone's wall clock, so a cadence survives DST — a spring-forward gap is skipped and a fall-back ambiguous time runs once — while cron fields gain comma lists and ranges (1,3-5), jobs without a timezone keep UTC semantics, and the web UI gains a Scheduled page in all five locales where it previously had no scheduling surface at all (#954, closes #953, thanks @ngoanpv). A screening request no longer dead-ends: a many-candidate shortlist counts as an answer rather than a stalled resolution and retires once a candidate is locked, and price validation stops reading ticker digits, localized dates, share counts, and position costs as quoted prices — while still refusing any quote outside recorded OHLC evidence (closes #955). Agent memory also gets exact index-anchor matching and a respected result bound (#956, #957, thanks @santhreal).
2026-08-02 🧠 Live model discovery, truthful runtime identity, and a verified dependency refresh: Settings now discovers configured-provider models on demand with stable warning codes and five-locale controls, while each reply records and reloads the immutable provider/model/reasoning identity that actually served it—cleared safely when sessions change (#924, thanks @QCYTSN). Nine hash-locked Python updates plus jsdom/postcss also landed with exact-version imports, 330 focused tests, the production build, 373 frontend tests, full main CI, and Dependency Graph green (#949, #948); the breaking MCP 2.0 bump remains unmerged pending a complete lock/runtime migration (#950).
2026-08-01 🧮 Options strategy analytics + market sentiment + auditable USD-M research: A new options payoff workflow analytically calculates expiry P&L extrema, exact breakevens—including continuous zero-P&L intervals—engine-aligned entry commissions, and spot × IV scenarios through Agent and MCP (#946, rebuilt from #883, thanks @he-yufeng). The read-only sentiment tool scores arbitrary text locally and retrieves the crypto Fear & Greed Index without an API key (#939, thanks @Robin1987China). Strict USD-M backtests now persist ordered fill, funding, risk, and liquidation events plus a fidelity summary, while rejecting unsupported 100× intervals (#936, thanks @honginp). Reliability improvements also ensure symbol and venue resolution precedes market-data calls, final quoted prices are checked against recorded OHLC evidence, scheduled research retries transient failures, and nested MCP results serialize cleanly.
2026-07-31 🔧 USD-M liquidation lifecycle + technical indicators + user-level state dirs: Opt-in perpetual_strict mode settles historical funding before fills and executes isolated/cross margin breaches as real liquidations (#903, thanks @honginp). A read-only technical_indicators tool computes RSI/MACD/Bollinger/SMA/EMA through the existing loaders (#921, refs #920, thanks @Robin1987China). Sessions, runs, swarm runs, and uploads now live under ~/.vibe-trading (relocatable via VIBE_TRADING_HOME) with a one-time automatic migration (#925, closes #904, thanks @MuggleJinx). Plus ten correctness fixes — Yahoo .SS classified as A-share, bare/prefix-style A-share codes, slash-delimited crypto pairs, nan/inf guards (#919, #926–#935, thanks @santhreal).
2026-07-30 🎨 Rebuilt WebUI + Korea (KRX) market + an OpenBB Workspace bridge: The web UI lands its guided-minimalism overhaul — no first-frame flash, one durable activity object per turn with a live reasoning whisper and a reload-safe tool trail, LLM-written session titles, full five-locale parity. Korea equity (KRX: KOSPI/KOSDAQ) becomes the 9th backtest engine — execution-time ±30% band, long-only, 2026 0.20% transaction tax, optional pykrx loader (#693, thanks @JungHoonGhae) — plus an OpenBB Workspace bridge (#817, thanks @shugaoye) and a read-only Taiwan snapshot tool (#848, thanks @TSENGCHIENFENG). Correctness: daily price bands are judged at execution time, not from the decision bar's close; a session runs one attempt at a time (HTTP 409) and a user stop is its own terminal state (#676, thanks @tyj147454413-cmd). Plus durable traces (#662), secret-scrubbed tool results (#675), fail-closed tool arguments (#913/#911, thanks @santhreal), direct-OpenAI reasoning_effort (#755, thanks @1anter), and numeric guards across the risk x-ray / edge density / options engine (#909/#908/#907).
2026-07-29 🔧 Gap-safe returns + liquidation risk modeling + a risk x-ray in every run: bar_returns no longer erases the real move across a trading halt longer than the forward-fill window — the resumption move was silently recorded as 0, understating volatility and inflating Sharpe — and an inf prior price can no longer read as a clean −100% (#895, thanks @darkknight4563). Annualisation now covers all 24 data sources at every interval, with a coverage test that fails CI when a loader lands without entries (#891, closes #884, thanks @Robin1987China). USD-M perpetual research gains deterministic isolated & cross margin liquidation evaluation (#889, thanks @honginp), and every portfolio backtest now emits risk x-ray artifacts (risk_xray.json/.md) with headline concentration/vol/drawdown metrics (#900, thanks @he-yufeng). The connector CLI now loads ~/.vibe-trading/.env, so env-sourced broker credentials resolve again (#902, closes #901, thanks @MuggleJinx). Plus indent-preserving channel message splits and skill-frontmatter parsing at EOF (#867/#861, thanks @santhreal).
2026-07-28 🔧 Next-gen Claude models unblocked + sign-safe returns: Claude models that deprecate the temperature field (opus-4-7, opus-5, sonnet-5) now work — the adapter drops the field when the API rejects it, retries once, and remembers the model, so no per-release patch is needed (#890, closes #856, thanks @yagnikpipaliya). Non-interactive vibe-trading run now injects a host session id: research-goal tools previously failed on every call while the run still reported success (#885). Buy-and-hold returns are sign-safe — a near-zero prior close no longer explodes the compounded benchmark, and an exact-zero close no longer yields inf/nan (#872, thanks @darkknight4563). The frontend moves to Node 22 + React Router 8, clearing a high-severity advisory.
2026-07-27 🔧 Correlation integrity + vn.py 4.0 export repair + an encoding batch: The rolling correlation matrix no longer forward-fills missing closes — a halted session was being scored as a fabricated 0% return against the peer's real move, distorting the matrix (#873, thanks @ddy4633). The vn.py export skill is repaired for the vn.py 4.x layout, where vnpy.app.cta_strategy no longer exists upstream — templates now import from vnpy_ctastrategy (#869, thanks @y85998607). Plus a six-fix batch: UTF-16 BOM decoding in the document reader and trade-journal CSVs, currency symbols stripped before numeric coercion, BTCUSDT-style symbols inferred as crypto, lowercase 1h/1d intervals annualized correctly, and CJK characters preserved in skill directory slugs (#862, #863, #864, #865, #866, #868, thanks @santhreal).
2026-07-26 🔒 Dependency lock + universe transparency: Docker’s hash-locked install works again, with a new CI lock check (#858, closes #847). alpha bench now discloses CSI300/SP500 sources, counts, degraded fallbacks, and survivorship bias (#859, closes #845). Actions and five frontend dependencies were also refreshed (#850–#852).
2026-07-25 🔧 Perpetual realism + MCP crash fix + a correctness batch: USD-M perpetuals gain margin state contracts (#798, thanks @honginp) and the engine now consumes historical funding rates instead of fetching-and-ignoring them (#819, thanks @g0rdonL). MCP dataclass results no longer crash on a false Circular reference detected (#849, thanks @Echoandelementwebsites), and alpha bench CLI/HTML forward the _meta survivorship disclosure (#841, closes #797, thanks @AmirF194). Plus 12 correctness fixes across journals, connectors, and channels (#799–#810, thanks @santhreal), and a real account label in CLI balances (#843, closes #846, thanks @Robin1987China).
2026-07-24 🔀 Memory Tier 2, composable optimizer constraints + an interval-handling sweep: Persistent memory gains Tier 2 structural organization (#815, thanks @shadowinlife), and backtest optimizers accept composable weight constraints (#818, thanks @he-yufeng). Correctness: the daily-bar validator can opt in to non-positive prices — opening on negative bars while still rejecting zero (#816, closes #571, thanks @darkknight4563). Plus a 19-PR loader interval-normalization sweep: lowercase 1h/4h/1d/1w aliases accepted everywhere, unsupported intervals now fail fast instead of silently returning daily bars, Yahoo 4H maps to 1h, and MT5 accepts 1W/1M (#812–#838, thanks @santhreal), a trade-journal fix for Eastmoney Excel-serial dates (#811, thanks @santhreal), and a README nav-anchor fix (#840, thanks @dvirarad).
2026-07-23 🔧 Reliability sweep + strict alpha-bench surfaced + opt-in memory lifecycle: A 22-PR contributor batch. A broad reliability sweep fixes timeframe handling end to end — yfinance 1M→monthly (not minute), CCXT 1W/1M, akshare/india-broker rejecting unsupported intervals instead of silent daily, and the Tiger/Alpaca/OKX/Shoonya/Longbridge connectors keeping 1H/4H as hour bars — plus trade-journal Excel-date normalization (eastmoney float YYYYMMDD, Futu/Tonghuashun serial dates), finite-JSON report_audit, blank holding_days validation, and Feishu/CLI markdown table edges (#778–#794, thanks @santhreal). MT5 trading_history now coerces numpy scalars so JSON serialization no longer dies on int64 (#776, closes #774, thanks @shadowinlife), and PIT fundamentals dedup restated rows and stop the snapshot regressing to an older fiscal period on a late restatement (#772, closes #771, thanks @klmtseng). New: alpha bench --strict finally wires the strict same-universe random-control + OOS gate that shipped unreachable since 0.1.9 (#796, closes #773, thanks @he-yufeng), an opt-in memory lifecycle (quality scoring, Ebbinghaus decay, archive-only GC — all off by default) (#733, closes #732, thanks @shadowinlife), and backtest rebalance-notes artifacts + turnover metrics (#795, thanks @he-yufeng).
2026-07-22 🚀 v0.1.12 released (Release notes, pip install -U vibe-trading-ai): The correlation regime timeline adds a GET /correlation/regime endpoint + an opt-in Correlation-tab strip — edge density run through a causal hysteresis state machine that marks FUSED market episodes, descriptive risk context rather than a signal (#756, closes #719, thanks @ebujinovch). Provider endpoint resolution now falls back to each provider's canonical base URL and gracefully handles non-SSE endpoints, fixing the native zai provider on glm-5.1 (#758). Plus a strict-JSON / finite-number reliability sweep across metrics, factors, pattern, session, and journal (#761–#770, thanks @santhreal) and a Binance maintenance-bracket decouple that keeps -PERP backtests zero-credential (#757, thanks @honginp). Rolls up ~90 fixes since 0.1.11.
2026-07-21 🔧 Data-loader completeness + a reliability fix sweep: Partial market-data results now complete the missing symbols through the fallback chain and fail closed instead of silently shrinking the backtest universe (#689, closes #681, thanks @xkam7ar), and OKX bars use the history-candles endpoint with rate-limit retry for deep backfills (#644, thanks @tyj147454413-cmd). Plus a fix sweep: the MCP network guard accepts IPv6 / case-variant hosts (#750, thanks @Robin1987China), trade-journal parsers skip blank/NaN symbol rows (#749, thanks @Robin1987China), the Shadow Account skips the mined entry-hour gate on daily bars (#748, thanks @Robin1987China), and MiniMax regional API endpoints are selectable (#731, thanks @octo-patch).
2026-07-20 🔀 Providers, MetaTrader 5, and a reliability sweep: Native Anthropic Messages API (optional [anthropic] extra, #695, thanks @jelech), SiliconFlow (#565, thanks @UNHNQ), and iFlytek Spark (#537, thanks @FenjuFu) join the provider roster, and a MetaTrader 5 (Exness) broker connector + mt5 forex/metal data source lands (broker connectors → 12, #481, thanks @StaniellG). Plus a provider-agnostic llm-vision OCR engine (#548, thanks @shadowinlife), an 80× signal-alignment vectorization (#698, thanks @shadowinlife), historical Binance USD-M funding/bracket data (#716, thanks @honginp), a swarm MCP-discovery cache (#704), and a reliability consolidation closing 13 SSE/session/CLI/swarm/scheduler issues (#584, thanks @xkam7ar). Correctness: options partial-close now honors the requested quantity instead of flattening the lot (#577), centralized provider credential resolution (#563), queued-cancel handling (#641), a frontend streaming-DOM race (#717, thanks @Marnie0415), and the connector CLI renderers (#726, thanks @nareshkps).
2026-07-19 🔧 Real US/HK stock-news articles + MCP factor-analysis fix + a robustness pass: The stock-news tool now returns real Yahoo Finance articles (title/url/source/published/snippet) for US and HK tickers instead of related-instrument matches, still routed through the frozen IP-throttled client (#730, thanks @yxhuang). The MCP factor_analysis tool is realigned to the registered tool's real CSV contract, so calls no longer die on KeyError before running (#715, closes #635, thanks @Robin1987China). Plus a robustness pass: the whole Kimi K-series (k2/k3/…/for-coding) now auto-forces temperature=1 as the API requires (#701, thanks @sambazhu), and split_message, PDF page ranges, and trade-journal date filters all fail fast on degenerate or inverted input instead of hanging or silently returning nothing (#727–#729, thanks @santhreal).
2026-07-18 🔧 Binance crypto fallback + parallel-execution and correctness fixes: A Binance loader joins the crypto historical-data fallback chain (#643, thanks @tyj147454413-cmd), and the IBKR connector moves to a thread-local connection pool with snapshot quotes, fixing hangs under parallel agent runs (#636, thanks @MikeCer). Plus a correctness pass: factor analysis rejects non-positive n_groups, inverted period ranges and non-positive detection windows fail fast, an unnamed DatetimeIndex in the correlation matrix is handled, equity.csv nav/value column aliases are accepted, and empty A-share codes are no longer coerced to 000000.SZ (#709–#714, thanks @santhreal). A correlation-rewiring stability factor joins the academic zoo (#705, thanks @ebujinovch), the fundamental zoo is whitelisted for factor analysis (#707, thanks @sambazhu), persisted run state is now fsync-durable (#645, thanks @tyj147454413-cmd), and the dev extra installs the documented Black/Ruff toolchain (#634, thanks @xkam7ar).
2026-07-17 🧩 Correlation-regime skill + a broad backtest / data / live-safety correctness pass: a new correlation-regime detection skill (bundled skills → 88, #557, thanks @ebujinovch), a Longbridge runtime connection card (#569, thanks @fanfpy), and user-defined swarm presets loaded from ~/.vibe-trading (#570, thanks @darkknight4563). Plus hardening across the stack: silent-data-corruption fixes in the Futu / Tencent / CCXT / mootdx loaders, look-ahead-bias and strict-OOS guards in the factor bench and Shadow Account, live-trading safety (signed exposure caps, atomic daily order limits, consent-first mandate commits, fail-closed live state), and journal / QVeris-budget / swarm / CI-gate improvements (#552, thanks @xor-xe; much of the correctness work by @xkam7ar).
2026-07-16 🔧 Dependency lock repaired + Windows settings save fix: the hash-verified runtime lock is regenerated so Docker's pip install --require-hashes resolves cleanly again, fixing the incompatible caio/pydantic-core/websockets pins (#564, closes #558, thanks @tianrking). Saving Agent LLM settings from the Web UI no longer returns HTTP 500 on Windows — the POSIX-only os.fchmod hardening is now platform-guarded, with a regression test for platforms without fchmod (#561, thanks @CRui5in).
2026-07-15 🧮 Backtest correctness + Portfolio Studio core: A 10-PR convergence pass made rebalances causal and order-independent, charged terminal close costs, reported fill-derived turnover, enforced exposure caps, and kept validation output finite and strict (#530/#531/#532/#540). Charts now reuse the run's actual data source, repeatable market queries are no longer dropped, and .env loads refresh cached config (#535/#544/#554). Portfolio Studio #456 and config bug #541 are closed; provider fixes #528/#529 closed too. Thanks @YZY0108, @santhreal, @Robin1987China, @xkam7ar, @Marnie0415, and @marichu99.
2026-07-14 🌉 Longbridge market data + modern MCP transport + provider reliability: Longbridge joins the historical-data fallback layer with key-gated credentials, date-window splitting, strict completeness checks, and an opt-in SDK dependency; four China-market flow tools gain verified Tushare fallbacks, and negative final equity no longer crashes backtest metrics. The MCP server now supports Streamable HTTP, write_file safely recovers aliased or missing path arguments, hypothesis updates reject unsupported fields, and Correlation requests are authenticated. NVIDIA NIM is now a first-class provider across Web Settings and both CLI onboarding paths, with a versioned compatibility User-Agent to address the reported 403; Web Settings now writes to the canonical ~/.vibe-trading/.env, migrates legacy configuration, and reports permission failures clearly, fixing the DeepSeek save-time 500 (#534, closes #516/#524; #528/#529). Thanks @fanfpy, @asahikiko, @santhreal, @sTunnaSu, @abhishekjaisinghani, @huangcheng, @ShiroKSH, @Meru143, @DIEGOD79, and @not-knope for the code, reports, and diagnosis.
2026-07-13 🔒 Security hardening: all 10 external-audit findings closed + contributor batch: every finding from the 2026-07-10 external security audit (issue #476, discussion #468) is now addressed on main — Docker multi-stage rebuild with digest-pinned images, an AST-hardened backtest sandbox blocking network/subprocess/eval/os.environ/unsafe-open (including inside nested function bodies), short-lived single-use SSE auth tickets, hardened Compose (read-only rootfs, dropped capabilities, resource limits), auth + rate limiting on /correlation, security headers, hash-locked dependencies, and more. Also merged: opt-in TAP mode for Alpaca key isolation (#377, thanks @0xZKnw), realized portfolio turnover surfaced in backtest metrics (#478, thanks @Robin1987China), a Frazzini-Pedersen betting-against-beta academic factor (Alpha Zoo → 461, #480, thanks @YogeshModi24), a look-ahead-bias fix across all 5 portfolio optimizers (#487, thanks @YZY0108), and two preflight/provider-config fixes (#479/#484, closes #477/#482, thanks @ananaymital/@Bortlesboat).
2026-07-12 🧪 Strategy Development Manager + contributor fix batch: the new strategy-dev-manager skill (#87) turns academic papers and broker research into registered factors/strategies with a persistent artifact store and automated IC/Sharpe decay monitoring — sdm_register / sdm_status / sdm_decay_scan drive an active → monitoring → decayed → disabled lifecycle over ~/.vibe-trading/ (#457, closes #455, thanks @shadowinlife). Also merged: the Correlation tab accepts bare tickers (AAPL,SPY) and walks the full loader fallback chain (#472, closes #471, thanks @yxhuang), the local loader honors requested intervals via OHLCV resampling (#467, thanks @Shizoqua), Binance USD-M perpetual history lands with explicit BTC-USDT-PERP routing + execution/mark price separation as the first #462 slice (#470, thanks @honginp), FastMCP transport imports now work across both module layouts (#469, thanks @roberttidball), and Requesty is available as an OpenAI-compatible LLM gateway provider (#474, thanks @Thibaultjaigu).
2026-07-11 🚀 v0.1.11 released (pip install -U vibe-trading-ai): rolls up three weeks since 0.1.10 — first-class Indian equity (NSE/BSE) backtesting, the PIT-safe fundamental factor layer (Alpha Zoo → 460), the 16-adapter IM channel runtime, end-to-end scheduled research, optional QVeris premium data, and today's contributor batch: a turnover-aware optimizer (#466, thanks @Robin1987China), an analyze_image vision tool + NapCat DM pairing + the IM-media read fix (#464/#463/#465, thanks @fei-moss), Longbridge Decimal serialization (#459, thanks @fanfpy), and packaged-manifest count guards (#461, thanks @asahikiko). Full details: CHANGELOG · release notes.
2026-07-10 🇮🇳 Indian equity (NSE/BSE) support + centralized env config: a dedicated IndiaEquityEngine lands — T+1 delivery, circuit bands, and a config-driven STT/stamp/exchange/SEBI/GST cost stack — with .NS/.BO symbol routing, an opt-in read-only Shoonya/Dhan data bridge, and 255 alpha101/qlib158 factors opted into the new equity_in universe (#305, thanks @muku314115). Environment variables now flow through a single Pydantic EnvConfig schema with an AST-based CI gate against future os.getenv sprawl (#440, closes #438, thanks @shadowinlife). Also: a second-confirmation dialog before committing a real trading mandate plus unified error toasts (#453, thanks @wison1717-maker), scheduled-research route tests (#452, thanks @Robin1987China), and GLM thinking models no longer lose their reasoning stream on the zhipu provider (#458).
2026-07-09 🧯 Docker startup unblocked + provider/CLI contributor batch: Docker/server startup no longer crashes when FastAPI route iteration sees an included-router-like entry without path (#450, thanks @Penn-Live). We also landed the queued quick-win contributor fixes: loader fetch() signatures now match the protocol across OKX / Tushare / yfinance (#437, thanks @shadowinlife), the CLI resume prompt preserves the first user message (#448, closes #447, thanks @morluto), Codex OAuth defaults to openai-codex/gpt-5.4 (#446, thanks @morluto), Kimi for Coding is available as a distinct provider (#435, thanks @yxhuang), opencode provider mappings are wired (#444, thanks @imsankz), and Tushare reference code fences now say python instead of pyhton (#449, thanks @flash1234pku). Validation included focused server/CLI/provider/loader tests plus a Docker build and /health smoke.
2026-07-08 💎 Fundamental factor layer (Phase 1) + optional QVeris premium data + maintainer day: PIT-safe SEC fundamentals now flow into daily factor panels — fund:* panel columns, filed-date anchoring with restatement and YTD-frame protection, and 4 new quality/value factors (registry now 460 alphas). Data routing gains an optional premium track: the 18 free sources stay the default, while QVeris unlocks 63+ providers via Settings → QVeris or vibe-trading data mode paid (see the QVeris section below). Also: api_server modularization completed (1,103 → 371 lines, #424 closing #331, thanks @shadowinlife), backtest validation.json no longer requires a pre-existing artifacts dir (#429, thanks @isaveall), clearer --swarm-run errors (#428, thanks @isaveall), and we reverted the governance stack that broke session chats (#433, thanks @yxhuang for the precise diagnosis).
2026-07-07 ✅ Contributor PR batch: merged the queued contributor work for IM channel timeout configuration (#413, thanks @SyntaxSawdust), Alpha Library social previews and the beginner tutorial (#396, #393, thanks @kadaliao), value-investing skills / tools / committee presets (#407, thanks @sambazhu), zero-sized order-field handling in trading_place_order (#417, thanks @irfanallana-oss), and timezone-aware UTC timestamps across session/API paths (#397, thanks @mustafakamal88).
2026-07-06 🧭 Preflight hardening, API slices, and CN search fallback: provider preflight no longer follows redirects (#404, closes #402, thanks @SyntaxSawdust), the remaining API routes moved into focused modules (#387, superseding #383-#386, thanks @shadowinlife), and CN web-search fallbacks now include Alibaba Cloud IQS (#408, thanks @sambazhu). Maintainer cleanup added no-network fallback tests and EOF whitespace cleanup (fbac74f); main CI is green (run 28780619018).
2026-07-05 ✅ Contributor PR queue closed + Windows baseline green: merged the four non-draft PRs selected for today's maintainer pass. A-share mootdx batch pulls now let KeyboardInterrupt / SystemExit propagate instead of being swallowed by a bare except (#399, closes #398, thanks @shadowinlife). The Settings route slice and patched dependency floors are now merged under their original contributor PRs (#382, #390, thanks @shadowinlife and @aeonframework). Windows baseline compatibility now isolates loader caches, makes OAuth cache assertions platform-aware, skips one fork-only mock test on Windows, and bypasses proxies for MCP loopback fixtures (#401, thanks @Elfsa-Miranda). Validation: 4701 passed, 47 skipped.
2026-07-04 🧩 API route slices, tutorial docs, and dependency floors: IM channel and Settings routes moved out of api_server.py into src/api/channels_routes.py and src/api/settings_routes.py, continuing the narrow #331 modularization path from contributor work (#379, #382, thanks @shadowinlife). The wiki gained a Chinese beginner tutorial for non-finance readers (#393, thanks @kadaliao), and dependency floors now keep Pillow / LangChain / LangGraph on the installable patched track (#390, thanks @aeonframework).
2026-07-04 🧹 UTC timestamp cleanup for session and API paths: tightened the #395 timestamp fix so session, goal, channel, and API timestamps now emit timezone-aware UTC values in explicit ISO form.
2026-07-03 🛡️ Robinhood MCP refresh + API modularization + SSRF guard: Robinhood Agentic Trading now uses the current MCP tool names across generic reads, live-runner plumbing, default read-only seeds, and mandate-gate tests, while interactive startup honors the same .env search order as the provider loader (~/.vibe-trading/.env → agent/.env → $CWD/.env) (#391, closes #381 and #380). System routes (/health, /correlation, /system/shutdown, /skills, /api) moved into src/api/system_routes.py as the next narrow API modularization slice (#378, thanks @shadowinlife). Channel media SSRF defenses now reject CGNAT/mesh/non-global targets and QQ media redirects-to-internal before fetching (#389, thanks @hobostay).
2026-07-02 ⚡ Factor acceleration + safer runtime boundaries: hot rolling factor operators now use bottleneck/NumPy fast paths, alpha bench parallelism avoids repeated large-panel worker payloads, and base equity math has regression coverage (#376, closes #339, original work from #342 by @shadowinlife). Upload and Shadow report routes moved out of the monolithic api_server.py as the first narrow API modularization slice while #331 stays open (#375, based on #358, thanks @shadowinlife). Generated backtests now inherit only an allowlisted subprocess environment instead of the parent secrets surface (#374, closes #332), and IM channels gained /new session reset plus case-insensitive pairing commands (#372, closes #371, thanks @shadowinlife).
2026-07-01 🧹 Security polish + tracker cleanup: tightened API/Docker/frontend dev defaults, stabilized Settings channel and zh-CN edges, cleared frontend dependency/CSP alerts, and closed stale WhatsApp + paper-trading tracker items (#338, #351, #349, #365, #367, #350, #335, #283).
2026-06-30 💬 IM channel runtime for research delivery: Vibe-Trading can now attach the same agent session runtime to 16 built-in message adapters — WebSocket, Telegram, Slack, Discord, Matrix, WhatsApp, Signal, QQ/NapCat, WeChat/WeCom, Feishu/Lark, DingTalk, Teams, email, and Mochat. CLI (vibe-trading channels status/start/stop/login/pairing), REST (/channels/status, /channels/start, /channels/stop, /channels/pairing/command), and the Web UI Settings panel expose status, recovery hints, start/stop, and sender pairing; SDK-backed adapters stay behind extras such as vibe-trading-ai[telegram] or vibe-trading-ai[channels] (#341).
2026-06-29 🛡️ Live advisory safety + Trading 212 read-only connector + Windows/Gemini fixes: live order guards now have an opt-in, broker-agnostic PreTradeAdvisoryInterface that records advisory reviews without bypassing the mandate gate, kill switch, or audit trail (#328, closes #317, thanks @shadowinlife). Trading 212 joins the connector layer with read-only account, positions, orders, history, and instrument-metadata support; place_order / cancel_order still hard-refuse until a structural paper/live boundary exists (#321, closes #309, thanks @mvanhorn). Windows startup avoids the pandas 3.0 Timestamp crash via the <3.0.0 constraint (#329, closes #324, thanks @hannibal-lee); Gemini thought_signature dict-history replay was verified/fixed on main (#318); .US financial statements now route to SEC EDGAR instead of Eastmoney (#325); and the Alpha Library landing page got cache/date/selector/noscript/DNS-prefetch hardening while heavier CSP and social-card follow-ups stay tracked (#323).
2026-06-28 🧰 Cross-platform setup/dev + runtime and file-tool hardening: vibe-trading setup and vibe-trading dev now handle Windows TypeScript builds, launch the backend from the right cwd, use the Vite 5899 port, and shut child processes down cleanly (#292, thanks @digger-yu). Runtime status polling now degrades instead of crashing (#322); MCP OAuth cache keys are sanitized (#313); OpenAI defaults and Robinhood agent.json validation were tightened (#319, #320, thanks @mvanhorn); and file tools got isolated read/write roots plus broader sandbox tests (#299, thanks @skloxo).
2026-06-27 🧯 Content-filter resilience + Shadow Account feature contract cleanup: event-driven and swarm runs now skip individual LLM content-moderation hits, warn in run cards when filter rates are high, and recognize Gemini safety finish reasons instead of aborting an entire analysis (#308, closes #307, thanks @shadowinlife). Shadow Account extraction/codegen now share one PRICE_FEATURES contract and keep four-decimal return bounds, preventing rule/codegen drift and precision loss on prior_5d_return (#316, thanks @Robin1987China).
2026-06-26 🎯 Shadow Account conditional entry + tushare ETF/index/HK routing: extracted Shadow Account rules now carry RSI / prior-return bounds, so the generated SignalEngine enters on real conditions (RSI in range, prior-return in range) instead of blindly replaying the holding cadence (#314, follows #302, thanks @Robin1987China). The tushare loader also routes ETF/LOF → fund_daily(), indices → index_daily(), and HK equities → hk_daily() instead of always calling daily() (which silently returns empty for non-stocks), with per-symbol empty-result + partial-fetch warnings (#315, closes #310, thanks @shadowinlife).
2026-06-25 🧪 Strict validation JSON + calmer agent context: standalone backtest validation now normalizes nested NaN / Infinity values before writing artifacts/validation.json or CLI stdout, so strict JSON parsers no longer choke on validation payloads (#306, thanks @gyx09212214-prog). The agent prompt also derives the current data-source count from the loader registry, and _microcompact() now waits for real token pressure instead of clearing older tool results during short runs (#296, closes #282, thanks @MarkfuGod).
2026-06-24 🎯 Shadow Account price context + reactive Chinese UI + LAN auth fix: Shadow Account rule extraction now sees PIT-safe entry context — entry_rsi14 and prior_5d_return fetched through the loader registry as of buy_dt, with graceful offline/no-data degradation (#302, follows #295, thanks @Robin1987China). The main Web UI panels now use reactive English / zh-CN translations across charts, chat, Alpha Library, Correlation, and Run Detail (#301, thanks @skloxo). Remote same-origin Web UI deployments with API_AUTH_KEY can post and upload again after the CSRF hardening, while mismatched cross-site origins remain blocked (#304, thanks @Hinotoi-agent).
2026-06-23 🛡️ Local API CSRF hardening: a malicious web page can no longer drive unsafe cross-site requests (POST/PUT/DELETE) against the loopback API — CORS blocks reading the response but not the side effect, so loopback dev-mode trust now applies the existing cross-site guard to unsafe methods before honoring it. Safe methods and local CLI / non-browser uploads are unaffected (#293, thanks @Hinotoi-agent).
2026-06-22 🔧 Live-authorize OAuth fix + Alpha Zoo headline fix: connector authorize now holds the OAuth handshake open through a multi-minute broker sign-in (tunable via VIBE_LIVE_AUTHORIZE_TIMEOUT_SECONDS) and no longer spawns a competing callback server on retry, so the token actually persists (#281, closes #259, thanks @Robin1987China). The Alpha Zoo page no longer prints its alpha count twice (#287, closes #286, thanks @digger-yu). Scheduled research also picked up end-to-end usage docs (#288).
2026-06-21 ⏰ Scheduled-research executor + Reports library + post-backtest attribution: scheduled research now runs end to end — a default-off background executor (VIBE_TRADING_ENABLE_SCHEDULER) fires due interval/cron jobs through the session runtime (#278, thanks @mvanhorn, closing #254). A new /reports Run Library page lists, searches, and filters report-worthy runs with links into Run Detail + Compare (#224, thanks @LemonCANDY42). And after every backtest the agent now runs layered attribution — trade-level winners/losers, beta regression, market-regime analysis, and a Monte Carlo permutation test, gated by data availability and routing (#280, thanks @shadowinlife).
2026-06-20 🔬 Research Autopilot loop closes (Phase 3) + loader OHLC integrity guard + 4 academic alphas: Research Autopilot now runs hypothesis → signal-engine → backtest end to end — scaffold_signal_engine writes a contract-correct engine and link_autopilot_backtest feeds run metrics back to the hypothesis (68 tools) (#267). A structural OHLC sanity check drops dirty bars (high < low, non-positive prices, bad bracketing) centrally at the loader boundary, guarding every data source (#274, thanks @Shizoqua). And the academic alpha family grows 6 → 10 — Jegadeesh reversal, George-Hwang 52-week-high, Amihud illiquidity, Harvey-Siddique skew (456 factors) (#277, thanks @Robin1987China).
2026-06-19 🚀 v0.1.10 — Global data layer: market-data sources grow 10 → 18 (free Eastmoney / Sina / Stooq / Yahoo + key-gated Finnhub / Alpha Vantage / Tiingo / FMP, ban-risk fallback) plus 18 read-only data tools (fund flow, dragon-tiger, northbound, margin, block trades, SEC EDGAR + XBRL, financials, options chains, full-market screening…) across A-share / US / HK, all over MCP. Also bundles everything since 0.1.9 — 10 broker connectors, alpha compare, the provider-reliability overhaul, and the opt-in data cache. pip install -U vibe-trading-ai
2026-06-18 🔬 Research Autopilot Phase 1 + a local Data Bridge loader, + a Discord security notice: new run_research_autopilot + generate_backtest_config wire Hypothesis → Research Goal → backtest end to end (now 50 tools), and a local loader reads OHLCV straight from your own CSV / Parquet / DuckDB files (#260, #252, thanks @Robin1987China), alongside DeepSeek DSML tool-call parsing and an identifier-containment hardening wave. ⚠️ Security: the old community Discord invite now points to a server we don't control running a fake Collab.Land wallet-"verification" phishing scam — removed everywhere; the only official Discord is the HKUDS server (discord.gg/6TdQnT5xcF), and we'll never ask you to connect a wallet.
2026-06-17 🧩 Install compatibility + Opus/Kimi provider fixes: Baseline pip install vibe-trading-ai no longer pulls the optional pyharmonics / ta dependency chain; harmonic detection now lives behind vibe-trading-ai[harmonic] while the bundled detector remains available (#250, closes #249). The agent loop also avoids assistant-prefill handoff messages rejected by Opus 4.8+, and Kimi/Moonshot can override the client User-Agent with MOONSHOT_USER_AGENT (#248, closes #246 and #204); follow-up tests now directly cover background-result and auto-compact handoff paths (#251).
2026-06-16 🛡️ Security/API hardening + GLM/Zhipu alias: Settings writes require auth when configured (#245); API shell-capable tools require explicit VIBE_TRADING_ENABLE_SHELL_TOOLS=1 opt-in (#243); local shutdown requires auth when an API key is configured (#241); and untrusted loopback-looking hosts are rejected instead of treated as local (#242). Runtime edges also got cleaned up: Web chat syncs completed attempts (#236), run cards emit strict JSON for non-finite metrics (#238), malformed RSSHUB_TIMEOUT_S / RSSHUB_FETCH_BUDGET_S falls back safely (#240), and ddgs retry fallback is regression-covered (#239). GLM/Zhipu is now a first-class provider alias with model-name inference (#247, closes #237).
2026-06-15 🧭 Web-search resilience + Web UI run-continuity fixes: web_search no longer fails when a single engine is rate-limited — it now queries several free, no-key engines in order (DuckDuckGo, Google, Bing, Brave, Mojeek, Yahoo) with retry/backoff, treats "no results" as an empty answer rather than an error, and returns an actionable message instead of a bare ❌ when every engine is throttled (override the engine list with VIBE_TRADING_SEARCH_BACKENDS) (#232, closes #231, thanks @Ethan-sun01). In the Web UI, switching pages during a run no longer freezes it — the chat re-subscribes to the live stream and replays missed progress on return (#234) — and the Stop button now takes effect mid-stream and between tools instead of only at iteration boundaries (#235), closing both halves of #229 (thanks @kalkinj). The baostock loader also accepts native sh.601398 / sz.000001 codes alongside tushare-style 601398.SH (#230, thanks @bhlt).
2026-06-14 📊 Per-run token usage + progressive Run Detail charts: Every agent run now persists provider-reported token usage as a run-scoped llm_usage.json — provider/model, aggregate totals, and per-iteration counts — surfaced additively on /runs/{id}, so a finished run's token cost stays auditable after the live stream is gone (provider-reported only; no prompt/content capture, no price estimation) (#223, thanks @LemonCANDY42). The Run Detail page no longer loads every symbol's candlesticks up front: the default /runs/{id} response is unchanged, but the UI now renders the run summary first and loads each symbol's chart on demand through opt-in ?chart_payload=summary / ?chart_symbol= modes, with per-symbol loading state and a load-all-with-progress control (#225, thanks @LemonCANDY42). Two loader fixes close the cycle: yfinance's exclusive end boundary no longer drops the final requested trading day — the download now passes end + 1 day while cache keys keep the original range (#226, thanks @gyx09212214-prog) — and a malformed CCXT_TIMEOUT_MS / OKX_TIMEOUT_S value now warns and falls back to its default instead of raising at import and blocking startup (#227, thanks @gyx09212214-prog).
2026-06-13 ↩️ Resume a past session by ID from the CLI: The interactive CLI now prints the session-id on exit, with a copy-paste vibe-trading resume <session-id> hint — so locating the trace for a finished run no longer means guessing which folder under agent/sessions/ is newest by timestamp. The new vibe-trading resume <session-id> subcommand reopens that exact session and replays its recent turns into the loop; an unknown id fails fast instead of silently starting a blank session (#218, thanks @zwrong).
2026-06-12 🩺 Provider reliability overhaul — DeepSeek hangs, Kimi access, streaming liveness: A cluster of provider reports — DeepSeek runs stuck on "Agent is working…" (#208, thanks @XYWOX), reached max iterations masking empty model responses (#203, thanks @mojianliang), the UI never recovering after a stall (#195, thanks @mafia23), and Kimi rejecting the client (#204, thanks @liao497) — shared one root: every OpenAI-compatible provider ran through a single shim that applied DeepSeek/Kimi/Gemini quirks globally and silently swallowed stream failures. Provider-specific behavior now lives in an explicit capability layer — reasoning capture/replay, Gemini thought signatures, the Kimi User-Agent, OpenRouter's reasoning body are each gated to their own provider instead of cross-contaminating. Reasoning-only streams show a live "Reasoning…" indicator instead of dead air; a stream failure raises a contextual provider_stream_error with one automatic retry for transient resets (deterministic 4xx fail fast) instead of silently falling back to a slow non-streaming call; an empty model response is reported as empty_model_response instead of "max iterations"; SSE heartbeats no longer break reconnect replay; and a stuck read-only tool times out instead of hiding behind heartbeats forever. A new vibe-trading provider doctor prints a redacted provider/model/package/proxy snapshot for one-command triage of environment-side hangs. DeepSeek users can opt into the official native adapter with pip install "vibe-trading-ai[deepseek]", and kimi-k2.x's temperature=1 requirement is applied automatically — the Kimi path is verified end-to-end against the live API (tool calls + strict multi-turn reasoning replay on kimi-k2.6).
2026-06-11 🐝 Swarm workers now pull market data through the loader layer: An investment-committee run on NVDA exposed a chain of gaps — workers wrote ad-hoc yfinance scripts, trusted a malformed latest bar (volume present, OHLC empty), leaked NaN into non-strict JSON, and a context-free continuation prompt re-routed to the wrong preset (#198, thanks @BillDin for an exceptional diagnosis plus both fixes). Swarm workers now get a local get_market_data tool backed by the same normalized loader registry as MCP — strict JSON, non-finite floats serialize as null — wired into every market-data preset (21 workers across 13 presets) with a prompt policy that steers OHLCV work tool-first (#199); run_swarm takes an explicit preset_name and refuses ambiguous continuation fragments instead of silently falling back to equity_research_team (#200). Grounding got smarter too: a bare US ticker like NVDA in a swarm prompt is promoted to NVDA.US (stopword-guarded), so workers start from authoritative pre-fetched prices. The tool joins the main agent registry as well — 48 tools now. Also: your Docker data now survives updates — persistent memory, the session search index, user-created skills, shadow accounts and broker config live in named volumes, so docker compose up --build no longer wipes them (#197, thanks @FlyerJ).
2026-06-10 🐳 Docker reaches a host-side Ollama out of the box: Inside the container localhost is the container itself, so the shipped OLLAMA_BASE_URL=http://localhost:11434 failed the LLM preflight for every Dockerized Ollama setup. docker-compose.yml now defaults to http://host.docker.internal:11434 (export OLLAMA_BASE_URL to point elsewhere) and adds the host-gateway extra_hosts mapping so the same file works on Linux as well as Docker Desktop (#196, thanks @ShahNewazKhan).
2026-06-09 🔑 Clearer error when the Web UI is opened from another machine: Reaching the chat from a non-loopback client (another machine, a VM host, a phone on your LAN) without API_AUTH_KEY set returned 403 on every sensitive endpoint — sending a message, listing sessions, live status — but the chat only showed a generic "Failed to send message, please retry." The send path now surfaces the real reason — "Remote API access requires an API key. Add it in Settings, or run the backend on localhost for local-only use." — and the README's web-UI setup spells out the localhost-vs-LAN rule plus the three fixes (browse via localhost on the same machine; set API_AUTH_KEY and enter it once in Settings; or VIBE_TRADING_TRUST_DOCKER_LOOPBACK=1 for Docker Desktop's host gateway) (#191, thanks @mafia23).
2026-06-08 🔧 Gemini 3.x multi-turn tool-calling fix: This completes the Gemini 3.x thinking-model fix. The 6/05 round-trip (#176) only covered in-memory history, but the real agent loop replays history as OpenAI-format dicts where LangChain dropped the per-tool-call thought_signature before the request was built — so multi-turn tool calling still 400'd with missing thought_signature. It is now re-attached at the single _convert_input chokepoint both invoke and stream pass through (parallel calls, where only the first of N is signed, included) (#184, thanks @ngoanpv).
2026-06-07 🐝 Live swarm status in the chat timeline: When the agent launches a multi-agent swarm (investment committee, quant desk, risk committee, …), the chat now renders an inline status card that streams each worker's state — waiting / running / done / failed / blocked / retrying — in real time, the same per-agent visibility the standalone swarm dashboard already had. Runtime events are bridged into the session SSE stream without changing the existing /swarm/runs API, and a finished card rehydrates from the final run_swarm result on reconnect or history replay (#188, thanks @BillDin). Preset routing also got sharper: an explicitly named preset (e.g. investment_committee, with or without underscores) now wins over keyword scoring, and the bare IV derivatives keyword no longer false-matches inside ordinary words like "given" (#189, thanks @BillDin).
2026-06-06 ⚖️ Alpha compare — head-to-head across CLI, Web UI, REST & agent: A new alpha compare benches a hand-picked shortlist of Alpha Zoo alphas against each other on a universe and period, then ranks them by IC mean/std, IR, IC-positive ratio or sample count — each with its gap to the leader. Unlike a full-zoo bench it evaluates only the alphas you name (a new run_bench(only=…) subset filter), so comparing three alphas no longer scores all 191 in their zoo. One shared core powers every surface: vibe-trading alpha compare <id1> <id2> … --sort ir (CLI), a Compare view in the Alpha Zoo Web UI (tick alphas in the catalogue → one-click compare with a streamed ranking table), POST /alpha/compare + SSE (REST), and a read-only alpha_compare agent tool (47 tools now).
2026-06-05 🇮🇳 Dhan + Shoonya connectors (India) — 10 brokers total: The connector-first trading layer adds Dhan and Shoonya for the Indian market (NSE/BSE equities + F&O), bringing the roster to ten brokers. Both are paper + read-only — like Longbridge, their APIs expose no runtime paper/live discriminator, so their place_order / cancel_order hard-refuse any non-paper config at the first line (the rule: a broker with no structural paper/live guard is capped at paper + read-only) (#181, closes #174). This cycle also fixes Gemini 2.5 / 3.x thinking models: their per-tool-call thoughtSignature now round-trips through the OpenAI-compatible path, so multi-turn function calling no longer fails with INVALID_ARGUMENT (#176, closes #170, thanks @mvanhorn & @jliu6789). Chinese docstrings landed on all 452 Alpha Zoo factors (#180, thanks @LeeCQiang), and a frontend test suite (197 vitest tests) plus backend auth / path-traversal / CORS security tests joined CI (#175, thanks @sambazhu).
2026-06-04 🗃️ Opt-in local data cache for all 7 data sources: A new VIBE_TRADING_DATA_CACHE switch lets every backtest loader — tushare, okx, ccxt, akshare, mootdx, yfinance, futu — cache settled historical bars under ~/.vibe-trading/cache (user home, never the repo), so repeated and long-horizon / cross-market backtests skip the network and avoid provider rate limits. Off by default. Batch and connection loaders (yfinance, futu) skip the bulk download / FutuOpenD connection entirely on a full cache hit, a staleness guard never caches a range ending today (its last bar is still forming), and cached frames round-trip byte-identical to freshly fetched ones (#177, thanks @mvanhorn). A new contributor guide for AI / automation-assisted PRs also landed, mapping safe local checks and high-risk broker/MCP/credential surfaces (#173).
2026-06-03 🧹 Community triage + trace correlation: Tool-call trace entries now carry the originating call_id, so a tool_result can be matched back to its tool_call when replaying a run trace — arg previews stay truncated to keep trace files small (#168, thanks @zwrong). Source comments no longer point at an internal-only docs path that external contributors couldn't find (#166, thanks @jaleelpersonal). Also clarified that the langchain-community resolver warning on install is a harmless leftover-package notice, not a failure (#167), and scoped Gemini 2.5/3.0 thoughtSignature round-tripping for function calls as a help wanted task with a full fix plan (#170, thanks @jliu6789).
2026-06-02 🔌 Six new broker connectors (Tiger / Longbridge / Alpaca / OKX / Binance / Futu): The connector-first trading layer gains a direct-SDK transport alongside IBKR (local) and Robinhood (MCP). Each connector exposes read-only account / positions / orders / quote / history plus paper-account order placement — test your strategies across these broker paper accounts. Five of them (Tiger, Alpaca, OKX, Binance, Futu) also support bounded, mandate-gated order placement behind the same safety model as Robinhood: a user-committed mandate (symbol universe / order size / exposure / leverage / daily cap), a filesystem kill switch, a fail-closed pre-trade gate, and a full audit ledger. Longbridge is paper + read-only only (its API exposes no runtime paper/live discriminator). Every paper/live distinction is a structural per-broker guard — account-id format, host separation, demo flag, or trade environment. New trading_place_order / trading_cancel_order tools; HK and A-share asset classes added to the mandate universe. Experimental / use at your own risk.
2026-06-01 🚀 v0.1.9 released (pip install -U vibe-trading-ai): Rolls up everything since 0.1.8. Connector-first broker profiles (IBKR local read-only TWS / IB Gateway + Robinhood Agentic Trading behind OAuth, a committed mandate, order guard, audit ledger, and instant halt). Research Goal runtime across CLI / REST / MCP / Web. A swarm pass — live reconcile + MCP keepalive, operator-configured worker MCP tools, a strict alpha-bench random control, and a new retry_run to relaunch failed/stale runs (36 MCP tools now). The agent/cli/ package refactor with a refreshed terminal UI, the mootdx no-token A-share loader, and a robustness pass across backtest / agent loop / sessions. --version now always matches the installed package, fixing the 0.1.8 drift (#156).
2026-05-31 🔌 Connector-first broker architecture (IBKR + Robinhood): Trading access now starts from a selectable connector profile instead of separate broker/live entry points. vibe-trading connector list/use/check/account/positions/orders/quote/history and the MCP trading_* tools share the same selected profile, where paper/live is an attribute of the connector. IBKR can be used immediately through a local read-only TWS / IB Gateway profile, while the official IBKR remote MCP path is seeded as an OAuth mcp.read probe until stable read tool names are available. Robinhood Agentic Trading remains the bounded live MCP connector behind OAuth, a committed mandate, order guard, audit ledger, and instant halt.
2026-05-30 🧰 Robustness pass — backtest, agent loop, sessions: LLM-generated signal engines now pass pre-flight interface validation before instantiation, catching circular self-imports, a missing generate(), non-defaulted __init__ args, and wrong return types with actionable JSON errors instead of raw tracebacks (#149); a follow-up routes source-level AST validation errors through the same clean JSON envelope. The agent loop no longer burns all 50 iterations into a failed status with no output — it mirrors the swarm worker's wrap-up nudge at 80% of the iteration budget and drops tool definitions on the last iteration to force a final text answer (#148), guarded to fire only mid-run so it never displaces research-goal context. Session message writes now flush + fsync each append so expensive AI responses survive a mid-write crash, and the read path skips corrupted JSONL lines (logging the first 200 chars for recovery) instead of 500-ing the whole /messages endpoint (#147). The Web composer also fixes IME Enter handling so a composition-confirming Enter no longer submits mid-word (#146).
2026-05-29 🔐 Robinhood Agentic Trading support (opt-in, bounded autonomy): Adds support for Robinhood Agentic Trading (remote MCP, OAuth). Off and read-only by default; the agent acts only inside a user-committed mandate (symbols / order size / exposure / leverage / daily cap), with a filesystem-level instant kill switch, preemptive flatten, mandate auto-expiry, a full audit ledger, and a persistent autonomous runner. No custody, no venue — the broker holds funds and executes; we only relay intent. Experimental / use at your own risk.
2026-05-28 🧪 Swarm safety + strict alpha gate + worker MCP: Swarm DAG blocks downstream tasks when upstream fails (#145). New run_bench_strict() adds a same-universe random control + OOS split to catch factors that just track market beta (#143, thanks @Soli22de). Swarm workers can call operator-configured external MCP servers, with trust boundary pinned (#142, thanks @shadowinlife).
2026-05-27 📊 mootdx A-share data source + output polish: New mootdx loader speaks the native 通达信 TCP protocol for A-share OHLCV (no auth, no IP rate-limit, daily + intraday with 25-page walk-back pagination), slotting between tushare and akshare in the fallback chain (#107). CCXT loader now reads HTTP_PROXY/HTTPS_PROXY/ALL_PROXY so Binance/OKX public data works from restricted networks (#126, thanks @ruok808). Final-answer rendering also dropped the ugly full-width --- horizontal separators on CLI and Web: the system prompt now nudges the agent toward markdown tables and ## headings, the CLI renderer strips standalone HRs as defense-in-depth, and the chat bubble hides any <hr> that slips through (#139, thanks @sdwxm188).
2026-05-26 ✅ Research Goal lifecycle closure: Goal mode now behaves like a real task runner: Web UI goal creation creates or binds the session and immediately sends the kickoff turn; active goals can be continued, edited, cancelled, and completed across Web/API/CLI/MCP; and the agent advances from the current goal snapshot (criteria, evidence, claims, open items) instead of only the original prompt. Covered-but-still-active goals now enter an audit/status update instead of stopping silently, with regression coverage across backend, CLI, MCP, and frontend events.
2026-05-25 🧼 Cleaner chat UI + composer workflow: The Web UI keeps chat focused on the next action: upload, swarm, and research-goal modes now live behind the composer + menu instead of floating panels. Active context appears above the input as compact chips, and goal details expand inline only when needed. The UI also drops the old custom i18n layer in favor of direct English copy, gates Full Report cards to report-worthy runs, and hardens local dev startup/status reporting for reliable browser smoke tests.
2026-05-24 🎯 Research Goal runtime: Added a session-scoped Research Goal layer across backend, CLI, API/MCP, SSE, and Web UI. Goals persist claims, acceptance criteria, evidence rows, budgets, and completion policy; agent tools can create goals and attach evidence; /goal gives the CLI a direct entry point; REST/MCP expose goal snapshots and evidence writes; SSE keeps chat clients fresh. Follow-up audit fixes locked down verified evidence, blocked live-trading risk tiers through agent tools, wired CLI-created goals into later turns, cleaned goal ledgers on session deletion, enabled replay-all, and fixed cross-session frontend races.
2026-05-23 🖥️ Interactive CLI refresh: The terminal front door now opens with a larger Vibe-Trading banner, a cleaner prompt divider, prior-turn recap, post-run timing, and a Claude Code-style activity rail for live agent work. Tool calls, web/data fetches, shell-style actions, Markdown answers, and pipe tables render in a more readable transcript, while piped or non-TTY runs keep plain-text output for automation. Generated CLI screenshots are now treated as local artifacts instead of committed docs files, keeping the repository lighter.
2026-05-22 🧭 Swarm recovery + MCP keepalive: Swarm status now reconciles from live task files on every read, so API/MCP/SSE/list views recover crashed or stale runs instead of showing permanent running snapshots. run_swarm sends MCP progress heartbeats while it polls, with a fixed first frame of swarm_started run_id=<id> for clients that reconnect after transport drops; workers now heartbeat through LLM streaming, grounding fetches, and tool execution. The stale-run reaper uses per-run thresholds and derives terminal status from task states, SwarmTool no longer cancels a still-running team just because its wait budget elapsed, and MCP clients can call reap_stale_runs() for explicit cleanup. Today's DX pass also refreshed provider default models and aligned CI syntax checks with the new agent/cli/ package. 22 new regressions cover hydration, terminal recovery, stale reaping, keepalive cadence, env parsing, and heartbeat wiring; the full swarm/MCP suite is at 169 passed, 4 skipped.
2026-05-21 🧱 CLI package refactor: agent/cli.py (3216 LOC) split into the agent/cli/ package — interactive front door, slash router, Rich components, plus a _legacy.py shim that preserves every subcommand and re-exports every public symbol so cli.cmd_* / cli._INIT_ENV_PATH / cli.Confirm keep working. New FastAPI middleware serves the SPA shell when a browser opens /runs/{id} or /correlation directly; same narrowing landed in the Vite dev proxy. Version unified via cli/_version.py (no more drift between --version and the banner), python -m cli restored via __main__.py, and the chat-gate narrowed so chat --help / chat extra reach legacy argparse instead of being swallowed by the REPL.
2026-05-20 🔬 Hypothesis Registry CLI: Closes the CLI side of the Hypothesis Registry shipped backend-only on 2026-05-16. vibe-trading hypothesis list prints a Rich table or JSON (--status filter, --limit); show <id> renders a detail panel including linked run cards; invalidate <id> --note "..." flips status to rejected while preserving prior invalidation notes when --note is omitted. Honors the existing VIBE_TRADING_HYPOTHESES_PATH env override and adds a per-invocation --path. 22 new tests cover wiring, JSON output, status filter, limit, missing-id errors, and note persistence.
2026-05-19 ✨ Live tool feedback + graceful cancel: Long-running tools (backtests, large PDFs, swarm workers) no longer look frozen. Each tool call now emits a 3-second heartbeat plus structured per-stage progress — run_backtest shows phase markers (validate / simulate / finalize), read_document ticks per page on PDF or per sheet on Excel, read_url marks fetch / parse. The CLI Rich Live dashboard renders a Unicode spinner, ASCII progress bar, ETA, and stacks up to 3 parallel tools keyed by name; the frontend chat ships a new ToolProgressIndicator with rAF-coalesced renders, ARIA role="status" + hidden native <progress> for screen readers, and a determinate ProgressRing SVG when total is known. First Ctrl+C during a CLI run now calls agent.cancel() for graceful exit (current step finishes, trace closes cleanly); a second within 2s force-quits. Reusable primitives extracted along the way: ProgressBar.tsx and lib/tools.ts (shared tool-name i18n).
2026-05-18 🧹 Cleanup pass + three latent bug fixes: CompositeEngine no longer misroutes bare Chinese-futures codes like RB2410 to GlobalFuturesEngine — _is_china_futures moved into a shared _market_hooks module with a case-normalized product table and a non-CN exchange guard, plus 9 new regression cases. Session FTS5 indexes now persist timestamps so cross-session search can sort by date; the same path also fixed a re-upsert that was wall-clocking every session's started_at. The Vite dev-mode proxy gained the missing /alpha entry so the AlphaZoo page resolves on npm run dev. tests/test_e2e_harness_v2.py (real-LLM e2e suite) is now gated behind VIBE_TRADING_RUN_LIVE_E2E=1 so CI no longer changes shape based on env-key presence. Ruff per-file-ignores added for the factor zoo (3783 → 0 F401 noise), frontend tsconfig enables noUnusedLocals / noUnusedParameters as regression guards, and 76 unused vw = vwap(...) boilerplate lines were dropped from gtja191 alphas. Net -918 LOC.
2026-05-17 🧬 Alpha Zoo v1 (0.1.8): 452 pre-built quant alphas across 4 zoos — qlib158 (Microsoft Qlib, Apache-2 attribution), alpha101 (Kakushadze 101 Formulaic Alphas, paper rewrite from arXiv:1601.00991), gtja191 (Guotai Junan 2014 short-horizon factor report), and academic (Fama-French 5 + Carhart price-based proxies). One-line CLI to bench any zoo on your universe: vibe-trading alpha bench --zoo gtja191 --universe csi300 --period 2018-2025. Ships with AST purity gate, lookahead-guard test, pytest-socket network kill-switch, per-zoo LICENSE.md, and a Developer Certificate of Origin (DCO) workflow for community PRs. Auto-rendered Alpha Library at vibetrading.wiki/alpha-library/ + research-lab post Which of the 191 GTJA alphas still work in 2026?.
2026-05-16 🧪 Research spine update: Added a backend Hypothesis Registry with create_hypothesis, update_hypothesis, link_backtest, and search_hypotheses; external-content readers now attach warning-only security_warnings; and Shadow Account scanning now uses deterministic OHLCV feature evaluation instead of the old calendar-phase stub.
2026-05-15 🪪 The run detail page now surfaces the Trust Layer run card alongside metrics and artifacts, completing the UI side of the run_card.json work landed on 2026-05-12. PersistentMemory.add() was also hardened on length, empty/whitespace-only names, and C0/C1 control bytes from the #108/#109/#110 triage (#112, thanks @Teerapat-Vatpitak).
2026-05-14 🌐 the public wiki is now live at vibetrading.wiki with docs, tutorials, Research Lab, and Alpha Library sections deployed through Cloudflare Pages. Persistent memory is also inspectable from the CLI via vibe-trading memory list/show/search/forget (#102, thanks @Teerapat-Vatpitak), and memory tokenization/slugs now support Thai, Arabic, Hebrew, and Cyrillic text (#104).
2026-05-13 🧭 Swarm runs now ground workers with fetched market data and cleaner persisted reports (#93, #84).
2026-05-12 🧾 Backtests now emit run_card.json and run_card.md alongside artifacts for reproducible research runs.
2026-05-11 🧭 Memory slugs, swarm accounting, and CLI preflight: Persistent memory now preserves CJK characters when generating file slugs, preventing silent filename collisions for Chinese/Japanese/Korean notes (#95, thanks @voidborne-d). Swarm run totals now prefer provider-reported token usage with the existing estimate fallback (#94, thanks @Teerapat-Vatpitak), and the CLI run UI gained a startup preflight check for common environment issues (#96, thanks @ykykj).
2026-05-10 🧱 Regression guardrails + run metadata: Memory recall now treats underscores as token boundaries, so snake_case saved memories such as mcp_wiring_test match natural-language queries like "mcp wiring" (#87, thanks @hp083625). The MCP server has a subprocess smoke test covering initialize → tools/list → tools/call to guard the first-call deadlock path (#86), while low-risk hardening landed for Windows path-sensitive tests, API best-effort exception handling, backtest run_dir allowed-root validation, and SwarmRun provider/model metadata (#88, #90, #91, #92, thanks @Teerapat-Vatpitak).
2026-05-09 🛡️ API path hardening + MCP server stability: API run/session routes now validate path IDs before lookup, rejecting malformed newline-containing parameters and pinning the behavior in the auth/security regression suite (#80, thanks @SJoon99). The MCP server now pre-warms the tool registry on the main thread before serving tools/call, avoiding a first-call deadlock in lazy tool discovery (#85, thanks @Teerapat-Vatpitak). The Vite dev proxy also honors VITE_API_URL for non-default backend targets (#82, thanks @voidborne-d).
2026-05-08 🧾 Tushare statement fields in filters: A-share daily backtests can now request PIT-safe financial statement fields through fundamental_fields, so signal engines can screen on income_total_revenue, income_n_income, balancesheet_total_hldr_eqy_exc_min_int, fina_indicator_roe, and similar table-prefixed columns after their announcement/disclosure dates (#76, thanks @mrbob-git). Follow-up hardening makes explicit statement-field requests fail fast if Tushare enrichment cannot run, instead of silently falling back to raw price bars (#77).
2026-05-07 📈 Tushare fundamentals + community triage: Added a point-in-time TushareFundamentalProvider contract for fundamental research workflows, with regression coverage for the project TUSHARE_TOKEN environment path (#74). Community triage also clarified that Vibe-Trading keeps rapid iteration focused on one UI language for now, avoids adding redundant search dependencies while DuckDuckGo-backed web_search is already bundled, and treats unofficial hosted deployments as untrusted places for API keys or data-source tokens.
2026-05-06 🚀 v0.1.7 released (Release notes, pip install -U vibe-trading-ai): Security-boundary hardening is now published on PyPI and ClawHub, covering safer API/read/upload/file/URL/generated-code/shell-tool/Docker defaults while keeping localhost CLI/Web UI workflows low-friction. This cycle also includes Web UI Settings, correlation heatmap, OpenAI Codex OAuth, A-share pre-ST filtering, interactive CLI UX, swarm preset inspection, dividend analysis, dev workflow polish, and audited frontend build-dependency floors. Thanks to the 0.1.7 contributors and to lemi9090 (S2W) for coordinated security validation.
2026-05-05 🛡️ Security boundary follow-up: Completes the remaining security-boundary hardening around explicit CORS origins, Settings credential indicators, web URL reading, and Shadow Account code generation, with regression tests added for each path. Normal localhost CLI/Web UI workflows stay the same; remote deployments should continue using API_AUTH_KEY and explicit trusted origins.
2026-05-04 🖥️ Interactive CLI UX + CI cleanup: Interactive mode now has a live bottom status bar showing provider/model, session duration, last-run latency, and cumulative tool-call stats, plus prompt history navigation and cursor editing with arrow keys via prompt_toolkit (#69). The CLI still falls back to Rich prompts when prompt_toolkit or a TTY is unavailable. CI path expectations were also aligned with the hardened file-import sandbox and cross-platform /tmp resolution, returning main to green (bb67dc7).
2026-05-03 🛡️ Security hardening patch: Tightens default API authentication for non-local deployments, protects sensitive run/session/swarm reads, restricts upload and local file-reading boundaries, gates shell-capable tools by entry point, validates generated strategy loading before import, and runs the Docker image as a non-root user with a localhost-only published port by default. Local CLI and localhost Web UI workflows remain low-friction; remote API/Web deployments should set API_AUTH_KEY.
2026-05-02 🧭 Dividend analysis + sharper roadmap: Added the dividend-analysis skill for income stocks, payout sustainability, dividend growth, shareholder yield, ex-dividend mechanics, and yield-trap checks, pinned by bundled-skill regression tests. The public roadmap now focuses on upcoming work: Research Autopilot, Data Bridge, Options Lab, Portfolio Studio, Alpha Zoo, Research Delivery, Trust Layer, and Community sharing.
2026-05-01 🔥 Correlation heatmap + OpenAI Codex OAuth + A-share pre-ST filter: New correlation dashboard/API computes rolling return correlations and renders an ECharts heatmap for portfolio and symbol analysis (#64). OpenAI Codex provider support now uses ChatGPT OAuth via vibe-trading provider login openai-codex, with Settings metadata and adapter regression tests (#65). Added and hardened the ashare-pre-st-filter skill for A-share ST/*ST risk screening, including Sina penalty relevance filtering so securities-account mentions do not inflate E2 counts (#63).
2026-04-30 ⚙️ Web UI Settings + validation CLI hardening: New Settings page for LLM provider/model, base URL, reasoning effort, and data source credentials, backed by local/auth-protected settings APIs and data-driven provider metadata (#57). Also hardens python -m backtest.validation <run_dir> so missing, blank, malformed, non-existent, and non-directory inputs fail with clear operator-facing messages before validation starts (#60).
2026-04-28 🚀 v0.1.6 released (pip install -U vibe-trading-ai): Fixes vibe-trading --swarm-presets returning empty after pip install / uv tool install (#55) — preset YAMLs now bundled inside the src.swarm package and pinned by a 6-test regression suite. Plus AKShare loader correctly routes ETFs (510300.SH) and forex (USDCNH) to the right endpoints with hardened registry fallback. Rolls up everything since v0.1.5: benchmark comparison panel, /upload streaming + size limits, Futu loader (HK + A-share), vnpy export skill, security hardening, frontend lazy loading (688KB → 262KB).
2026-04-27 📊 Benchmark panel + upload safety: Backtest output now ships a benchmark comparison panel (ticker / benchmark return / excess return / information ratio) with yfinance-backed resolution for SPY, CSI 300, etc. (#48). Plus /upload streams the request body in 1 MB chunks and aborts past MAX_UPLOAD_SIZE, bounding memory under oversized/malformed clients (#53) — pinned by a 4-case regression suite.
2026-04-22 🛡️ Hardening + new integrations: Path containment enforced in safe_path + journal/shadow tool sandbox, MANIFEST.in ships .env.example / tests / Docker files in sdist, route-level lazy loading shrinks frontend initial bundle 688KB → 262KB. Plus Futu data loader for HK & A-share equities (#47) and vnpy CtaTemplate export skill (#46).
2026-04-21 🛡️ Workspace + docs: Relative run_dir normalized to active run dir (#43). README usage examples (#45).
2026-04-20 🔌 Reasoning + Swarm: reasoning_content preserved across all ChatOpenAI paths — Kimi / DeepSeek / Qwen thinking work end-to-end (#39). Swarm streaming + clean Ctrl+C (#42).
2026-04-19 📦 v0.1.5: Published to PyPI & ClawHub. python-multipart CVE floor bump, 5 new MCP tools wired (analyze_trade_journal + 4 shadow-account tools), pattern_recognition → pattern registry fix, Docker dep parity, SKILL manifest synced (22 MCP tools / 71 skills).
2026-04-18 👥 Shadow Account: Extract your strategy rules from a broker journal → backtest the shadow across markets → 8-section HTML/PDF report showing exactly how much you leave on the table (rule violations, early exits, missed signals, counterfactual trades). 4 new tools, 1 skill, 32 tools total. Trade Journal + Shadow Account samples now live in the web UI welcome screen.
2026-04-17 📊 Trade Journal Analyzer + Universal File Reader: Upload broker exports (同花顺/东财/富途/generic CSV) → auto trading profile (holding days, win rate, PnL ratio, drawdown) + 4 bias diagnostics (disposition effect, overtrading, chasing momentum, anchoring). read_document now dispatches PDF, Word, Excel, PowerPoint, images (OCR), and 40+ text formats behind one unified call.
2026-04-16 🧠 Agent Harness: Persistent cross-session memory, FTS5 session search, self-evolving skills (full CRUD), 5-layer context compression, read/write tool batching. 27 tools, 107 new tests.
2026-04-15 🤖 Z.ai + MiniMax: Z.ai provider (#35), MiniMax temperature fix + model update (#33). 13 providers.
2026-04-14 🔧 MCP Stability: Fixed backtest tool Connection closed error on stdio transport (#32).
2026-04-13 🌐 Cross-Market Composite Backtest: New CompositeEngine backtests mixed-market portfolios (e.g. A-shares + crypto) with shared capital pool and per-market rules. Also fixed swarm template variable fallback and frontend timeout.
2026-04-12 🌍 Multi-Platform Export: /pine exports strategies to TradingView (Pine Script v6), TDX (通达信/同花顺/东方财富), and MetaTrader 5 (MQL5) in one command.
2026-04-11 🛡️ Reliability & DX: vibe-trading init .env bootstrap (#19), preflight checks, runtime data-source fallback, hardened backtest engine. Multi-language README (#21).
2026-04-10 📦 v0.1.4: Docker fix (#8), web_search MCP tool, 12 LLM providers, akshare/ccxt deps. Published to PyPI and ClawHub.
2026-04-09 📊 Backtest Wave 2: ChinaFutures, GlobalFutures, Forex, Options v2 engines. Monte Carlo, Bootstrap CI, Walk-Forward validation.
2026-04-08 🔧 Multi-market backtest with per-market rules, Pine Script v6 export, 5 data sources with auto-fallback.

✨ Key Features

🔍 Self-Improving Trading Agent • Natural-language market research • Strategy drafts and file/web analysis • Memory-backed workflows	🐝 Multi-Agent Trading Teams • Investment, quant, crypto, and risk teams • Streaming progress and persisted reports • Workers grounded with fetched market data
📊 Cross-Market Data & Backtesting • A / HK / US / Canada / India / Korea equities, crypto, futures, and forex • Data fallback and composite backtests • PIT data, validation, and run cards	👥 Shadow Account • Broker-journal behavior diagnostics • Rule-based Shadow Account comparisons • Exportable audit reports and strategy code

💡 What Is Vibe-Trading?

Vibe-Trading is an open-source research workspace for turning finance questions into runnable analysis. It connects natural-language prompts to market-data loaders, strategy generation, backtest engines, reports, exports, and persistent research memory.

It is designed for research, simulation, and backtesting — and, when you choose, autonomous trading through a broker you authorize yourself (e.g. Robinhood Agentic Trading). It holds no funds and never trades outside the limits you set, and you can halt it instantly.

✨ What You Can Do

Task	Output
Ask a trading question	Market research with tools, data, documents, and reusable session context.
Backtest a strategy idea	Strategy code, metrics, benchmark context, validation artifacts, and run cards.
Review your own trades	Broker-journal parsing, behavior diagnostics, rule extraction, and Shadow Account comparisons.
Read documents & charts	Parse PDF / DOCX / XLSX / PPTX / images with pluggable OCR (`read_document`), and read chart screenshots semantically with a vision model (`analyze_image`).
Read institutional filings & fund books	SEC 13F manager books with quarter-over-quarter position diffs, ETF constituents across markets, event-contract implied probability, and arXiv / OpenAlex factor extraction — all read-only, on free public sources.
Improve repeated research	Persistent memory and editable skills turn useful routines into reusable workflows.
Run analyst teams	Multi-agent research reviews for investment, quant, crypto, macro, and risk workflows.
Put research into IM channels	Run the same session runtime through WebSocket, Telegram, Slack, Discord, Matrix, WhatsApp, Signal, QQ/NapCat, WeChat/WeCom, Feishu/Lark, DingTalk, Teams, email, and Mochat with CLI, REST, and Web UI controls.
Ship usable artifacts	Reports, TradingView Pine Script, TDX, MetaTrader 5, MCP tools, and later research sessions.
Bench a pre-built alpha zoo	One-line IC + alive/reversed/dead categorisation across 462 alphas (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental) on your universe.
Spot correlation regimes	An edge-density + hysteresis timeline on the `/correlation` surface showing when markets fuse into one bloc — descriptive risk context, not a signal.

⚡ Quick Example

pip install vibe-trading-ai

# Natural-language research
vibe-trading run -p "Backtest a BTC-USDT 20/50 moving-average strategy for 2024, summarize return and drawdown, then export the report"

# Bench a pre-built alpha zoo (one line)
vibe-trading alpha bench --zoo gtja191 --universe csi300 --period 2018-2025 --top 20

vibe-trading --upload trades_export.csv
vibe-trading run -p "Analyze my trading behavior, extract my shadow strategy, and compare it with my actual trades"

👥 Shadow Account

Shadow Account starts from your own trading records instead of a generic strategy template.

Upload a broker export, let the agent summarize your behavior, then compare the actual trading path with a rule-based shadow strategy.

Step	Agent output
1. Read your journal	Parses broker exports from 同花顺, 东方财富, 富途, and generic CSV formats.
2. Profile your behavior	Holding days, win rate, PnL ratio, drawdown, disposition effect, overtrading, momentum chasing, and anchoring checks.
3. Extract your rules	Turns recurring entries/exits into an explicit strategy profile instead of a hand-wavy summary.
4. Run the shadow	Backtests the extracted rules and highlights rule breaks, early exits, missed signals, and alternative trade paths.
5. Deliver the report	Produces an HTML/PDF report that can be inspected, archived, or refined in a later session.

vibe-trading --upload trades_export.csv
vibe-trading run -p "Analyze my trading behavior, extract my shadow strategy, and compare it with my actual trades"

🧪 Research Workflow

Most runs follow the same evidence path: route the request, load the right market context, execute tools, validate outputs, and keep the artifacts inspectable.

Layer	What happens
Plan	Selects the relevant finance skills, tools, data sources, and swarm preset when useful.
Ground	Pulls A-shares, HK/US/Canada equities, crypto, futures, forex, documents, or web context through the available loaders.
Execute	Generates testable strategy code, runs tools, and uses the matching backtest engine or analysis workflow.
Validate	Adds metrics, benchmark comparison, Monte Carlo, Bootstrap, Walk-Forward, run cards, and warnings where applicable.
Deliver	Returns reports, artifacts, tool traces, and exports for TradingView, TDX, MetaTrader 5, MCP clients, or later sessions.

📡 Data Sources & Smart Fallback

One get_market_data call, 23 free market-data sources (plus the optional QVeris premium marketplace). Set source: "auto" — the loader picks by symbol, then walks a per-market chain ordered by IP-ban risk: never-banned public sources first, throttled / key-gated ones last. Zero config, no single point of failure.

Source	Markets	Auth	Role
`tencent` · `mootdx`	A-share + HK	none	never IP-banned (`mootdx` = 通达信 TCP)
`eastmoney`	A / US / HK	none	OHLCV + deep fundamentals & flow tools (throttled)
`baostock` · `akshare`	A (+ US/HK/futures/macro/fx)	none	free fallbacks
`tushare`	A / HK / futures / fund / macro	token	richest A-share
`yahoo`	US / HK / Canada	none	direct chart/quotes/options; TSX `.TO` / TSXV `.V`
`sina` · `stooq`	US	none	K-line to 1984 · EOD CSV
`yfinance`	US / HK / Canada	none	wrapper; TSX `.TO` / TSXV `.V` pass through
`longbridge`	US / HK	App Key + App Secret + Access Token	optional historical OHLCV source; install the optional SDK
`finnhub` · `alphavantage` · `tiingo` · `fmp`	US	key	optional providers
`qveris`	global multi-asset	key · credits	premium marketplace — 63+ providers via one key (explicit-only, never in auto fallback)
`okx` · `ccxt` · `binance`	crypto	none	OKX + 100+ exchanges + Binance historical / USD-M perps
`futu`	HK / A	OpenD	optional local FutuOpenD
`mt5`	forex / metals	MT5 terminal	MetaTrader 5 (Exness-style) forex / metal bars, 1m–1D
`pykrx`	Korea (KRX: KOSPI/KOSDAQ)	none	daily KOSPI / KOSDAQ bars for `.KS` / `.KQ` (optional `krx` extra)
`india_broker`	India (NSE/BSE)	broker login	read-only Shoonya / Dhan bars for `.NS` / `.BO` (fallback-chain tail)
`local`	any	none	your own CSV / Parquet / DuckDB via `local:` prefix

Fallback chains (by IP-ban risk):

A-share → tencent · mootdx · eastmoney · baostock · akshare · tushare · local
US → yahoo · stooq · sina · eastmoney · yfinance · tiingo · fmp · finnhub · alphavantage · longbridge · akshare · local
HK → tencent · eastmoney · yahoo · futu · akshare · yfinance · tushare · longbridge · local
India (NSE/BSE) → yahoo · yfinance · india_broker · local
Korea (KOSPI/KOSDAQ) → pykrx · yahoo · yfinance · local
Crypto → okx · ccxt · binance · yfinance · local
Forex / metals → mt5 · yfinance · akshare · local · (futures / fund / macro → tushare/akshare → local)

Using Longbridge explicitly

Longbridge is an optional US/HK historical OHLCV loader. Install its SDK with:

pip install "vibe-trading-ai[longbridge]"

Configure the three credentials in .env:

LONGBRIDGE_APP_KEY=...
LONGBRIDGE_APP_SECRET=...
LONGBRIDGE_ACCESS_TOKEN=...

For a backtest, set source in config.json:

{
  "codes": ["QQQ.US"],
  "start_date": "2025-01-01",
  "end_date": "2025-01-10",
  "interval": "1D",
  "source": "longbridge"
}

In an Agent conversation, ask explicitly: "Use Longbridge to fetch QQQ.US historical data." The explicit source request is separate from source: "auto"; auto keeps the normal per-market fallback chain.

Beyond OHLCV, 22 read-only data tools reach into fundamentals & flow — fund flow, dragon-tiger, northbound, margin, block trades, shareholder count, lockup, sector, research reports, news, SEC filings, financial statements, options chains, stock profile, market screening, symbol search, macro, iwencai, institutional holdings (13F), ETF look-through, prediction markets, and research papers — all exposed over MCP. An explicit local: symbol never silently falls back to a network source.

💎 Optional premium data — QVeris

Data: free routing or premium, your choice. Free stays the default: 23 built-in sources with ban-risk fallback, no key, no cost. Premium via QVeris adds 10,000+ capabilities (per QVeris) across 63+ providers for options Greeks, premium fundamentals, China/HK/global data, macro, crypto, news, and filings; failed calls are not charged. Enable it in Settings -> QVeris or vibe-trading data mode paid.

QVeris disclosure: signing up through the Vibe-Trading referral link gets you +1,000 bonus credits and supports the project.

🔩 Detailed Capabilities

Detailed inventories are folded below to keep the main README scannable. Open them when you want to inspect the available building blocks.

Finance Skill Library _{89 skills across 9 categories}

📊 89 specialized finance skills organized into 9 categories
🌐 Complete coverage from traditional markets to crypto & DeFi
🔬 Comprehensive capabilities spanning data sourcing to quantitative research

Category	Skills	Examples
Data Source	10	`data-routing`, `tushare`, `yfinance`, `okx-market`, `akshare`, `mootdx`, `ccxt`, `eastmoney`, `sec-edgar`, `qveris`
Strategy	19	`strategy-generate`, `cross-market-strategy`, `technical-basic`, `candlestick`, `ichimoku`, `elliott-wave`, `smc`, `multi-factor`, `ml-strategy`
Analysis	23	`factor-research`, `correlation-regime`, `macro-analysis`, `global-macro`, `valuation-model`, `investor-lenses`, `credit-analysis`, `dividend-analysis`
Asset Class	9	`options-strategy`, `options-advanced`, `convertible-bond`, `etf-analysis`, `asset-allocation`, `sector-rotation`
Crypto	7	`perp-funding-basis`, `liquidation-heatmap`, `stablecoin-flow`, `defi-yield`, `onchain-analysis`
Flow	8	`hk-connect-flow`, `us-etf-flow`, `edgar-sec-filings`, `financial-statement`, `adr-hshare`
Tool	10	`backtest-diagnose`, `report-generate`, `pine-script`, `doc-reader`, `web-reader`, `vnpy-export`, `trade-journal`
Research	2	`alpha-zoo`, `strategy-dev-manager`
Risk Analysis	1	`ashare-pre-st-filter`

Custom Data Source _{register your own historical OHLCV loader}

Need a market or vendor we don't ship a loader for? Add your own historical-bar loader and select it with source="<name>". The steps edit package source, so run from a clone (pip install -e .).

Write the loader — create agent/backtest/loaders/<name>_loader.py with a class that satisfies DataLoaderProtocol (duck-typed, no base class needed) and is tagged with @register:

import pandas as pd
from backtest.loaders.registry import register

@register
class DataLoader:
    name = "mysource"            # the value you pass as source=
    markets = {"us_equity"}      # a_share/us_equity/hk_equity/crypto/futures/fund/macro/forex
    requires_auth = False

    def is_available(self) -> bool:
        return True              # token present? network reachable?

    def fetch(self, codes, start_date, end_date, *, interval="1D", fields=None):
        # return {symbol: DataFrame indexed by trade_date,
        #         columns: open, high, low, close, volume}
        ...

Register the module so @register fires — add "backtest.loaders.<name>_loader" to _loader_modules in agent/backtest/loaders/registry.py.
Allow the name through config validation — add "mysource" to _VALID_SOURCES in agent/backtest/runner.py.
(Optional) slot it into a market's FALLBACK_CHAINS in registry.py so source="auto" can reach it.
Use it — source="mysource" in a backtest config, or via the CLI / agent.

Real-time ticks / order-book depth are out of scope for loaders — the loader layer is point-in-time historical bars only. Live market data flows through the broker connectors instead: okx / binance / ccxt for crypto, futu / tiger for equities.

Broker Connectors _{12 brokers — read + paper, bounded-live where supported}

Connector-first profiles. Most do read + paper-account order placement — IBKR is read-only, Robinhood is live-only (no paper account), and Trading 212 refuses order placement entirely, paper included; live order placement is bounded by a user-defined mandate (symbol allowlist, order-size / exposure caps, daily trade cap, instant kill switch) and never holds funds — the broker executes. Order-placing tools stay off MCP (agent + CLI only). Research / backtest paths are structurally barred from any live endpoint.

Broker	Markets	Capabilities
IBKR	global	local TWS / Gateway, read-only
Robinhood	US	Agentic MCP (desktop OAuth) — read + bounded live
Tiger	US / HK / A	read + paper + bounded live
Alpaca	US	read + paper + bounded live (+ TAP credential-isolation mode)
OKX · Binance	crypto	read + paper + bounded live
Futu	HK / US / A	read + paper + bounded live
MetaTrader 5	forex / CFD	read + paper + bounded live (Exness-style; demo ⇔ paper identity guard)
Longbridge · Dhan · Shoonya	US / HK · India (NSE/BSE)	read + paper only — no runtime paper/live discriminator, so live order placement is hard-refused
Trading 212	UK / EU	fully read-only — `place_order` / `cancel_order` hard-refuse even paper

Paper-vs-live is a structural per-broker runtime guard (account-id format, host separation, demo flag, or trade environment), never a config flag the agent can flip. A broker exposing no such discriminator is capped at paper + read-only.

Preset Trading Teams _{30 swarm presets}

🏢 30 ready-to-use agent teams
⚡ Pre-configured finance workflows
🎯 Investment, trading & risk management presets

Preset	Workflow
`investment_committee`	Bull/bear debate → risk review → PM final call
`global_equities_desk`	A-share + HK/US + crypto researcher → global strategist
`crypto_trading_desk`	Funding/basis + liquidation + flow → risk manager
`earnings_research_desk`	Fundamental + revision + options → earnings strategist
`macro_rates_fx_desk`	Rates + FX + commodity → macro PM
`quant_strategy_desk`	Screening + factor research → backtest → risk audit
`technical_analysis_panel`	Classic TA + Ichimoku + harmonic + Elliott + SMC → consensus
`risk_committee`	Drawdown + tail risk + regime review → sign-off
`global_allocation_committee`	A-shares + crypto + HK/US → cross-market allocation

_{Plus 20+ additional specialist presets — run vibe-trading --swarm-presets to explore all. Bring your own: drop preset YAMLs into ~/.vibe-trading/swarm/presets/ — they are listed alongside the bundled roster (same-name files override it, like user skills) and survive upgrades.}

Alpha Zoo _{462 pre-built quant alphas across 5 families}

🧬 462 cross-sectional alphas, lookahead-banned at the operator layer
📈 IC + IR + alive/reversed/dead categorisation in one CLI command
🔬 AST purity gate + 300-row lookahead sentinel test + pytest-socket network kill-switch
📦 Apache-2 attribution for Qlib; per-zoo LICENSE.md declaring formulas as mathematical content
🤝 Developer Certificate of Origin (DCO) sign-off workflow for community PRs

Zoo	Count	Source	License
qlib158	154	Microsoft Qlib `Alpha158` (Apache-2.0, commit-pinned)	Apache-2.0
alpha101	101	Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991	Formulas are mathematical content
gtja191	191	Guotai Junan (2014), "191 Short-period Trading Alpha Factors"	Formulas are mathematical content
academic	12	Fama-French 5 + Carhart momentum + Jegadeesh reversal + George-Hwang 52-week-high + Amihud illiquidity + Harvey-Siddique skew + Frazzini-Pedersen betting-against-beta + correlation-rewiring stability (price-based proxies)	Public academic literature
fundamental	4	PIT-safe SEC company facts — earnings yield, ROE, gross profitability, asset growth (filed-date anchored)	Public financial data

Run vibe-trading alpha list to browse, vibe-trading alpha show <id> for formulas + source, vibe-trading alpha bench --zoo X --universe Y --period Z to score a whole zoo, and vibe-trading alpha compare --all to rank zoos side by side.

Backtest Engines _{9 engines + options portfolio, cross-market composite}

Engine	Market	Notes
ChinaA	A-share	T+1, price limits, pre-ST filter
GlobalEquity	US / HK / Canada	same-session trading; market-specific lots, ticks, and costs
IndiaEquity	India (NSE/BSE)	T+1, circuit bands, config-driven STT / stamp / SEBI / GST cost stack
KoreaEquity	Korea (KRX: KOSPI/KOSDAQ)	long-only, ±30% band judged at execution time on the unified tick grid, 2026 0.20% transaction tax
Crypto	crypto spot / USD-M perps	funding settlements, execution/mark split
ChinaFutures · GlobalFutures	futures	margin, contract multipliers
Forex	FX / metals	via the `mt5` loader
Composite	cross-market	one shared capital pool across markets (`source="auto"`)
options_portfolio	options	multi-leg, Greeks, payoff/scenario

Intraday bars: 1m / 5m / 15m / 30m / 1H / 4H / 1D. 15 metrics + benchmark comparison, 5 portfolio optimizers (equal-volatility / risk-parity / mean-variance / max-diversification / turnover-aware), and 3 validation tools (Monte Carlo / Bootstrap / Walk-Forward).

🎬 Demo

https://github.com/user-attachments/assets/4e4dcb80-7358-4b9a-92f0-1e29612e6e86	https://github.com/user-attachments/assets/3754a414-c3ee-464f-b1e8-78e1a74fbd30
_{☝️ Natural-language backtest & multi-agent swarm debate — Web UI + CLI}

🚀 Quick Start

One-line install (PyPI)

pip install vibe-trading-ai

Then run a first research task:

vibe-trading init
vibe-trading run -p "Backtest a BTC-USDT 20/50 moving-average strategy for 2024 and summarize return and drawdown"

Upgrading from an older version? 0.1.10 moved to LangChain 1.x. If imports break after pip install -U vibe-trading-ai over a pre-0.1.10 install (e.g. langgraph fails to import), recreate the venv or run pip install --force-reinstall vibe-trading-ai. A fresh install is unaffected.

Package name vs commands: The PyPI package is vibe-trading-ai. Once installed, you get three commands:

Command Purpose

vibe-trading Interactive CLI / TUI

vibe-trading serve Launch FastAPI web server

vibe-trading-mcp Start MCP server (for Claude Desktop, OpenClaw, Cursor, etc.)

Command	Purpose
`vibe-trading`	Interactive CLI / TUI
`vibe-trading serve`	Launch FastAPI web server
`vibe-trading-mcp`	Start MCP server (for Claude Desktop, OpenClaw, Cursor, etc.)

vibe-trading init              # interactive .env setup
vibe-trading                   # launch CLI
vibe-trading serve --port 8899 # launch web UI
vibe-trading-mcp               # start MCP server (stdio)

Or choose a path

Path	Best for	Time
A. Docker	Try it now, zero local setup	2 min
B. Local install	Development, full CLI access	5 min
C. MCP plugin	Plug into your existing agent	3 min
D. ClawHub	One command, no cloning	1 min

Prerequisites

An LLM API key from any supported provider — or run locally with Ollama (no key needed)
Python 3.11+ for Path B
Docker for Path A
OpenAI Codex can also be used with ChatGPT OAuth: set LANGCHAIN_PROVIDER=openai-codex, then run vibe-trading provider login openai-codex. This does not use OPENAI_API_KEY.

Supported LLM providers: OpenRouter, Requesty, OpenAI, Anthropic (native Messages API), DeepSeek, Gemini, Groq, DashScope/Qwen, Zhipu, Moonshot/Kimi, MiniMax, SiliconFlow (CN + Global), Xiaomi MIMO, iFlytek Spark, Z.ai, NVIDIA NIM, ModelScope, Ollama (local). When no *_BASE_URL is set, each provider falls back to its canonical endpoint, so just a key is enough. See .env.example for config.

Tip: All markets work without any API keys thanks to automatic fallback. yfinance/Yahoo (HK/US/Canada), OKX (crypto), mootdx (A-shares, TCP-direct, no IP throttle), and AKShare (A-shares, US, HK, futures, forex) are all free. Tushare token is optional — mootdx is the preferred no-token A-share fallback, with AKShare as a broader backup.

Path A: Docker (zero setup)

git clone https://github.com/HKUDS/Vibe-Trading.git
cd Vibe-Trading
cp agent/.env.example agent/.env
# Edit agent/.env — uncomment your LLM provider and set API key
docker compose up --build

Open http://localhost:8899. Backend + frontend in one container.

Docker publishes the backend on 127.0.0.1:8899 by default and runs the app as a non-root container user. If you intentionally expose the API beyond your own machine, set a strong API_AUTH_KEY and send Authorization: Bearer <key> from clients.

Note

Using Ollama with Docker: the container reaches a host-side Ollama via host.docker.internal, not localhost (inside the container localhost is the container itself). docker-compose.yml defaults OLLAMA_BASE_URL to http://host.docker.internal:11434; export OLLAMA_BASE_URL (or set it in a top-level .env) to point elsewhere. This relies on the host-gateway mapping in extra_hosts, which requires Docker Engine ≥ 20.10 / Compose v2 (provided automatically on Docker Desktop).

Your data survives updates: persistent memory, the cross-session search index, user-created skills, shadow accounts, broker connector config, web sessions, backtest runs, swarm history, and uploads all live in named Docker volumes, so git pull && docker compose up --build keeps them. They are deleted only by docker compose down -v.

Path B: Local install

git clone https://github.com/HKUDS/Vibe-Trading.git
cd Vibe-Trading
python -m venv .venv

# Activate
source .venv/bin/activate          # Linux / macOS
# .venv\Scripts\activate.bat       # Windows CMD
# .venv\Scripts\Activate.ps1       # Windows PowerShell

pip install -e .
cp agent/.env.example agent/.env   # Edit — set your LLM provider API key
vibe-trading                       # Launch interactive TUI

Note

On Windows: cp is a PowerShell alias for Copy-Item, so the snippets above work as-is in PowerShell. CMD has no cp — use copy agent\.env.example agent\.env instead (this applies to the Docker snippet above too). If PowerShell refuses to run Activate.ps1, run Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned first; it applies to that shell session only.

Start web UI (optional)

# Terminal 1: API server
vibe-trading serve --port 8899

# Terminal 2: Frontend dev server
cd frontend && npm install && npm run dev  # requires Node >= 22.22

Open http://localhost:5899. The frontend proxies API calls to localhost:8899.

Production mode (single server):

cd frontend && npm run build && cd ..
vibe-trading serve --port 8899     # FastAPI serves dist/ as static files

Note

vibe-trading serve binds 0.0.0.0 and is loopback-only by default: opening the UI on the same machine (http://localhost:8899) works with zero config. If you browse from another machine, a VM host, or a phone on your LAN, sensitive endpoints return 403 and the chat shows "Remote API access requires an API key" — set a strong API_AUTH_KEY in agent/.env, restart, and enter the same key once in Settings. (Docker Desktop's host gateway: set VIBE_TRADING_TRUST_DOCKER_LOOPBACK=1 with the default 127.0.0.1 port bind.)

Path C: MCP plugin

See MCP Plugin section below.

Path D: ClawHub (one command)

npx clawhub@latest install vibe-trading --force

The skill + MCP config is downloaded into your agent's skills directory. See ClawHub install for details.

🧠 Environment Variables

Copy agent/.env.example to agent/.env and uncomment the provider block you want. Each provider needs 3-4 variables:

Variable	Required	Description
`LANGCHAIN_PROVIDER`	Yes	Provider name (`openrouter`, `deepseek`, `groq`, `ollama`, etc.)
`<PROVIDER>_API_KEY`	Yes*	API key (`OPENROUTER_API_KEY`, `DEEPSEEK_API_KEY`, etc.)
`<PROVIDER>_BASE_URL`	Yes	API endpoint URL
`LANGCHAIN_MODEL_NAME`	Yes	Model name (e.g. `deepseek-v4-pro`)
`TUSHARE_TOKEN`	No	Tushare Pro token for A-share data (falls back to AKShare)
`TIMEOUT_SECONDS`	No	LLM call timeout, default 120s
`API_AUTH_KEY`	Recommended for network deployments	Bearer token required when the API is reachable from non-local clients
`VIBE_TRADING_ENABLE_SHELL_TOOLS`	No	Explicit opt-in for shell-capable tools in remote API/MCP-SSE style deployments
`VIBE_TRADING_ALLOWED_FILE_ROOTS`	No	Extra comma-separated roots for document and broker-journal imports
`VIBE_TRADING_ALLOWED_RUN_ROOTS`	No	Extra comma-separated roots for generated-code run directories
`VIBE_TW_STOCK_DB`	No	Path to a Taiwan-market SQLite snapshot; the read-only `taiwan_stock_data` tool registers only when it is schema-valid
`VIBE_TRADING_EXTRA_CORS_ORIGINS`	No	Comma-separated origins added to the loopback CORS defaults (`CORS_ORIGINS` replaces them instead)
`CONTENT_FILTER_WARNING_THRESHOLD`	No	Content-filter warning ratio threshold (default 0.05 = 5%). When the ratio of LLM responses blocked by content moderation exceeds this, the run card warns you to switch providers.

_{* Ollama does not require an API key. OpenAI Codex uses ChatGPT OAuth and stores tokens via oauth-cli-kit, not in agent/.env.}

Free data (no key needed): A-shares via AKShare, HK/US/Canada equities via Yahoo/yfinance, crypto via OKX, 100+ crypto exchanges via CCXT. The system automatically selects the best available source for each market.

🎯 Recommended Models

Vibe-Trading is a tool-heavy agent — skills, backtests, memory, and swarms all flow through tool calls. Model choice directly decides whether the agent uses its tools or fabricates answers from training data.

Tier	Examples	When to use
Best	`anthropic/claude-opus-4.7`, `anthropic/claude-sonnet-4.6`, `openai/gpt-5.5-pro`, `google/gemini-3.5-flash`	Complex swarms (3+ agents), long research sessions, paper-grade analysis
Sweet spot (default)	`deepseek-v4-pro`, `deepseek/deepseek-v4-pro`, `x-ai/grok-4.20`, `z-ai/glm-5.1`, `moonshotai/kimi-k2.6`, `qwen/qwen3-max-thinking`	Daily driver — reliable tool-calling at ~1/10 the cost
Avoid for agent use	`-nano`, `-flash-lite`, `*-coder-next`, small / distilled variants	Tool-calling is unreliable — the agent will appear to "answer from memory" instead of loading skills or running backtests

The default agent/.env.example ships with DeepSeek official API + deepseek-v4-pro; OpenRouter users can use deepseek/deepseek-v4-pro.

🖥 CLI Reference

The interactive TUI (vibe-trading) now uses a terminal-native transcript: a startup banner, prompt rule, previous-turn recap, live activity rail, Markdown/table rendering, and run timing all stay in the CLI. Non-interactive invocations such as vibe-trading run, pipes, and --json remain script-friendly.

vibe-trading               # interactive TUI
vibe-trading run -p "..."  # single run
vibe-trading serve         # API server
vibe-trading alpha list    # browse 462 pre-built alphas; show / bench / compare / export-manifest sub-commands available
vibe-trading playbook list # five scheduled-research templates; show / create sub-commands available
vibe-trading channels status --local  # inspect IM channel config and install hints
vibe-trading provider doctor  # print redacted provider/proxy/package diagnostics

Slash commands inside TUI

Command	Description
`/help`	Show keyboard shortcuts and command list
`/model`	Switch LLM provider and model
`/memory`	Show / manage persistent memory
`/history`	Browse and resume prior sessions
`/goal`	Start / inspect a finance research goal
`/search`	Full-text search across all sessions
`/swarm`	Multi-agent presets (committee / quant / risk)
`/skill`	List / load / unload skills
`/show`	Show prior run by id
`/clear`	Clear current conversation
`/pine`	Export current strategy as Pine Script
`/journal`	Analyze trade journal CSV
`/shadow`	Train / view shadow account
`/export`	Export current session (md / json)
`/debug`	Toggle debug panel (token usage / latency)
`/comps`	Comparable company analysis (peer multiples -> implied range)
`/dcf`	Discounted cash flow valuation with sensitivity grid
`/attrib`	Brinson-Fachler attribution (allocation vs selection)
`/memo`	Investment memo — thesis, variant view, scenarios, kill criteria
`/earnings`	Earnings review — surprise bridge from revenue to EPS
`/screen`	Systematic idea screen — hypothesis, funnel, survivor queue
`/playbook`	Scheduled research templates (list / run / schedule)
`/connector`	Trading connector profiles (status / start / halt)
`/halt`	Kill switch — halt ALL live trading now
`/resume`	Clear the kill switch (re-enable live trading)
`/data`	Data routing mode
`/quit`	Exit (also: q, exit, :q)

Single run & flags

vibe-trading run -p "Backtest BTC-USDT MACD strategy, last 30 days"
vibe-trading run -p "Analyze AAPL momentum" --json
vibe-trading run -f strategy.txt
echo "Backtest 000001.SZ RSI" | vibe-trading run

vibe-trading -p "your prompt"
vibe-trading --skills
vibe-trading --swarm-presets
vibe-trading --swarm-run investment_committee '{"topic":"BTC outlook"}'
vibe-trading --list
vibe-trading --show <run_id>
vibe-trading --code <run_id>
vibe-trading --pine <run_id>           # Export indicators (TradingView + TDX + MT5)
vibe-trading --trace <run_id>
vibe-trading --continue <run_id> "refine the strategy"
vibe-trading --upload report.pdf

vibe-trading alpha list --zoo gtja191 --limit 10
vibe-trading alpha show gtja191_171
vibe-trading alpha bench --zoo gtja191 --universe csi300 --period 2018-2025 --top 20

IM channels

IM channel adapters connect outside chat apps to the same session runtime used by the Web UI and CLI. Configure enabled adapters under channels in ~/.vibe-trading/agent.json; SDK-backed adapters are optional extras, and missing SDKs report recovery hints instead of crashing the runtime.

For long-running channel tasks, tune the central assistant-reply wait budget with replyTimeoutS (seconds, default 600):

{
  "channels": {
    "replyTimeoutS": 1800,
    "feishu": {
      "enabled": true
    }
  }
}

This controls how long the shared channel runtime waits for the agent session to produce an assistant message; adapter HTTP/socket timeouts remain adapter-specific.

vibe-trading channels status --local   # inspect config and missing SDK hints without API
vibe-trading channels status           # query the running API runtime
vibe-trading channels start            # start enabled adapters through the API
vibe-trading channels stop             # stop enabled adapters through the API
vibe-trading channels login weixin     # run an adapter login hook when needed
vibe-trading channels pairing --channel telegram list

The built-in adapters cover websocket, telegram, slack, discord, matrix, whatsapp, signal, qq, napcat, weixin, wecom, feishu, dingtalk, msteams, email, and mochat. Use narrow extras such as pip install "vibe-trading-ai[telegram]", or install the full channel set with pip install "vibe-trading-ai[channels]".

In-chat slash commands (channel-agnostic, work in all 16 adapters):

Command	Description
`/new`	Reset the current session — the next message starts a fresh conversation
`/reset`	Alias for `/new`
`/newsession`	Alias for `/new`
`/pairing list`	Show pending sender-pairing requests (operators only)

Commands are case-insensitive and must be sent as the entire message (e.g. hello /new is treated as a regular message, not a reset).

/pairing is operator-gated. In-chat pairing-control commands are rejected unless the sender is listed as an operator — set channels.operators (cross-channel authority) or a channel section's own operators list in your channels config. With no operators configured, in-chat /pairing is refused (fail-closed) and pairing is managed only through the authenticated CLI (vibe-trading channels pairing …) and the auth-gated REST endpoint. This prevents any allow-listed group member from taking over pairing across channels.

💡 Examples

Strategy & Backtesting

# Moving average crossover on US equities
vibe-trading run -p "Backtest a 20/50-day moving average crossover on AAPL for the past year, show Sharpe ratio and max drawdown"

# RSI mean-reversion on crypto
vibe-trading run -p "Test RSI(14) mean-reversion on BTC-USDT: buy below 30, sell above 70, last 6 months"

# Multi-factor strategy on A-shares
vibe-trading run -p "Backtest a momentum + value + quality multi-factor strategy on CSI 300 constituents over 2 years"

# After backtesting, export to TradingView / TDX / MetaTrader 5
vibe-trading --pine <run_id>

Bench a pre-built alpha zoo (one line):

vibe-trading alpha bench --zoo gtja191 --universe csi300 --period 2018-2025 --top 20

Browse the catalogue and inspect a single alpha:

vibe-trading alpha list --zoo gtja191 --theme reversal --limit 10
vibe-trading alpha show gtja191_171

Compose a multi-factor signal from the zoo (Python):

from src.skills.multi_factor.zoo_signal_engine import ZooSignalEngine
engine = ZooSignalEngine.from_zoo(["gtja191_171", "gtja191_111", "gtja191_163"])
panel = ...  # your wide OHLCV panel
signal = engine.compute_signal(panel)

Market Research

# Equity deep-dive
vibe-trading run -p "Research NVDA: earnings trend, analyst consensus, option flow, and key risks for next quarter"

# Macro analysis
vibe-trading run -p "Analyze the current Fed rate path, USD strength, and impact on EM equities and gold"

# Crypto on-chain
vibe-trading run -p "Deep dive BTC on-chain: whale flows, exchange balances, miner activity, and funding rates"

Swarm Workflows

# Bull/bear debate on a stock
vibe-trading --swarm-run investment_committee '{"topic": "Is TSLA a buy at current levels?"}'

# Quant strategy from screening to backtest
vibe-trading --swarm-run quant_strategy_desk '{"universe": "S&P 500", "horizon": "3 months"}'

# Crypto desk: funding + liquidation + flow → risk manager
vibe-trading --swarm-run crypto_trading_desk '{"asset": "ETH-USDT", "timeframe": "1w"}'

# Global macro portfolio allocation
vibe-trading --swarm-run macro_rates_fx_desk '{"focus": "Fed pivot impact on EM bonds"}'

Cross-Session Memory

# Save your preferences once
vibe-trading run -p "Remember: I prefer RSI-based strategies, max 10% drawdown, hold period 5–20 days"

# The agent recalls them in future sessions automatically
vibe-trading run -p "Build a crypto strategy that fits my risk profile"

Upload & Analyze Documents

# Analyze a broker export or earnings report
vibe-trading --upload trades_export.csv
vibe-trading run -p "Profile my trading behavior and identify any biases"

vibe-trading --upload NVDA_Q1_earnings.pdf
vibe-trading run -p "Summarize the key risks and beats/misses from this earnings report"

🌐 API Server

vibe-trading serve --port 8899

Method	Endpoint	Description
`GET`	`/runs`	List runs
`GET`	`/runs/{run_id}`	Run details
`GET`	`/runs/{run_id}/pine`	Multi-platform indicator export
`POST`	`/sessions`	Create session
`POST`	`/sessions/{id}/messages`	Send message
`GET`	`/sessions/{id}/events`	SSE event stream
`POST`	`/upload`	Upload PDF/file
`GET`	`/swarm/presets`	List swarm presets
`POST`	`/swarm/runs`	Start swarm run
`GET`	`/swarm/runs/{id}/events`	Swarm SSE stream
`GET`	`/alpha/list`	List alphas (filter by zoo/theme/universe)
`GET`	`/alpha/{alpha_id}`	Alpha metadata + source code
`POST`	`/alpha/bench`	Start a bench job (returns `job_id`)
`GET`	`/alpha/bench/{job_id}/stream`	SSE progress stream
`GET`	`/settings/llm`	Read Web UI LLM settings
`PUT`	`/settings/llm`	Update local LLM settings
`GET`	`/settings/data-sources`	Read local data source settings
`PUT`	`/settings/data-sources`	Update local data source settings
`GET`	`/channels/status`	Read IM channel runtime and adapter status
`POST`	`/channels/start`	Start configured IM channel adapters
`POST`	`/channels/stop`	Stop configured IM channel adapters
`POST`	`/channels/pairing/command`	Run a sender-pairing command against the shared store
`POST`	`/scheduled-runs`	Create a scheduled research job (interval-ms or cron)
`GET`	`/scheduled-runs`	List scheduled jobs
`DELETE`	`/scheduled-runs/{job_id}`	Cancel a scheduled job
`GET`	`/scheduled-runs/playbooks`	List the research templates
`GET`	`/scheduled-runs/playbooks/{slug}`	Show one template, with its variables
`POST`	`/scheduled-runs/playbooks/{slug}`	Schedule a job from a template
`POST`	`/sessions/{id}/cancel`	Stop the session's in-flight run (recorded as cancelled, not failed)
`POST`	`/sessions/{id}/title/auto`	Summarize the first exchange into a session title (never overwrites a manual rename)
`GET`	`/correlation/regime`	Correlation edge-density regime timeline
`GET`	`/agents.json` · `POST` `/v1/query`	OpenBB Workspace bridge — registered only with the optional `openbb` extra; `/v1/query` requires auth

Interactive docs are available at http://localhost:8899/docs in keyless loopback development mode. When API_AUTH_KEY is configured, /docs and /redoc are disabled; authenticated tooling can fetch /openapi.json with an Authorization: Bearer <key> header.

Security defaults

For localhost development, vibe-trading serve keeps the browser workflow simple. For any non-local client, sensitive API endpoints require API_AUTH_KEY; use Authorization: Bearer <key> for JSON/upload requests. Browser EventSource streams are handled by the Web UI after you enter the same key once in Settings.

Shell-capable process tools (bash / background_run / cancel_background) are enabled only for the interactive local CLI. Every other surface — the HTTP/SSE API and the MCP server on all transports (stdio included) — keeps them off unless you explicitly opt in with VIBE_TRADING_ENABLE_SHELL_TOOLS=1 (or pass --enable-shell-tools to vibe-trading-mcp). Transport type never implicitly grants shell access. cancel_background stops only the tracked task ID returned by background_run; broad Python process-name termination is refused because it could terminate Vibe-Trading itself. Document and journal readers are limited to upload/import roots by default; place files under ~/.vibe-trading/uploads, ~/.vibe-trading/runs, ./uploads, ./data (or the legacy agent/uploads / agent/runs), or add a dedicated directory through VIBE_TRADING_ALLOWED_FILE_ROOTS. Sessions, runs, swarm runs, uploads, and the sessions.db index live under ~/.vibe-trading (relocatable via the VIBE_TRADING_HOME shell environment variable); pre-existing history is moved there automatically on first run.

Generated backtest code runs as a local Python subprocess and can make network requests through the configured market-data loaders. Its environment is intentionally narrow: the runner keeps OS/Python basics, proxy/certificate settings, VIBE_TRADING_ALLOWED_RUN_ROOTS, and read-only market-data keys such as TUSHARE_TOKEN, FMP_API_KEY, FRED_API_KEY, and VIBE_TRADING_IWENCAI_KEY. It does not pass LLM provider keys, API auth tokens, shell-tool switches, broker trading secrets, or live/advisory toggles to generated strategy code by default.

Web UI Settings

The Web UI Settings page lets local users update the LLM provider/model, base URL, generation parameters, reasoning effort, and optional market data credentials such as the Tushare token. Settings are persisted to agent/.env; provider defaults are loaded from agent/src/providers/llm_providers.json.

Settings reads are side-effect free: GET /settings/llm and GET /settings/data-sources never create agent/.env, and they only return project-relative paths. Settings reads and writes can expose credential state or update credentials/runtime environment, so they require API_AUTH_KEY when configured. If API_AUTH_KEY is unset for dev mode, settings access is accepted only from loopback clients.

The same Settings page includes an IM Channels panel for local operators. It polls /channels/status, shows configured/enabled/available/loaded/running states, surfaces adapter recovery hints, and can start or stop the configured channel runtime without going back to the terminal.

Scheduled research

Run a research prompt or backtest on a repeating schedule — from the Scheduled page in the web UI or over REST. The background executor is off by default — start the server with VIBE_TRADING_ENABLE_SCHEDULER=1 to enable it:

VIBE_TRADING_ENABLE_SCHEDULER=1 vibe-trading serve --port 8899

Then create jobs over REST. schedule is either a bare integer (interval in milliseconds) or a 5-field cron expression (min hour dom mon dow; each field takes *, */n, numbers, comma lists, or low-high ranges like 1-5). Cron runs on the wall clock of the job's optional timezone (an IANA key), so the cadence holds across DST transitions — a spring-forward gap time is skipped, and a fall-back ambiguous time runs once, at its first occurrence. Jobs without a timezone keep plain UTC semantics:

# every 6 hours (cron)
curl -X POST http://localhost:8899/scheduled-runs \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Scan CSI300 for momentum breakouts and backtest the top 5","schedule":"0 */6 * * *"}'

# weekdays at 23:30 Auckland wall time — DST-proof
curl -X POST http://localhost:8899/scheduled-runs \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Pre-open scan of NZX names","schedule":"30 23 * * 1-5","timezone":"Pacific/Auckland"}'

# list / cancel
curl http://localhost:8899/scheduled-runs
curl -X DELETE http://localhost:8899/scheduled-runs/<job_id>

Each fire runs the prompt through a fresh agent session (optional backtest parameters go in config), and jobs persist under ~/.vibe-trading/ so they survive restarts. Without the flag, the /scheduled-runs endpoints still record jobs but nothing fires. Add -H "Authorization: Bearer <key>" to each call when API_AUTH_KEY is set.

Five ready-to-schedule templates ship with the scheduler — premarket-brief, earnings-season-tracker, portfolio-checkup, a-share-money-flow, institutional-holdings-diff. Each states the data a run needs in plain language instead of naming tools, so a template keeps working as the tool surface grows, and each is required to name a missing input rather than fill it from memory. Reach them from the CLI, over REST, or with /playbook in the TUI:

vibe-trading playbook list                     # the five templates
vibe-trading playbook show premarket-brief     # body, declared variables, suggested cadence
vibe-trading playbook create premarket-brief \
  --var home_market="US equities" --var watchlist="AAPL, MSFT, NVDA" \
  --timezone America/New_York

curl http://localhost:8899/scheduled-runs/playbooks
curl http://localhost:8899/scheduled-runs/playbooks/premarket-brief
curl -X POST http://localhost:8899/scheduled-runs/playbooks/premarket-brief \
  -H "Content-Type: application/json" \
  -d '{"variables":{"home_market":"US equities","watchlist":"AAPL, MSFT, NVDA"}}'

Posting {} schedules a template on its own suggested cadence with its declared defaults. The rendered body becomes the job prompt verbatim, and an undeclared variable is rejected rather than silently ignored.

🔌 MCP Plugin

Vibe-Trading exposes 64 MCP tools for any MCP-compatible client. Runs as a stdio subprocess — no server setup needed. Core research tools work with zero API keys for HK/US/crypto; trading connector tools use the selected connector profile, and run_swarm needs an LLM key.

Environment variables: the client spawns the server itself, so a shell export never reaches it — set them in the client's env block. Generated backtest code is sandboxed to the allowed run roots, so writing results into a workspace of your own needs VIBE_TRADING_ALLOWED_RUN_ROOTS:

{
  "mcpServers": {
    "vibe-trading": {
      "command": "vibe-trading-mcp",
      "env": { "VIBE_TRADING_ALLOWED_RUN_ROOTS": "C:\\Users\\me\\research" }
    }
  }
}

Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "vibe-trading": {
      "command": "vibe-trading-mcp"
    }
  }
}

OpenClaw

Add to ~/.openclaw/config.yaml:

skills:
  - name: vibe-trading
    command: vibe-trading-mcp

For a first research-only smoke test, confirm tool discovery and run a market data or backtest request before selecting a trading connector profile. Core research tools can run without broker credentials; connector-backed trading_* tools should be used only after you intentionally select and check a connector profile. run_swarm requires an LLM key.

Cursor / Windsurf / other MCP clients

vibe-trading-mcp                   # stdio (default)
vibe-trading-mcp --transport http  # Streamable HTTP (current MCP spec default) at http://127.0.0.1:8900/mcp
vibe-trading-mcp --transport sse   # legacy SSE (deprecated) for older clients

For HTTP clients (QwenPaw, and any client that negotiates by POSTing an InitializeRequest), use --transport http and point the client at the single /mcp endpoint — e.g. http://127.0.0.1:8900/mcp. Do not point an HTTP client at /sse; that path belongs to the deprecated two-endpoint SSE transport and will return 405 Method Not Allowed on POST. Override the bind address with --host / --port.

MCP tools exposed (64): list_skills, load_skill, start_research_goal, get_research_goal, add_goal_evidence, update_research_goal_status, backtest, factor_analysis, alpha_zoo, alpha_bench, analyze_options, analyze_options_payoff, pattern_recognition, read_url, read_document, web_search, write_file, read_file, trading_connections, trading_select_connection, trading_check, trading_account, trading_positions, trading_orders, trading_quote, trading_history, list_swarm_presets, run_swarm, get_market_data, get_fund_flow, get_dragon_tiger, get_northbound_flow, get_margin_trading, get_block_trades, get_shareholder_count, get_lockup_expiry, get_sector_info, get_research_reports, get_stock_news, get_sec_filings, get_financial_statements, get_options_chain, get_stock_profile, screen_market, search_symbol, get_macro_series, iwencai_search, qveris_search, qveris_inspect, qveris_execute, get_institutional_holdings, etf_holdings, prediction_market, research_papers, get_swarm_status, get_run_result, list_runs, reap_stale_runs, retry_run, analyze_trade_journal, extract_shadow_strategy, run_shadow_backtest, render_shadow_report, scan_shadow_signals.

SWARM external MCP tools

run_swarm workers can call operator-approved tools from external MCP servers. Configure the server-side allowlist in VIBE_TRADING_SWARM_AGENT_CONFIG, ~/.vibe-trading/swarm-agent.json, or the fallback ~/.vibe-trading/agent.json; then list remote tools in a swarm preset using the local MCP wrapper name, such as mcp_internal_kb_search. Caller-provided variables stay template data only and cannot inject MCP URLs, commands, environment variables, or allowlist overrides.

Install from ClawHub (one command)

npx clawhub@latest install vibe-trading --force

--force is required because the skill references external APIs, which triggers VirusTotal's automated scan. The code is fully open-source and safe to inspect.

This downloads the skill + MCP config into your agent's skills directory. No cloning needed.

Browse on ClawHub: clawhub.ai/skills/vibe-trading

OpenSpace — self-evolving skills

All 89 finance skills are published on open-space.cloud and evolve autonomously through OpenSpace's self-evolution engine.

To use with OpenSpace, add both MCP servers to your agent config:

{
  "mcpServers": {
    "openspace": {
      "command": "openspace-mcp",
      "toolTimeout": 600,
      "env": {
        "OPENSPACE_HOST_SKILL_DIRS": "/path/to/vibe-trading/agent/src/skills",
        "OPENSPACE_WORKSPACE": "/path/to/OpenSpace"
      }
    },
    "vibe-trading": {
      "command": "vibe-trading-mcp"
    }
  }
}

OpenSpace will auto-discover all 89 skills, enabling auto-fix, auto-improve, and community sharing. Search for Vibe-Trading skills via search_skills("finance backtest") in any OpenSpace-connected agent.

MetaTrader 5 (Exness and other MT5 brokers)

Connects to a locally running MT5 terminal through the official MetaTrader5 package (Windows only):

pip install "vibe-trading-ai[mt5]"

Configure ~/.vibe-trading/mt5.json (create it yourself; chmod 600 where supported):

{
  "login": 12345678,
  "password": "...",
  "server": "Exness-MT5Trial8",
  "symbol_suffix": "m",
  "max_order_volume": 1.0,
  "max_order_notional_usd": 10000
}

Then:

vibe-trading connector use mt5-paper-sdk
vibe-trading connector check
vibe-trading connector account
vibe-trading connector quote EURUSD
vibe-trading connector history EURUSD

Profile	Account	Orders
`mt5-paper-sdk`	demo	read-only
`mt5-live-sdk-readonly`	real	read-only
`mt5-paper-trade`	demo	direct placement (connector per-order size guards apply)
`mt5-live-trade`	real	mandate + kill-switch gated

Safety boundary: "paper" means the broker's own demo account, re-verified on every call — the terminal reports account_info().trade_mode and the logged-in account number, so pointing a paper profile at a real-money account (or the reverse) is refused outright. MT5 sizes orders in lots (1 lot EURUSD = 100,000 EUR); the live mandate gate prices lots through the connector's USD hook, and the connector's own max_order_volume / max_order_notional_usd guards apply on demo as well as live, failing closed when a notional cannot be priced. On hedging accounts (the Exness default), note that an opposing order opens a hedge position — close by ticket instead (pass the position ticket to trading_cancel_order) so the fill is pinned to that position and can only reduce exposure. Rollback / halt path: the kill switch blocks new live orders, while cancellation stays available and is written to the audit log. Mandate limits are denominated in USD; a non-USD account currency is margined by the broker in its own currency.

The mt5 market-data loader — the head of the forex fallback chain — shares this same mt5.json. With no such file it attaches read-only to the most recently used terminal that is already logged in.

🔌 eToro Public API Connector

Connects to eToro's Public API for demo and real accounts via API key pair (x-api-key + x-user-key). Demo and real environments are separated structurally: demo keys only reach /demo API paths.

Configure ~/.vibe-trading/etoro.json (create it yourself; chmod 600 where supported):

{
  "api_key": "YOUR_PUBLIC_API_KEY",
  "user_key": "YOUR_USER_KEY",
  "profile": "paper"
}

Alternatively set ETORO_API_KEY and ETORO_USER_KEY in ~/.vibe-trading/.env.

Then:

vibe-trading connector use etoro-paper-sdk
vibe-trading connector check
vibe-trading connector account
vibe-trading connector positions
vibe-trading connector quote BTC

Profile	Account	Orders
`etoro-paper-sdk`	demo	read-only
`etoro-live-sdk-readonly`	real	read-only
`etoro-paper-trade`	demo	direct placement on demo paths
`etoro-live-trade`	real	mandate + kill-switch gated

Symbol lookup uses eToro's internalSymbolFull search (e.g. BTC → instrument id 100000). Use the etoro_search_instruments agent tool to resolve tickers before trading.

Safety boundary: demo and real are path-separated and key-bound (paper_guard: path_separated_key_bound). Live risk-increasing actions (open and copy-start/increase) require an authorized mandate, a clear halt state, and a verified USD account for copy-notional enforcement. Validated full and partial position closes, open-order cancellation, and copy close remain available when halted and are audit-logged. Cancelling a pending close or editing position stops is paper-only: the live path fails closed because those operations can increase exposure or transfer extra margin without enough API data to quantify the incremental USD risk. Copy amounts are denominated in the eToro account currency, and every copy start/adjust requires a caller-supplied 1-35 character URL-safe reference id for polling. eToro-specific write tools (etoro_close_position, etoro_copy_*, etc.) are agent tools only — not exposed via MCP or CLI. Rollback: revert the connector commit(s) or disable profiles; halt blocks new live risk-increasing actions.

🔌 Loading Tools from External MCP Servers (MCP Client Mode)

This is the opposite direction from the MCP Plugin above. The MCP Plugin lets other agents call Vibe-Trading tools. This section lets the built-in Vibe-Trading agent call tools from your external MCP servers.

Quick start

Create ~/.vibe-trading/agent.json:

{
  "mcpServers": {
    "my-server": {
      "command": "uvx",
      "args": ["my-mcp-server"]
    }
  }
}

Run any CLI command — tools from ordinary external servers are automatically injected into the agent's registry after local tools:

vibe-trading run "use my-server to do X"

Official IBKR MCP read-only probe

Vibe-Trading can connect directly to Interactive Brokers' official remote MCP endpoint in read-only mode. Add this to ~/.vibe-trading/agent.json:

{
  "mcpServers": {
    "ibkr": {
      "type": "streamableHttp",
      "url": "https://api.ibkr.com/v1/api/mcp",
      "auth": {
        "type": "oauth",
        "scopes": ["mcp.read"],
        "clientName": "Vibe-Trading",
        "cacheDir": "~/.vibe-trading/live/ibkr/oauth"
      },
      "enabledTools": ["*"]
    }
  }
}

Then start the browser OAuth flow:

vibe-trading connector authorize ibkr-live-official-mcp-readonly

The wildcard is accepted only for IBKR's mcp.read probe. Authorizing this profile confirms access to IBKR's official read scope; generic trading_account and trading_positions calls stay disabled until IBKR publishes stable read tool names that Vibe-Trading can map safely. A config that adds mcp.write must pin an explicit tool allowlist and still passes through the live order guard.

If IBKR issues a pre-registered OAuth client, add clientId and clientSecret inside auth.

Trading connectors: fastest path

For users who cannot wait for IBKR OAuth client approval, connect to a local TWS or IB Gateway session. Credentials stay inside IBKR's desktop app; Vibe- Trading only connects to 127.0.0.1 and exposes it as a connector profile.

Install the optional SDK:

pip install "vibe-trading-ai[ibkr]"

Open TWS paper trading or IB Gateway paper, enable API socket clients, then run:

vibe-trading connector list
vibe-trading connector use ibkr-paper-local
vibe-trading connector configure ibkr-paper-local --yes
vibe-trading connector check
vibe-trading connector account
vibe-trading connector positions
vibe-trading connector orders
vibe-trading connector quote AAPL
vibe-trading connector history AAPL --duration "30 D" --bar-size "1 day"

Default local ports:

App	Paper	Live read-only
TWS	`7497`	`7496`
IB Gateway	`4002`	`4001`

The agent exposes connector-scoped tools named trading_connections, trading_select_connection, trading_check, trading_account, trading_positions, trading_orders, trading_quote, and trading_history. Live-broker raw MCP tools are not registered directly as mcp_<broker>_*. No IBKR order-placement tool is registered.

🔐 TAP Mode — full credential isolation & human-approved writes

Opt-in, off by default. If the TAP_* variables below are unset, the connector behaves exactly as before (direct broker SDK) — nothing changes.

TAP (Tool Authorization Protocol) is a credential proxy: the agent never holds the raw broker API secret, and consequential writes are gated on human approval. With TAP mode on, every Alpaca call — order placement, cancel, and the reads (account/positions/orders/quote/bars) — is sent to the TAP proxy's /forward endpoint instead of the broker SDK; TAP injects the real key server-side, then forwards upstream.

The agent process holds no Alpaca key at all — and doesn't even need alpaca-py — because the whole egress goes through TAP. The secret is referenced by name (<CREDENTIAL:alpaca.key_id>) and TAP substitutes it.
Writes block on human approval. An order or cancel cannot reach the broker without a human approving it; even a prompt-injected "buy now" is held, and denying it means it never reaches Alpaca. Orders carry a deterministic client_order_id, so an approval-race retry is deduplicated rather than double-placed.
Reads auto-approve. Account/positions/orders/quote/bars are GETs that TAP forwards without a human step — this is credential isolation (no key in the process), not a gate, so there's ~zero added friction.
allowed_hosts on the TAP credential pins where the key may be sent, so a tampered target is rejected (403) before injection.

Enable it:

In the TAP dashboard, create a multi-secret credential named alpaca holding your Alpaca key pair as fields key_id and secret_key, assigned to your agent, with allowed hosts paper-api.alpaca.markets (or the live host api.alpaca.markets) and data.alpaca.markets (the market-data host used by quote/bars). Use separate TAP credentials for paper and live (e.g. alpaca-paper / alpaca-live, selected via TAP_ALPACA_CREDENTIAL), each with allowed_hosts pinned to its own API host — TAP then structurally refuses to send the paper key to the live host and vice versa, keeping the paper/live separation crisp end to end.
Add to agent/.env:

Variable	Required	Description
`TAP_PROXY_URL`	Yes	TAP proxy base URL (e.g. `https://proxy.tap.human.tech`)
`TAP_AGENT_KEY`	Yes	Your TAP agent API key (secret)
`TAP_ALPACA_CREDENTIAL`	No	TAP credential name for Alpaca (default `alpaca`)
`TAP_APPROVAL_TIMEOUT`	No	Seconds to wait for a human decision (default `300`)

When a write is placed, approve or deny it in your TAP channel (Telegram / dashboard). An approved order/cancel is forwarded to Alpaca; a denied or timed-out one returns an error and is never sent.

Known limitation — approval race. If the human approves right at the TAP_APPROVAL_TIMEOUT boundary, TAP may forward the order while the poll has already given up: the gate then reports an error even though the order reached the broker, and the max_trades_per_day counter under-counts by one. The deterministic client_order_id keeps a retry from double-placing that order; if you rely on a tight trades-per-day cap, check open orders after a TAP timeout error before retrying.

Scope: covers Alpaca order placement, cancel, and all five reads — the full connector egress, so the process holds no key on any path. HMAC-signed brokers (Binance/OKX) are follow-ups (client-side signing doesn't fit pure egress injection). The hooks are additive — they live inside the Alpaca connector and leave the live mandate gate unchanged.

Config reference

Field	Type	Default	Description
`type`	string	inferred for stdio; required for HTTP	Omit for stdio, or set to `sse` / `streamableHttp` for URL-based servers.
`command`	string	required for stdio	Executable to spawn for stdio servers. Invalid for `sse` / `streamableHttp` servers.
`args`	array	`[]`	Command-line arguments for stdio servers only.
`env`	object	`{}`	Extra environment variables merged into the subprocess env for stdio servers only.
`url`	string	required for `sse` / `streamableHttp`	Remote SSE / streamable HTTP endpoint URL. Not used for stdio servers.
`headers`	object	`{}`	Extra HTTP headers for `sse` / `streamableHttp` servers only.
`toolTimeout`	number	`30`	Per-tool call timeout in seconds
`initTimeout`	number	unset (`max(toolTimeout, 30)`)	MCP initialize / OAuth authorization timeout in seconds. Use this for slow browser authorization without widening ordinary tool calls.
`enabledTools`	array	`["*"]`	Tool allowlist. Use `["*"]` to expose all tools from the server

Config file location: ~/.vibe-trading/agent.json (JSON or YAML).

For URL-based transports, type is required. The agent no longer guesses between SSE and streamable HTTP from the URL suffix.

Per-session overrides (API)

When creating a session via the API you can pass mcpServers inside session.config to extend or override the global config for that session only:

{
  "config": {
    "mcpServers": {
      "research-server": {
        "command": "uvx",
        "args": ["research-mcp"],
        "enabledTools": ["search", "fetch"]
      }
    }
  }
}

Tool naming

Ordinary remote tools are exposed with stable names: mcp_<server>_<tool>. Live-broker MCP servers stay behind the trading_* connector surface.

If two server names produce the same ASCII-safe local prefix (e.g. foo-bar and foo_bar both become foo_bar), a deterministic hash suffix is appended at the server-segment level so names remain unique. The operator receives a warning:

WARNING: Configured MCP server 'foo-bar' collides with another server after local name
normalization. Using local tool prefix 'mcp_foo_bar_<hash>_<tool>' to keep generated
tool names unique. Rename the server in agent config if you want a different prefix.

v1 limits

Limit	Detail
Transport	stdio, SSE, and streamable HTTP
Execution	serial only — MCP tools never enter the parallel readonly path
Surfaces	tools only (resources and prompts excluded in v1)
Hot reload	not supported — restart the process to pick up config changes
Swarm path	MCP tools are not available inside Swarm worker registries in v1

📁 Project Structure

Click to expand

Vibe-Trading/
├── agent/                          # Backend (Python)
│   ├── cli/                        # CLI package — interactive TUI + subcommands
│   ├── api_server.py               # FastAPI server — runs, sessions, upload, swarm, SSE
│   ├── mcp_server.py               # MCP server — 64 tools for OpenClaw / Claude Desktop
│   │
│   ├── src/
│   │   ├── agent/                  # ReAct agent core
│   │   │   ├── loop.py             #   5-layer compression + read/write tool batching
│   │   │   ├── context.py          #   system prompt + auto-recall from persistent memory
│   │   │   ├── skills.py           #   skill loader (89 bundled + user-created via CRUD)
│   │   │   ├── tools.py            #   tool base class + registry
│   │   │   ├── memory.py           #   lightweight workspace state per run
│   │   │   ├── frontmatter.py      #   shared YAML frontmatter parser
│   │   │   └── trace.py            #   execution trace writer
│   │   │
│   │   ├── memory/                 # Cross-session persistent memory
│   │   │   └── persistent.py       #   file-based memory (~/.vibe-trading/memory/)
│   │   │
│   │   ├── tools/                  # 68 auto-discovered agent tools
│   │   │   ├── backtest_tool.py    #   run backtests
│   │   │   ├── remember_tool.py    #   cross-session memory (save/recall/forget)
│   │   │   ├── skill_writer_tool.py #  skill CRUD (save/patch/delete/file)
│   │   │   ├── session_search_tool.py # FTS5 cross-session search
│   │   │   ├── swarm_tool.py       #   launch swarm teams
│   │   │   ├── web_search_tool.py  #   DuckDuckGo web search
│   │   │   └── ...                 #   bash, file I/O, factor analysis, options, alpha browser + bench, etc.
│   │   │
│   │   ├── factors/                # Alpha Zoo — 462 alphas across 5 families
│   │   │   ├── base.py             #   19 operators (rank/scale/ts_*/delta/decay_linear/safe_div/vwap)
│   │   │   ├── registry.py         #   AST-only metadata load + lazy compute + sanity gates
│   │   │   ├── bench_runner.py     #   IC + alive/reversed/dead categorisation
│   │   │   └── zoo/                #   qlib158 (154) + alpha101 (101) + gtja191 (191) + academic (12) + fundamental (4)
│   │   │
│   │   ├── api/                    # FastAPI route modules
│   │   │   └── alpha_routes.py     #   /alpha/list, /alpha/{id}, /alpha/bench, SSE stream
│   │   │
│   │   ├── skills/                 # 89 finance skills in 9 categories (SKILL.md each)
│   │   ├── swarm/                  # Swarm DAG execution engine
│   │   │   └── presets/            #   30 swarm preset YAML definitions
│   │   ├── session/                # Multi-turn chat + FTS5 session search
│   │   └── providers/              # LLM provider abstraction
│   │
│   └── backtest/                   # Backtest engines
│       ├── engines/                #   8 engines + composite cross-market engine + options_portfolio
│       ├── loaders/                #   24 sources: tushare, okx, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker
│       │   ├── base.py             #   DataLoader Protocol
│       │   └── registry.py         #   Registry + auto-fallback chains
│       └── optimizers/             #   MVO, equal vol, max div, risk parity
│
├── frontend/                       # Web UI (React 19 + Vite + TypeScript)
│   └── src/
│       ├── pages/                  #   Home, Agent, AlphaZoo, RunDetail, Compare, Correlation, Settings
│       ├── components/             #   chat, charts, layout
│       └── stores/                 #   Zustand state management
│
├── Dockerfile                      # Multi-stage build
├── docker-compose.yml              # One-command deploy
├── pyproject.toml                  # Package config + CLI entrypoint
├── tools/                          # Repo-level CI helpers
│   └── ci_grep_gates.sh            # rejects yaml.load / trademark / per-stock-data leaks
└── LICENSE                         # MIT

🏛 Ecosystem

Vibe-Trading is part of the HKUDS agent ecosystem:

NanoBot
_{Ultra-Lightweight Personal AI Assistant}

AI-Trader
_{Agent-Native Signal & Copy Trading Platform}

CLI-Anything
_{Making All Software Agent-Native}

OpenSpace
_{Self-Evolving AI Agent Skills}

ClawTeam
_{Agent Swarm Intelligence}

🗺 Roadmap

We ship in phases. Items move to Issues when work begins.

Phase	Feature	Status
Trust Layer	Reproducible run cards are emitted and shown in Run Detail; v1 adds tool traces and citations	v0 Shipped
Hypothesis Registry	Durable research hypotheses with lifecycle status, data sources, skills, run-card links, and invalidation notes	Backend MVP Shipped
Research Autopilot	Manual-first research loop: hypothesis → deterministic backtest → evidence report	Phase 1–3 Shipped
Data Bridge	Bring-your-own data: local CSV/Parquet/SQL connectors with schema mapping	Local loader Shipped
Options Lab	Vol surface, Greeks dashboard, payoff/scenario explorer	Analytic payoff/scenario tool Shipped; surface/dashboard Planned
Portfolio Studio	Risk x-ray, constraints, turnover-aware optimizer, rebalance notes	Turnover-aware optimizer Shipped 0.1.11; rest Planned
Alpha Zoo	462 pre-built alphas (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental) with one-line bench, agent integration, and Web UI	Shipped 0.1.8, extended through 0.1.12
Strategy Development Manager	Register papers / broker research as factors & strategies with a persistent store + automated IC/Sharpe decay lifecycle	Shipped 0.1.11
Correlation Regime	Edge-density + hysteresis regime timeline layered on `/correlation` — spot when markets fuse into one bloc	Shipped 0.1.12
Research Delivery	Scheduled briefs and live research sessions through Slack / Telegram / email-style IM channels	Scheduler + IM Runtime Shipped
Community	Shareable skills, presets, and strategy cards	Exploring

Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

Good first issues are tagged with good first issue — pick one and get started.

Want to contribute something bigger? Check the Roadmap above and open an issue to discuss before starting.

Contributors

Thanks to everyone who has contributed to Vibe-Trading!

Recent v0.1.12 cycle contributors and credits:

@santhreal — a 30-PR correctness sweep: strict-JSON / finite-number hardening across metrics, factors, pattern, and options (#764/#765/#766/#767/#739/#740/#744), loader correctness (#761 yahoo 1m bars), and session / journal robustness (#762/#763/#768/#769/#770)
@xkam7ar — broad reliability across packaging, web, scheduler, swarm, and CLI (#584), cancellation before the first AgentLoop iteration (#641, closes #638), QVeris session budget + atomic credit accounting (#685/#686), CI / OOS gates (#630/#632), and journal month-filter / side-parse fixes (#626/#628)
@shadowinlife — the Strategy Development Manager skill (#457, closes #455), pluggable OCR + LLM-vision extraction (#548), centralized provider credentials (#563), the 80× signal-alignment vectorization (#698), and swarm MCP-discovery caching (#704)
@ebujinovch — the correlation regime timeline endpoint + UI (#756, closes #719) and its correlation-regime skill (#557), plus the academic_corr_rewire factor (#705)
@honginp — Binance USD-M routing with execution/mark separation (#470/#716) and the maintenance-bracket decouple that keeps -PERP backtests zero-credential (#757)
@StaniellG — the MetaTrader 5 (Exness) broker connector + mt5 data source (#481)
@tyj147454413-cmd — the Binance fallback loader (#643), bounded OKX history with rate-limit handling (#644), and codex stream-failure classification (#663)
@Marnie0415 — composite sub-engine fallback for unknown symbols (#734) and the frontend insertBefore streaming DOM-race fix (#717)
@YZY0108 — the look-ahead-bias fix across all five portfolio optimizers (#487)
@UNHNQ — the SiliconFlow CN + Global providers (#565)
@FenjuFu — the iFlytek Spark provider (#537)
@jelech — the native Anthropic Messages API adapter (#695)
@octo-patch — MiniMax regional API endpoints (#731)
@Thibaultjaigu — the Requesty OpenAI-compatible gateway provider (#474)
@Robin1987China — realized portfolio turnover metrics for every optimizer (#478)
@YogeshModi24 — the Frazzini-Pedersen betting-against-beta academic factor (#480)
@0xZKnw — opt-in TAP mode for Alpaca (#377)
@sambazhu — the fundamental zoo _VALID_ZOOS whitelist (#707)
@nareshkps — Robinhood connector account_number wiring (#726)
@darkknight4563 — user swarm-presets directory discovery (#570)
@MikeCer — IBKR thread-local connection pool + snapshot quotes (#636)
@Shizoqua — local loader interval resampling (#467)
@roberttidball — FastMCP transport import compatibility (#469)
@yxhuang — bare-ticker resolution in the correlation matrix (#472, closes #471)
@Bortlesboat — stale OPENAI_BASE_URL provider-switch fix (#484, closes #482)
@ananaymital — preflight EnvConfig stale-cache fix (#479, closes #477)
@GabbaTauchi — reported the native zai streaming / base-URL bug (#758)
@warren618 / Haozhe Wu — the correlation regime backend integration, the zai provider streaming + base-URL resolution fix (#758), release integration, and open-PR/issue triage

v0.1.11 cycle contributors

@shadowinlife — the api_server modularization capstone (1,103 → 371 lines, #424 closing #331), centralized env config with the AST CI gate (#440), loader fetch() protocol conformance (#437), and the Strategy Development Manager RFC in review (#455/#457) — 12 merged PRs this cycle
@Robin1987China — Research Autopilot Phase 3 loop closure (#267), 4 canonical academic alphas (#277), Shadow Account PIT-safe entry conditions (#302/#314/#316), the turnover-aware portfolio optimizer (#466), scheduled-research route tests (#452), and test-coverage batches for trade-journal / pattern / loader layers (#268/#269/#276)
@muku314115 — first-class Indian equity (NSE/BSE) support: the IndiaEquityEngine, cost stack, .NS/.BO routing, and the india_broker bridge (#305)
@mvanhorn — the end-to-end scheduled-research executor (#278), the Trading 212 read-only connector (#321), OpenAI default-model resolution (#319), and Robinhood config validation (#320)
@fei-moss — the analyze_image vision tool (#464), NapCat DM pairing (#463), and the IM-media allowed-roots report (#465)
@sambazhu — the value-investing toolkit: financial-rigor + report-audit tools, 4 skills, and the value_investing_committee preset (#407/#408)
@Elfsa-Miranda — the evidence-bound alpha research pipeline exploration (#405/#416, since re-scoped into #442)
@Hinotoi-agent — loopback CSRF rejection (#293) and authenticated remote same-origin UI requests (#304)
@dpersek — configurable IM reply timeout (#413) and the provider-preflight redirect fix (#404)
@digger-yu — cross-platform setup/dev commands (#292) and dev-dependency pre-checks (#349)
@skloxo — tilde expansion + file-roots safety fallback (#299) and reactive zh-CN localization (#301)
@kadaliao — the beginner tutorial (#393) and Alpha Library social cards (#396)
@morluto — CLI resume first-message preservation (#448) and the Codex OAuth default model (#446)
@yxhuang — the Kimi for Coding provider (#435) and the precise #433 diagnosis behind the governance-stack revert
@isaveall — the validation.json artifacts-dir fix (#429) and clearer --swarm-run errors (#428)
@mustafakamal88 — timezone-aware UTC timestamps (#397)
@irfanallana-oss — the zero-size order guard in trading_place_order (#417)
@Shizoqua — the central OHLC-invariant loader guard (#274)
@hobostay — SSRF-guard hardening for CGNAT/mesh ranges + the QQ media redirect fix (#389)
@aeonframework — Pillow / langchain CVE floor bumps (#390)
@hannibal-lee — the pandas version-constraint fix (#329)
@MarkfuGod — dynamic data-source counts + token-gated microcompaction (#296)
@gyx09212214-prog — strict JSON validation outputs (#306)
@LemonCANDY42 — the backtest report library (#224)
@fanfpy — Longbridge Decimal→float serialization (#459)
@asahikiko — packaged SKILL.md capability-count sync + the manifest guard test (#461)
@wison1717-maker — the mandate second-confirmation dialog + unified error toasts (#453)
@imsankz — opencode provider mappings (#444)
@flash1234pku — the tushare reference code-fence fix (#449)
@Penn-Live — the Docker startup route-iteration crash report (#450)
@warren618 / Haozhe Wu — the fundamental factor layer (PIT-safe SEC panels), the QVeris premium track, the IM channel runtime, India-equity integration review, CN search fallbacks, and release integration

v0.1.10 cycle contributors

@Hinotoi-agent — a security-hardening wave: local-shutdown auth (#241), loopback-host rebinding rejection (#242), agent shell-tool opt-in (#243), settings-write auth (#245), mandate proposal-id containment (#256), persistent-memory type validation (#257), and MCP swarm run-id containment (#258)
@mvanhorn — the opt-in local data cache (#177), Gemini thoughtSignature round-trip over OpenAI-compat tool calls (#176), the custom data loader guide (#194), and the glm/zhipu provider alias + model-name inference (#247)
@gyx09212214-prog — loader robustness for malformed crypto/RSSHub timeout env vars (#227, #240), requested yfinance end-date inclusion (#226), strict run-card JSON for non-finite metrics (#238), and ddgs retry-fallback coverage (#239)
@BillDin — swarm agent status in the chat UI (#188), explicit preset-name handling (#189), the loader-backed market-data tool for swarm workers (#199), and preset-context continuations (#200)
@Robin1987China — the Research Autopilot goal-hypothesis bridge (#260), the local CSV/Parquet/DuckDB data loader (#252), and an assistant-prefill fix + configurable Kimi User-Agent (#248)
@LemonCANDY42 — the read-only runtime status dashboard (#210), persisted AgentLoop usage artifacts (#223), and opt-in Run Detail chart payloads (#225)
@zwrong — the trace.jsonl overhaul with zero truncation + offload (#206) and session-id on exit + resume <session-id> (#218)
@forge-builder — the AI contributor guide (#173) and the OpenClaw MCP research-only smoke-test docs (#165)
@skloxo — Chinese (zh-CN) frontend localization (adopted from #217)
@LeeCQiang — Chinese docstrings across all 452 Alpha Zoo factors (#180)
@KaiLuettmann — GHCR pre-built image publishing on release (#187)
@ngoanpv — Gemini thought_signature preservation through the AgentLoop dict path (#184)
@ShahNewazKhan — Docker host-Ollama reachability via host.docker.internal (#196)
@sambazhu — frontend sync of completed chat attempts (#236)
@bhlt — baostock-native code format support (#230)
@octo-patch — MiniMax M3 default model upgrade (#162)
@warren618 / Haozhe Wu — the global data layer (8 sources + 18 read-only data tools), the 10 broker SDK connectors, the alpha-compare full stack, the provider-reliability overhaul, multi-engine web_search fallback, responsive Stop + SSE reconnect, and release integration

Disclaimer

Vibe-Trading is research and trading software. It is not investment advice, holds no funds, and runs no execution venue. Trading through a broker channel you explicitly authorize (e.g. Robinhood Agentic Trading) happens only within the limits you set and which you can halt at any time. This broker-trading capability is experimental and not verified by us against a real broker account — use it at your own risk. Past performance does not guarantee future results.

License

MIT License — see LICENSE

⭐ If Vibe-Trading helps your research, a star helps more people find it.

Thanks for visiting Vibe-Trading ✨

Graphify-Labs/graphify

Turn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.

Read this in other languages

Early access to the graphify platform is open before the public v1 launch: app.graphify.com

Type /graphify in your AI coding assistant and it maps your entire project (code, docs, PDFs, images, videos) into a knowledge graph you can query instead of grepping through files.

Code maps for free, fully local. Code is parsed with tree-sitter AST: deterministic, no LLM, nothing leaves your machine. (Docs, PDFs, images and video use your assistant's model, or a configured API key, for a semantic pass.)
Every edge is explained. Each connection is tagged EXTRACTED (explicit in the source) or INFERRED (resolved by graphify), so you can tell what was read directly from what was inferred.
Not a vector index. No embeddings, no vector store: a real graph you traverse. Ask a question, trace the path between two things, or explain one concept.

Want this always-on, updating in the background across your code, docs, and meetings rather than only on demand? That is what we are building at graphify.com, and early access is open now at app.graphify.com.

The FastAPI codebase mapped by graphify. Every node is a concept, colors are detected communities, and the whole thing is clickable in graph.html.

Get started (30 seconds):

uv tool install graphifyy      # install the CLI (or: pipx install graphifyy)
graphify install               # register the skill with your AI assistant

Then, in your AI assistant:

/graphify .

That's it. You get three files:

graphify-out/
├── graph.html       open in any browser — click nodes, filter, search
├── GRAPH_REPORT.md  the highlights: key concepts, surprising connections, suggested questions
└── graph.json       the full graph — query it anytime without re-reading your files

Works in Claude Code, Cursor, Codex, Gemini CLI, GitHub Copilot, and 15+ more — pick your platform.

See it in action

Once the graph is built you query it instead of reading files. Real output, graphify run on the FastAPI codebase shown above:

$ graphify explain "APIRouter"
Node: APIRouter
  Source:    routing.py L2210
  Community: 2
  Degree:    47

Connections (47):
  --> RequestValidationError [uses] [INFERRED]
  --> Dependant [uses] [INFERRED]
  --> .get() [method] [EXTRACTED]
  <-- __init__.py [imports] [EXTRACTED]
  ...

$ graphify path "FastAPI" "ModelField"
Shortest path (3 hops):
  FastAPI --uses--> DefaultPlaceholder <--references-- get_request_handler() --references--> ModelField

Every edge carries a confidence tag (EXTRACTED = explicit in the source, INFERRED = derived by resolution), so you can tell what was read directly from what was inferred. graphify query "<question>" returns a scoped subgraph for a plain-language question, and graphify path A B traces how any two things connect.

What it does

What you get out of the box:

Capability	What you get
God nodes	The most-connected concepts, so you see what everything flows through
Communities	The graph split into subsystems (Leiden), with LLM-free labels
Cross-file links	`calls` / `imports` / `inherits` / `mixes_in` resolved across ~40 languages via tree-sitter AST
Query, path, explain	Ask a question, trace the path between two things, or explain one concept, all against `graph.json`
Rationale + doc refs	`# NOTE:` / `# WHY:` comments and ADR/RFC citations become first-class nodes linked to the code
Beyond code	Docs, PDFs, images, and video/audio all map into the same graph
Local-first	Code is parsed locally with tree-sitter (no LLM, nothing leaves your machine); only the semantic pass over docs/media calls a backend, and only if you configure one

Benchmarks

Benchmark	Metric	graphify	Field
LOCOMO (n=300)	recall@10	0.497	mem0 0.048, supermemory 0.149
LOCOMO (n=300)	QA accuracy	45.3%	supermemory 49.7%, mem0 27.3%
LongMemEval-S (n=50)	QA accuracy	76%	tied with dense RAG
Graph build	LLM credits	0	per-token for most systems

Every system ran on the same harness with the same model and budgets, scored by a judge blind-validated against a second judge (90.6% agreement, Cohen's kappa 0.81). Full per-system tables, the code-intelligence result, and reproduction commands: BENCHMARKS.md.

Prerequisites

Requirement	Minimum	Check	Install
Python	3.10+	`python --version`	python.org
uv (recommended)	any	`uv --version`	`curl -LsSf https://astral.sh/uv/install.sh \| sh`
pipx (alternative)	any	`pipx --version`	`pip install pipx`

macOS quick install (Homebrew):

brew install python@3.12 uv

Windows quick install:

winget install astral-sh.uv

Ubuntu/Debian:

sudo apt install python3.12 python3-pip pipx
# or install uv:
curl -LsSf https://astral.sh/uv/install.sh | sh

Install

Official package: The PyPI package is graphifyy (double-y). Other graphify* packages on PyPI are not affiliated. The CLI command is still graphify.

Step 1 — install the package:

# Recommended (isolated env; if 'graphify' isn't found after, run: uv tool update-shell):
uv tool install graphifyy

# Alternatives:
pipx install graphifyy
pip install graphifyy  # may need PATH setup — see note below

Step 2 — register the skill with your AI assistant:

graphify install

That's it. Open your AI assistant and type /graphify .

To install the assistant skill into the current repository instead of your user profile, add --project:

graphify install --project
graphify install --project --platform codex

Project-scoped installs write under the current directory, for example .claude/skills/graphify/SKILL.md or .agents/skills/graphify/SKILL.md (plus a references/ sidecar the skill loads on demand), and print a git add hint for files that can be committed. Per-platform commands that support project-scoped installs accept the same flag, for example graphify claude install --project or graphify codex install --project.

PowerShell note: Use graphify . not /graphify . — the leading slash is a path separator in PowerShell.

graphify: command not found? uv tool install / pipx install put the graphify command in their tool bin dir (~/.local/bin). If your shell can't find it right after install — common on a fresh macOS + zsh setup — that dir isn't on your PATH yet: run uv tool update-shell (or pipx ensurepath), then open a new terminal. With plain pip, add ~/.local/bin (Linux) or ~/Library/Python/3.x/bin (Mac) to your PATH, or run python -m graphify.

Running with uvx / uv tool run instead of installing? Name the package, not the command: uvx --from graphifyy graphify install. Plain uvx graphify … fails (No solution found … no versions of graphify) because uv tool run reads the first word as a package, and the package is graphifyy — the graphify command lives inside it.

Avoid pip install on Mac/Windows if possible. The skill resolves Python at runtime from graphify-out/.graphify_python; if that points to a different environment than where pip installed the package, you'll get ModuleNotFoundError: No module named 'graphify'. uv tool install and pipx install isolate the package in their own env and avoid this entirely.

Git hooks and uv tool / pipx: graphify hook install embeds the current interpreter path directly into the hook scripts at install time, so the post-commit hook fires correctly even in GUI git clients and CI runners where ~/.local/bin is not on PATH. If you reinstall or upgrade graphify, re-run graphify hook install to refresh the embedded path.

Strict mode (Claude Code): graphify install --project --strict makes the assistant actually use the graph. The default install nudges it to run graphify query before reading files; strict mode blocks the first raw source read of a session and redirects it to the graph, then reverts to the nudge (so it fires at most once per session and never gets stuck). Toggle at runtime with GRAPHIFY_HOOK_STRICT=1/0; the default install is unchanged (soft nudge).

Pick your platform (20+ assistants, click to expand)

Platform	Install command
Claude Code (Linux/Mac)	`graphify install`
Claude Code (Windows)	`graphify install` (auto-detected) or `graphify install --platform windows`
CodeBuddy	`graphify install --platform codebuddy`
Codex	`graphify install --platform codex`
OpenCode	`graphify install --platform opencode`
Kilo Code	`graphify install --platform kilo`
GitHub Copilot CLI	`graphify install --platform copilot`
VS Code Copilot Chat	`graphify vscode install`
Aider	`graphify install --platform aider`
OpenClaw	`graphify install --platform claw`
Factory Droid	`graphify install --platform droid`
Trae	`graphify install --platform trae`
Trae CN	`graphify install --platform trae-cn`
Gemini CLI	`graphify install --platform gemini`
Hermes	`graphify install --platform hermes`
Kimi Code	`graphify install --platform kimi`
Amp	`graphify amp install`
Agent Skills (cross-framework)	`graphify install --platform agents` (alias `--platform skills`)
Kiro IDE/CLI	`graphify kiro install`
Pi coding agent	`graphify install --platform pi`
Cursor	`graphify cursor install`
Devin CLI	`graphify devin install`
Google Antigravity	`graphify antigravity install`

Codex users also need multi_agent = true under [features] in ~/.codex/config.toml for parallel extraction. CodeBuddy uses the same Agent tool and PreToolUse hook mechanism as Claude Code. Factory Droid uses the Task tool for parallel subagent dispatch. OpenClaw and Aider use sequential extraction (parallel agent support is still early on those platforms). Trae uses the Agent tool for parallel subagent dispatch and does not support PreToolUse hooks, so AGENTS.md is the always-on mechanism.

--platform agents (alias --platform skills) targets the generic cross-framework Agent-Skills locations: the spec's user-global ~/.agents/skills/ (read by npx skills and spec-compliant frameworks) for a global install, and ./.agents/skills/ for a project (--project) install. The bare graphify install stays single-platform (Claude Code) by design — use the named agents platform when you want the skill discoverable by any framework that reads .agents/skills.

Codex uses $graphify instead of /graphify.

Optional extras (install only what you need)

Extra	What it adds	Install
`pdf`	PDF extraction	`uv tool install "graphifyy[pdf]"`
`office`	`.docx` and `.xlsx` support	`uv tool install "graphifyy[office]"`
`google`	Google Sheets rendering	`uv tool install "graphifyy[google]"`
`video`	Video/audio transcription (faster-whisper + yt-dlp)	`uv tool install "graphifyy[video]"`
`mcp`	MCP stdio server	`uv tool install "graphifyy[mcp]"`
`neo4j`	Neo4j push support	`uv tool install "graphifyy[neo4j]"`
`falkordb`	FalkorDB push support	`uv tool install "graphifyy[falkordb]"`
`svg`	SVG graph export	`uv tool install "graphifyy[svg]"`
`leiden`	Leiden community detection (Python < 3.13 only)	`uv tool install "graphifyy[leiden]"`
`ollama`	Ollama local inference	`uv tool install "graphifyy[ollama]"`
`openai`	OpenAI / OpenAI-compatible APIs	`uv tool install "graphifyy[openai]"`
`gemini`	Google Gemini API	`uv tool install "graphifyy[gemini]"`
`anthropic`	Anthropic Claude API (`--backend claude`, uses `ANTHROPIC_API_KEY`)	`uv tool install "graphifyy[anthropic]"`
`bedrock`	AWS Bedrock (uses IAM, no API key)	`uv tool install "graphifyy[bedrock]"`
`azure`	Azure OpenAI Service (`--backend azure`, uses `AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_ENDPOINT`)	`uv tool install "graphifyy[openai]"`
`sql`	SQL schema extraction	`uv tool install "graphifyy[sql]"`
`postgres`	Live PostgreSQL introspection (`--postgres DSN`)	`uv tool install "graphifyy[postgres]"`
`dm`	BYOND DreamMaker `.dm`/`.dme` AST extraction (may need a C compiler + `python3-dev` if no wheel matches your platform)	`uv tool install "graphifyy[dm]"`
`terraform`	Terraform / HCL `.tf`/`.tfvars`/`.hcl` AST extraction	`uv tool install "graphifyy[terraform]"`
`pascal`	Pascal / Delphi `.pas`/`.dpr`/`.dpk`/`.inc` AST extraction (more accurate `calls`/`inherits` edges; falls back to a regex extractor when absent)	`uv tool install "graphifyy[pascal]"`
`chinese`	Chinese query segmentation (jieba)	`uv tool install "graphifyy[chinese]"`
`all`	Everything above	`uv tool install "graphifyy[all]"`

Make your assistant always use the graph

Run this once in your project after building a graph:

Platform	Command
Claude Code	`graphify claude install`
CodeBuddy	`graphify codebuddy install`
Codex	`graphify codex install`
OpenCode	`graphify opencode install`
Kilo Code	`graphify kilo install`
GitHub Copilot CLI	`graphify copilot install`
VS Code Copilot Chat	`graphify vscode install`
Aider	`graphify aider install`
OpenClaw	`graphify claw install`
Factory Droid	`graphify droid install`
Trae	`graphify trae install`
Trae CN	`graphify trae-cn install`
Cursor	`graphify cursor install`
Gemini CLI	`graphify gemini install`
Hermes	`graphify hermes install`
Kimi Code	`graphify install --platform kimi`
Amp	`graphify amp install`
Agent Skills (cross-framework)	`graphify agents install` (alias `graphify skills install`)
Kiro IDE/CLI	`graphify kiro install`
Pi coding agent	`graphify pi install`
Devin CLI	`graphify devin install`
Google Antigravity	`graphify antigravity install`

This writes a small config file that tells your assistant to consult the knowledge graph for codebase questions, preferring scoped queries like graphify query "<question>" over reading the full report or grepping raw files.

Hook platforms (Claude Code, Gemini CLI): a hook fires automatically before search-style tool calls (and, on Claude Code, before reading source files one by one via the Read/Glob tools) and nudges your assistant toward the graph path.
Instruction-file platforms (Codex, OpenCode, Cursor, etc.): persistent instruction files (AGENTS.md, .cursor/rules/, etc.) provide the same query-first guidance.

GRAPH_REPORT.md is still available for broad architecture review.

CodeBuddy does the same two things as Claude Code: writes a CODEBUDDY.md section telling CodeBuddy to read graphify-out/GRAPH_REPORT.md before answering architecture questions, and installs PreToolUse hooks (.codebuddy/settings.json) that fire before Bash search commands and file reads, nudging toward graphify query instead.

Codex writes to AGENTS.md, which is what actually carries the always-on graph guidance on this platform. graphify codex install also registers a PreToolUse hook in .codex/hooks.json (graphify hook-check), but that entry is deliberately a no-op: Codex Desktop rejects hookSpecificOutput.additionalContext on PreToolUse, so emitting a nudge there would break Bash tool calls. Unlike Claude Code, where the hook (graphify hook-guard) does the nudging, on Codex the hook fires and intentionally does nothing, and AGENTS.md is the always-on mechanism.

Kilo Code installs the Graphify skill to ~/.config/kilo/skills/graphify/SKILL.md and a native /graphify command to ~/.config/kilo/command/graphify.md. graphify kilo install also writes AGENTS.md plus a native tool.execute.before plugin (.kilo/plugins/graphify.js + .kilo/kilo.json or .kilo/kilo.jsonc registration) so Kilo gets the same always-on graph reminder behavior through native .kilo config.

Cursor writes .cursor/rules/graphify.mdc with alwaysApply: true, so Cursor includes it in every conversation automatically, no hook needed.

To remove graphify from all platforms at once: graphify uninstall (add --purge to also delete graphify-out/). Or use the per-platform command (e.g. graphify claude uninstall).

What's in the report

God nodes — the most-connected concepts in your project. Everything flows through these.
Surprising connections — links between things that live in different files or modules. Ranked by how unexpected they are.
The "why" — inline comments (# NOTE:, # WHY:, # HACK:), docstrings, and design rationale from docs are extracted as separate nodes linked to the code they explain.
Suggested questions — 4–5 questions the graph is uniquely positioned to answer.
Confidence tags — every inferred relationship is marked EXTRACTED, INFERRED, or AMBIGUOUS. You always know what was found vs guessed.

What files it handles

Type	Extensions
Code (36 tree-sitter grammars)	`.py .ts .mts .cts .js .jsx .tsx .mjs .go .rs .java .c .cpp .cc .cxx .h .hpp .cu .cuh .metal .rb .cs .kt .kts .scala .php .swift .lua .luau .toc .zig .ps1 .psm1 .psd1 .ex .exs .m .mm .jl .vue .svelte .astro .groovy .gradle .dart .v .sv .svh .sql .f .f90 .f95 .f03 .f08 .pas .pp .dpr .dpk .lpr .inc .dfm .lfm .lpk .sh .bash .json .dm .dme .dmi .dmm .dmf .sln .slnx .csproj .fsproj .vbproj .xaml .razor .cshtml` (`.dm`/`.dme` requires `uv tool install graphifyy[dm]`; `.mts`/`.cts` reuse the TypeScript grammar, `.cc`/`.cxx` and CUDA `.cu`/`.cuh` and Metal `.metal` reuse the C++ grammar)
Salesforce Apex	`.cls .trigger` (regex-based; classes, interfaces, enums, methods, triggers, SOQL/DML edges)
Terraform / HCL	`.tf .tfvars .hcl` (requires `uv tool install graphifyy[terraform]`)
MCP configs	`.mcp.json` `mcp.json` `mcp_servers.json` `claude_desktop_config.json` — extracts server nodes, package refs, env var requirements
Package manifests	`apm.yml` `pyproject.toml` `go.mod` `pom.xml` — one canonical package node per package (by name) plus `depends_on` edges, so a package referenced from many manifests is a single hub
Docs	`.md .mdx .qmd .html .txt .rst .yaml .yml` (markdown `[text](./other.md)` links and `[[wikilinks]]` become `references` edges between docs)
Office	`.docx .xlsx` (requires `uv tool install graphifyy[office]`)
Google Workspace	`.gdoc .gsheet .gslides` (opt-in; requires `gws` auth and `--google-workspace`; Sheets need `uv tool install graphifyy[google]`)
PDFs	`.pdf`
Images	`.png .jpg .webp .gif`
Video / Audio	`.mp4 .mov .mp3 .wav` and more (requires `uv tool install graphifyy[video]`)
YouTube / URLs	any video URL (requires `uv tool install graphifyy[video]`)

Code is extracted locally with no API calls (AST via tree-sitter). Everything else goes through your AI assistant's model API.

Google Drive for desktop .gdoc, .gsheet, and .gslides files are shortcut pointers, not document content. To include native Google Docs, Sheets, and Slides in a headless extraction, install and authenticate the gws CLI, then run:

uv tool install "graphifyy[google]"  # needed for Google Sheets table rendering
gws auth login -s drive
graphify extract ./docs --google-workspace

You can also set GRAPHIFY_GOOGLE_WORKSPACE=1. Graphify exports shortcuts into graphify-out/converted/ as Markdown sidecars, then extracts those files.

Common commands

/graphify .                        # build graph for current folder
/graphify ./docs --update          # re-extract only changed files
/graphify . --cluster-only         # rerun clustering without re-extracting
/graphify . --cluster-only --resolution 1.5      # more granular communities
/graphify . --cluster-only --exclude-hubs 99     # suppress utility super-hubs from god-node rankings
/graphify . --no-viz               # skip the HTML, just the report + JSON
/graphify . --wiki                 # build a markdown wiki from the graph
graphify export callflow-html      # Mermaid architecture/call-flow HTML (auto-regenerates on every git commit if hook is installed)

/graphify query "what connects auth to the database?"
/graphify path "UserService" "DatabasePool"
/graphify explain "RateLimiter"

/graphify add https://arxiv.org/abs/1706.03762   # fetch a paper and add it
/graphify add <youtube-url>                       # transcribe and add a video

graphify hook install              # auto-rebuild on git commit
graphify merge-graphs a.json b.json              # combine two graphs

graphify prs                       # PR dashboard: CI state, review status, worktree mapping
graphify prs 42                    # deep dive on PR #42 with graph impact
graphify prs --triage              # AI ranks your review queue (uses whatever backend is configured)
graphify prs --conflicts           # PRs sharing graph communities — merge-order risk

See the full command reference below.

Ignoring files

Create a .graphifyignore in your project root — same syntax as .gitignore, including ! negation.

.gitignore is respected automatically. graphify reads the .gitignore in each directory. If a .graphifyignore is also present, the two are merged — .graphifyignore patterns are evaluated last, so they win on conflicts (including ! negations). Adding a .graphifyignore only ever excludes more; it never re-includes a file your .gitignore already excluded. Subdirectory scoping works the same way as git — an ignore file only affects its own subtree.

Pass --no-gitignore to graphify extract when git-ignored generated or transpiled code belongs in the graph. This disables .gitignore and .git/info/exclude; .graphifyignore still applies.

# .graphifyignore
node_modules/
dist/
*.generated.py

# only index src/, ignore everything else
*
!src/
!src/**

Team setup

graphify-out/ is meant to be committed to git so everyone on the team starts with a map.

Recommended .gitignore additions:

graphify-out/cost.json        # local only
# graphify-out/cache/         # optional: commit for speed, skip to keep repo small

manifest.json is now portable — keys are stored as relative paths and re-anchored on load, so committing it is safe and avoids a full rebuild on first checkout.

Workflow:

One person runs /graphify . and commits graphify-out/.
Everyone pulls — their assistant reads the graph immediately.
Run graphify hook install to auto-rebuild after each commit (AST only, no API cost). This also sets up a git merge driver so graph.json is never left with conflict markers — two devs committing in parallel get their graphs union-merged automatically.
When docs or papers change, run /graphify --update to refresh those nodes.

Using the graph directly

# query the graph from the terminal
graphify query "show the auth flow"
graphify query "what connects DigestAuth to Response?" --graph graphify-out/graph.json

# expose the graph as an MCP server (for repeated tool-call access)
python -m graphify.serve graphify-out/graph.json
python -m graphify.serve --graph graphify-out/graph.json  # --graph flag also accepted

# register with Kimi Code:
kimi mcp add --transport stdio graphify -- python -m graphify.serve graphify-out/graph.json

# or serve over HTTP so a whole team points at one URL (no local graphify needed):
python -m graphify.serve graphify-out/graph.json --transport http --port 8080
python -m graphify.serve graphify-out/graph.json --transport http --host 0.0.0.0 --api-key "$SECRET"

The MCP server gives your assistant structured access: query_graph, get_node, get_neighbors, shortest_path, list_prs, get_pr_impact, triage_prs.

Shared HTTP server

--transport stdio (the default) spawns one local server per developer. --transport http serves the same tools over the MCP Streamable HTTP transport, so a single shared process can serve the graph for the whole team — clients point their IDE MCP config at http://<host>:8080/mcp instead of running graphify locally.

Flag	Default	Purpose
`--transport {stdio,http}`	`stdio`	Transport to serve on
`--host`	`127.0.0.1`	HTTP bind host (use `0.0.0.0` to expose beyond localhost)
`--port`	`8080`	HTTP bind port
`--api-key`	env `GRAPHIFY_API_KEY`	Require `Authorization: Bearer <key>` (or `X-API-Key`)
`--path`	`/mcp`	HTTP mount path
`--json-response`	off	Return plain JSON instead of SSE streams
`--stateless`	off	No per-session state (for load-balanced / CI deployments)
`--session-timeout`	`3600`	Reap idle stateful sessions after N seconds (`0` disables)

The default 127.0.0.1 bind is loopback-only. Set --host 0.0.0.0 and --api-key together when exposing on a shared host. Run it in a container:

docker build -t graphify .
docker run -p 8080:8080 -v "$(pwd)/graphify-out:/data" graphify \
  /data/graph.json --transport http --host 0.0.0.0 --api-key "$SECRET"

WSL / Linux note: Ubuntu ships python3, not python. Use a venv to avoid conflicts:
python3 -m venv .venv && .venv/bin/pip install "graphifyy[mcp]"

Environment variables

These are only needed for headless / CI extraction (graphify extract). When running via the /graphify skill inside your IDE, the model API is provided by your IDE session — no extra keys needed.

Variable	Used for	When required
`ANTHROPIC_API_KEY`	Claude (Anthropic) backend	`--backend claude`
`ANTHROPIC_BASE_URL`	Anthropic-compatible endpoint URL (LiteLLM proxy, gateways, ...)	`--backend claude` (default: `https://api.anthropic.com`)
`ANTHROPIC_MODEL`	Model name for the Claude backend — for custom endpoints, use the model name/alias your server exposes	`--backend claude` (default: `claude-sonnet-4-6`)
`GEMINI_API_KEY` or `GOOGLE_API_KEY`	Google Gemini backend	`--backend gemini`
`OPENAI_API_KEY`	OpenAI or OpenAI-compatible APIs	`--backend openai` (local servers accept any non-empty value)
`OPENAI_BASE_URL`	OpenAI-compatible server URL (llama.cpp, vLLM, LM Studio, ...)	`--backend openai` (default: `https://api.openai.com/v1`)
`OPENAI_MODEL`	Model name for the OpenAI backend — for self-hosted servers, use the model name/alias your server exposes (check its `/v1/models` endpoint), e.g. `LFM2.5-8B-A1B-UD-Q4_K_XL` for llama.cpp	`--backend openai` (default: `gpt-4.1-mini`)
`DEEPSEEK_API_KEY`	DeepSeek backend	`--backend deepseek`
`MOONSHOT_API_KEY`	Kimi Code backend	`--backend kimi`
`OLLAMA_BASE_URL`	Ollama local inference URL	`--backend ollama` (default: `http://localhost:11434`)
`OLLAMA_MODEL`	Ollama model name	`--backend ollama` (default: auto-detect)
`GRAPHIFY_OLLAMA_NUM_CTX`	Override Ollama KV-cache window size	optional — auto-sized by default
`GRAPHIFY_OLLAMA_KEEP_ALIVE`	Minutes to keep Ollama model loaded	optional — set `0` to unload after each chunk
`AZURE_OPENAI_API_KEY`	Azure OpenAI Service backend	`--backend azure`
`AZURE_OPENAI_ENDPOINT`	Azure resource endpoint URL	`--backend azure` (required alongside API key)
`AZURE_OPENAI_API_VERSION`	Azure API version override	optional — default `2024-12-01-preview`
`AZURE_OPENAI_DEPLOYMENT` or `GRAPHIFY_AZURE_MODEL`	Azure deployment name	optional — default `gpt-4o`
`AWS_*` / `~/.aws/credentials`	AWS Bedrock — standard credential chain	`--backend bedrock` (no API key, uses IAM)
`GRAPHIFY_MAX_WORKERS`	AST parallelism thread count	optional — also `--max-workers` flag
`GRAPHIFY_MAX_OUTPUT_TOKENS`	Raise output cap for dense corpora	optional — e.g. `32768` for large files
`GRAPHIFY_API_TIMEOUT`	Per-call timeout in seconds for HTTP, claude-cli, Anthropic SDK, and Bedrock backends (default: 600)	optional — also `--api-timeout` flag
`GRAPHIFY_MAX_RETRIES`	How many times to retry a rate-limited (429) request before giving up (default: 6; honors `Retry-After`)	optional — raise for strict per-org limits (e.g. kimi); `0` disables
`GRAPHIFY_FORCE`	Force graph rebuild even with fewer nodes	optional — also `--force` flag
`GRAPHIFY_GOOGLE_WORKSPACE`	Auto-enable Google Workspace export	optional — set to `1`
`GRAPHIFY_TRIAGE_BACKEND`	Backend for `graphify prs --triage`	optional — auto-detected from available keys
`GRAPHIFY_TRIAGE_MODEL`	Model override for triage	optional — e.g. `claude-opus-4-7`
`GRAPHIFY_QUERY_LOG_ENABLE`	Set to `1` to turn on the local query log at `~/.cache/graphify-queries.log` (records each query/path/explain question + corpus path). Off by default — nothing is written unless you opt in (#1797)	optional
`GRAPHIFY_QUERY_LOG`	Enable the query log and write it to this path instead of the default	optional — off unless this or `_ENABLE` is set
`GRAPHIFY_QUERY_LOG_DISABLE`	Set to `1` to force the query log off (wins over the enable vars)	optional
`GRAPHIFY_QUERY_LOG_RESPONSES`	When the log is enabled, also record full subgraph responses (off by default)	optional
`GRAPHIFY_MAX_GRAPH_BYTES`	Override the 512 MiB graph.json size cap — e.g. `700MB`, `2GB`, or plain bytes	optional — useful for very large corpora
`GRAPHIFY_MAX_CONTEXTS`	Maximum number of non-default project graphs retained by one multi-project MCP server	optional — default: `8`; invalid values use `8`, and values below `1` use `1`
`GRAPHIFY_LLM_TEMPERATURE`	Override LLM temperature for semantic extraction — e.g. `0.7`, or `none` to omit	optional — auto-omitted for o1/o3/o4/gpt-5 reasoning models

Privacy

Code files — processed locally via tree-sitter. Nothing leaves your machine. A code-only corpus requires no API key — graphify extract runs fully offline. On a mixed repo, add --code-only to index just the code and skip the docs/PDFs/images that would otherwise need an LLM.
Video / audio — transcribed locally with faster-whisper. Nothing leaves your machine.
Docs, PDFs, images — sent to your AI assistant for semantic extraction (via the /graphify skill, using whatever model your IDE session runs). Headless graphify extract requires GEMINI_API_KEY / GOOGLE_API_KEY (Gemini), MOONSHOT_API_KEY (Kimi), ANTHROPIC_API_KEY (Claude), OPENAI_API_KEY (OpenAI), DEEPSEEK_API_KEY (DeepSeek), a running Ollama instance (OLLAMA_BASE_URL), AWS credentials via the standard provider chain (Bedrock - no API key needed, uses IAM), or the claude CLI binary (Claude Code - no API key needed, uses your Claude subscription). The --dedup-llm flag uses the same key.
Data residency — graphify extract auto-detects which provider to use based on which API key is set (priority: Gemini → Kimi → Claude → OpenAI → DeepSeek → Azure → Bedrock → Ollama). For code with data-residency requirements, use --backend ollama (fully local) or pass an explicit --backend flag. Kimi (MOONSHOT_API_KEY) routes to Moonshot AI servers in China.
No telemetry, no usage tracking, no analytics.
Query logging — every graphify query, graphify path, graphify explain, and MCP query_graph call is logged to ~/.cache/graphify-queries.log in JSON Lines format (timestamp, question, corpus, nodes returned, duration). Full subgraph responses are not stored by default. Set GRAPHIFY_QUERY_LOG_DISABLE=1 to opt out, or GRAPHIFY_QUERY_LOG=/dev/null to silence without disabling the code path.

Troubleshooting

graphify: command not found after installing The CLI is installed but its bin directory isn't on your shell's PATH. Pick the fix for how you installed:

uv (uv tool install graphifyy): the command lands in uv's tool bin dir (~/.local/bin), which a fresh macOS/zsh setup often doesn't have on PATH. Run uv tool update-shell, then open a new terminal. (Find the dir with uv tool dir --bin.)
pipx (pipx install graphifyy): run pipx ensurepath, then open a new terminal.
pip (pip install graphifyy): pip installs scripts to a user bin dir that may not be on PATH — add ~/Library/Python/3.x/bin (macOS) or ~/.local/bin (Linux) to your PATH in ~/.zshrc/~/.bashrc, or just run python -m graphify.

uvx graphify … or uv tool run graphify … fails to resolve graphify The PyPI package is graphifyy; graphify is only the command it provides. uv tool run treats the first word as a package name, so it looks for a package called graphify and reports No solution found … no versions of graphify. Name the package explicitly: uvx --from graphifyy graphify install (same as uv tool run --from graphifyy graphify install). Or uv tool install graphifyy once and then call graphify directly.

uv run --with graphifyy python -m graphify silently runs an older install uv run uses your system Python, so if an older graphifyy also lives there (e.g. a past pip install graphifyy), Python can find that copy first on sys.path and --with graphifyy won't override it. It runs with no error, but you get the old version's behavior — e.g. env overrides like OPENAI_BASE_URL are silently ignored, so requests hit the default endpoint and fail with a 401 that looks like a bad key. The fingerprint is a warning: skill is from graphify <newer>, package is <older> line — that means a different install was loaded, not just a stale skill. Check which copy actually loaded:

python -c "import graphify; print(graphify.__file__)"

Then run the installed command directly (it uses the uv-managed copy), or drop the stale system copy:

uvx --from graphifyy graphify extract . --backend openai   # names the package explicitly
pip uninstall graphifyy                                    # or remove the old system install

python -m graphify works but graphify command doesn't Your shell's PATH doesn't include the bin directory the command was installed to. Prefer uv tool install / pipx install over plain pip, then run uv tool update-shell / pipx ensurepath and open a new terminal (see the install notes above).

/graphify . causes "path not recognized" in PowerShell PowerShell treats a leading / as a path separator. Use graphify . (no slash) on Windows.

Graph has fewer nodes after --update or rebuild If a refactor deleted files, the old nodes linger. Pass --force (or set GRAPHIFY_FORCE=1) to overwrite even when the rebuild has fewer nodes.

extract exits with "extraction was incomplete ... refusing to overwrite" When an extraction pass crashes or a walk can't fully read the corpus, the run would be smaller than a complete one, so graphify extract refuses to overwrite a larger existing graph with the partial result (protecting your graph.json). Fix the underlying failure and re-run, or pass --allow-partial to overwrite anyway.

Graph has duplicate nodes for the same entity (ghost duplicates) Ghost duplicates (same symbol appearing twice — once from AST extraction with a source location, once from semantic extraction without) are now automatically merged at build time. If you see this in a graph built before v0.8.33, run a full re-extract to clean up:

graphify extract . --force

Ollama runs out of VRAM / context window exceeded The KV-cache window is auto-sized but may be too large for your GPU. Reduce it:

GRAPHIFY_OLLAMA_NUM_CTX=8192 graphify extract ./docs --backend ollama --token-budget 4000

LLM returned invalid JSON / Unterminated string warnings The model's JSON response hit its output-token limit and was cut off mid-string. graphify auto-recovers (it splits the chunk and re-extracts the halves, and an oversized single document is first sliced at heading/paragraph boundaries so the whole file is still covered), so these warnings are noisy but not data loss. To reduce the churn, raise the output cap or shrink each chunk's output:

GRAPHIFY_MAX_OUTPUT_TOKENS=16384 graphify extract . --mode deep   # lift the cap
graphify extract . --mode deep --token-budget 4000                # smaller input chunks -> smaller output

With a cloud gateway like OpenRouter, prefer --backend openai (set OPENAI_BASE_URL) over the Ollama shim — it's a cleaner OpenAI-compatible path. If the model has its own max-output ceiling, lowering --token-budget is the reliable lever.

Graph HTML is too large to open in a browser (>5000 nodes) Skip HTML generation and use the JSON directly:

graphify cluster-only ./my-project --no-viz
graphify query "..."

graph.json has conflict markers after two devs commit at once Run graphify hook install — it sets up a git merge driver that union-merges graph.json automatically so conflicts never happen.

Extraction returns empty nodes/edges for docs or PDFs Docs, PDFs, and images require an LLM call — code-only corpora need no key. Check that your API key is set and the backend is correct:

ANTHROPIC_API_KEY=sk-... graphify extract ./docs --backend claude

Skill version mismatch warning in your IDE Your installed graphify version is different from the skill file. Update:

uv tool upgrade graphifyy
graphify install  # overwrites the skill file

Claude Code prompt cache invalidated after every graphify extract Graphify writes output files (graph.json, graphify-out/) into the workspace. If those paths aren't ignored, every write invalidates Claude Code's prompt cache, forcing a full re-upload at cache-write rates on the next turn. Add them to .claudeignore:

# .claudeignore
graph.json
graphify-out/

Full command reference

/graphify                          # run on current directory
/graphify ./raw                    # run on a specific folder
/graphify ./raw --mode deep        # more aggressive relationship extraction
graphify extract ./raw --code-only # index code only — local AST, no API key (skips docs/PDFs/images); an `extract` flag, not a skill flag
/graphify ./raw --update           # re-extract only changed files
/graphify ./raw --directed         # preserve edge direction
/graphify ./raw --cluster-only     # rerun clustering on existing graph
/graphify ./raw --no-viz           # skip HTML visualization
/graphify ./raw --obsidian         # generate Obsidian vault
/graphify ./raw --obsidian --obsidian-dir ~/vault  # write into an existing vault (never overwrites your own notes or .obsidian config)
/graphify ./raw --wiki             # build agent-crawlable markdown wiki
/graphify ./raw --svg              # export graph.svg
/graphify ./raw --graphml          # export for Gephi / yEd
/graphify ./raw --neo4j            # generate cypher.txt for Neo4j
/graphify ./raw --neo4j-push bolt://localhost:7687
/graphify ./raw --falkordb         # generate cypher.txt for FalkorDB
/graphify ./raw --falkordb-push falkordb://localhost:6379
/graphify ./raw --watch            # auto-sync as files change
/graphify ./raw --mcp              # start MCP stdio server

/graphify add https://arxiv.org/abs/1706.03762
/graphify add <video-url>
/graphify add https://... --author "Name" --contributor "Name"

/graphify query "what connects attention to the optimizer?"
/graphify query "..." --dfs --budget 1500
/graphify path "DigestAuth" "Response"
/graphify explain "SwinTransformer"

graphify save-result --question "Q" --answer "A" --nodes Foo Bar --outcome useful   # record how a Q&A turned out (work memory; outcome ∈ useful|dead_end|corrected)
graphify reflect                   # aggregate graphify-out/memory/ outcomes into reflections/LESSONS.md
graphify reflect --if-stale        # no-op when LESSONS.md is already newer than every input (cheap to run each session)
graphify reflect --out docs/LESSONS.md    # write the lessons doc somewhere else
graphify reflect --graph graphify-out/graph.json  # group lessons by community + write the work-memory overlay (.graphify_learning.json)
                                   # the overlay tags nodes preferred/tentative/contested (recency-weighted, with provenance);
                                   # graphify explain / query then show a "Lesson:" hint, flagged "code changed — re-verify" when the source moved on

graphify uninstall                 # remove from all platforms in one shot
graphify uninstall --purge         # also delete graphify-out/
graphify uninstall --project --platform codex  # remove project-scoped install files only

graphify hook install              # post-commit + post-checkout hooks
graphify hook uninstall
graphify hook status

# always-on assistant instructions - platform-specific
graphify claude install            # CLAUDE.md + PreToolUse hook (Claude Code)
graphify claude uninstall
graphify codebuddy install         # CODEBUDDY.md + PreToolUse hook (CodeBuddy)
graphify codebuddy uninstall
graphify codex install             # AGENTS.md + PreToolUse hook in .codex/hooks.json (Codex)
graphify opencode install          # AGENTS.md + tool.execute.before plugin (OpenCode)
graphify kilo install              # native Kilo skill + /graphify command + AGENTS.md + .kilo plugin
graphify kilo uninstall
graphify cursor install            # .cursor/rules/graphify.mdc (Cursor)
graphify cursor uninstall
graphify gemini install            # GEMINI.md + BeforeTool hook (Gemini CLI)
graphify gemini uninstall
graphify copilot install           # skill file (GitHub Copilot CLI)
graphify copilot uninstall
graphify aider install             # AGENTS.md (Aider)
graphify aider uninstall
graphify claw install              # AGENTS.md (OpenClaw)
graphify claw uninstall
graphify droid install             # AGENTS.md (Factory Droid)
graphify droid uninstall
graphify trae install              # AGENTS.md (Trae)
graphify trae uninstall
graphify trae-cn install           # AGENTS.md (Trae CN)
graphify trae-cn uninstall
graphify hermes install             # AGENTS.md + ~/.hermes/skills/ (Hermes)
graphify hermes uninstall
graphify amp install               # skill file (Amp)
graphify amp uninstall
graphify agents install            # ~/.agents/skills/ + AGENTS.md (cross-framework; alias: graphify skills)
graphify agents uninstall
graphify kiro install               # .kiro/skills/ + .kiro/steering/graphify.md (Kiro IDE/CLI)
graphify kiro uninstall
graphify pi install                # skill file (Pi coding agent)
graphify pi uninstall
graphify devin install             # skill file + .windsurf/rules/graphify.md (Devin CLI)
graphify devin uninstall
graphify antigravity install       # .agents/rules + .agents/workflows (Google Antigravity)
graphify antigravity uninstall

graphify extract ./docs                        # headless LLM extraction for CI (no IDE needed)
graphify extract ./docs --backend gemini       # explicit backend: gemini, kimi, claude, openai, deepseek, ollama, bedrock, or claude-cli
graphify extract ./docs --backend gemini --model gemini-3.1-pro-preview
graphify extract ./docs --backend ollama       # local Ollama (set OLLAMA_BASE_URL / OLLAMA_MODEL) - no API key needed for loopback
OPENAI_BASE_URL=http://localhost:8080/v1 OPENAI_MODEL=my-model graphify extract ./docs --backend openai   # any OpenAI-compatible server (llama.cpp, vLLM, LM Studio)
ANTHROPIC_BASE_URL=http://localhost:4000 ANTHROPIC_MODEL=my-model graphify extract ./docs --backend claude   # any Anthropic-compatible endpoint (LiteLLM proxy, gateways)
GRAPHIFY_OLLAMA_NUM_CTX=32768 graphify extract ./docs --backend ollama   # override KV-cache window (auto-sized by default)
GRAPHIFY_OLLAMA_KEEP_ALIVE=0 graphify extract ./docs --backend ollama    # unload model after each chunk (saves VRAM on small GPUs)
graphify extract ./docs --backend bedrock      # AWS Bedrock via IAM - no API key, uses AWS credential chain
graphify extract ./docs --backend claude-cli   # route through Claude Code CLI - no API key, uses your Claude subscription
graphify extract ./docs --backend azure        # Azure OpenAI (set AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT)
graphify extract ./docs --max-workers 16       # AST parallelism (also GRAPHIFY_MAX_WORKERS)
graphify extract --postgres "postgresql://user:pass@host/db"   # introspect live PostgreSQL schema directly
graphify extract ./my-workspace --cargo        # introspect Rust Cargo workspace dependencies directly
graphify extract ./docs --token-budget 30000   # smaller semantic chunks for local/small models
graphify extract ./docs --max-concurrency 2    # fewer parallel LLM calls (useful for local inference)
graphify extract ./docs --api-timeout 900      # longer HTTP timeout for slow local models (default 600s)
graphify extract ./docs --google-workspace     # export .gdoc/.gsheet/.gslides via gws before extraction
graphify extract ./src --no-gitignore          # include git-ignored source; still honor .graphifyignore
graphify extract ./docs --mode deep            # richer semantic extraction via extended system prompt
graphify extract ./docs --no-cluster           # raw extraction only, skip clustering
graphify extract ./docs --timing               # print per-stage wall-clock timings to stderr (also works on cluster-only)
graphify extract ./docs --force                # overwrite graph.json even if new graph has fewer nodes (use after refactors or to clear ghost duplicates)
graphify extract ./docs --dedup-llm            # LLM tiebreaker for ambiguous entity pairs (uses same API key)
graphify extract ./docs --global --as myrepo   # extract and register into the cross-project global graph
GRAPHIFY_MAX_OUTPUT_TOKENS=32768 graphify extract ./docs --backend claude  # raise output cap for dense corpora

graphify export callflow-html                       # graphify-out/<project>-callflow.html
graphify export callflow-html --max-sections 8      # cap generated architecture sections
graphify export callflow-html --output docs/arch.html
graphify export callflow-html ./some-repo/graphify-out

graphify global add graphify-out/graph.json --as myrepo   # register a project graph into ~/.graphify/global-graph.json
graphify global remove myrepo                         # remove a project from the global graph
graphify global list                                  # show all registered repos + node/edge counts
graphify global path                                  # print path to the global graph file

graphify prs                              # PR dashboard: CI, review, worktree, graph impact
graphify prs 42                           # deep dive on PR #42
graphify prs --triage                     # AI triage ranking (auto-detects backend from env)
graphify prs --worktrees                  # worktree → branch → PR mapping
graphify prs --conflicts                  # PRs sharing graph communities (merge-order risk)
graphify prs --base main                  # filter to PRs targeting a specific base branch
graphify prs --repo owner/repo            # run against a different GitHub repo
GRAPHIFY_TRIAGE_BACKEND=kimi graphify prs --triage   # use a specific backend for triage

graphify clone https://github.com/karpathy/nanoGPT
graphify merge-graphs a.json b.json --out merged.json
graphify --version                                    # print installed version
graphify watch ./src
graphify check-update ./src
graphify update ./src
graphify update ./src --no-cluster  # skip reclustering, write raw AST graph only
graphify update ./src --force       # overwrite even if new graph has fewer nodes
graphify cluster-only ./my-project
graphify cluster-only ./my-project --graph path/to/graph.json  # custom graph location
graphify cluster-only ./my-project --max-concurrency 16 --batch-size 200  # parallel community labeling (large graphs)
graphify cluster-only ./my-project --resolution 1.5            # more, smaller communities
graphify cluster-only ./my-project --exclude-hubs 99           # exclude p99 degree nodes from partitioning
graphify cluster-only ./my-project --no-label                  # keep "Community N" placeholders
graphify cluster-only ./my-project --backend=gemini            # backend for community naming
graphify cluster-only ./my-project --backend=gemini --model gemini-2.5-pro  # specific model
graphify label ./my-project                                    # (re)name communities with the configured backend
graphify label ./my-project --backend=openai --model gpt-4o   # force a specific backend and model

Community names: inside an agent (Claude Code, Gemini CLI) the agent names communities itself. When you run the bare CLI, cluster-only auto-names them with the configured backend (built-in or custom OpenAI-compatible provider) — pass --no-label to keep Community N, or run graphify label to (re)generate names on demand.

Learn more

How it works — the extraction pipeline, community detection, confidence scoring, benchmarks
ARCHITECTURE.md — module breakdown, how to add a language
Optional integrations — Docker MCP Toolkit + SQLite
The Memory Layer — the book on the ideas behind graphify, the architecture end to end

graphify Enterprise

graphify Enterprise is the always-on layer built on top of graphify — it applies the same graph approach to your entire working context: meetings, files, docs, and code, updating continuously in the background.

Built for people and teams whose work lives across hundreds of conversations and documents they can never fully reconstruct.

Join the waitlist at graphify.com. Free trial launching soon.

Contributing

Development setup

The project uses uv for dev workflow. Install it once, then:

git clone https://github.com/safishamsi/graphify.git
cd graphify
git checkout v8                        # active development branch

# Create the project venv and install graphify + all extras + the dev group
# (pytest). uv installs the dev dependency group by default; pass --no-dev to
# skip it.
uv sync --all-extras

Verify the editable install:

uv run graphify --version
uv run python -c "import graphify; print(graphify.__file__)"

Running tests

uv run pytest tests/ -q                # run the full suite
uv run pytest tests/test_extract.py -q # one module
uv run pytest tests/ -q -k "python"    # filter by name

macOS note: the test suite includes both sample.f90 and sample.F90 fixtures. These collide on case-insensitive HFS+ / APFS file systems. Run on Linux or in a Docker container if you need to test both Fortran variants simultaneously.

Git workflow

Active development happens on the v8 branch.
Commit style: fix: <description> / feat: <description> / docs: <description>
Before opening a PR, run uv run pytest tests/ -q and confirm it passes.
Add a fixture file to tests/fixtures/ and tests to tests/test_languages.py for any new language extractor.

What to contribute

Worked examples are the most useful contribution. Run /graphify on a real corpus, save the output to worked/{slug}/, write an honest review.md covering what the graph got right and wrong, and open a PR.

Extraction bugs — open an issue with the input file, the cache entry (graphify-out/cache/), and what was missed or wrong.

See ARCHITECTURE.md for module responsibilities and how to add a language.

Community and links

usestrix/strix

Open-source AI penetration testing tool to find and fix your app’s vulnerabilities.

Strix

The open-source AI pentesting tool. Autonomous AI hackers that find and fix your app’s vulnerabilities.

Tip

New! Strix integrates seamlessly with GitHub Actions and CI/CD pipelines. Automatically scan for vulnerabilities on every pull request and block insecure code before it reaches production - Get started with no setup required.

Strix Overview

Strix are autonomous AI penetration testing agents that act just like real hackers - they run your code dynamically, find vulnerabilities, and validate them through actual proofs-of-concept. Built for developers and security teams who need fast, accurate security testing without the overhead of manual pentesting or the false positives of static analysis tools.

Key Capabilities:

Full pentesting toolkit - reconnaissance, exploitation, and validation out of the box
Multi-agent orchestration - teams of AI pentesters that collaborate and scale
Real exploit validation - working PoCs, not false positives like legacy vulnerability scanners
Developer‑first CLI - actionable findings with remediation guidance
Auto‑fix & reporting - generate patches and compliance-ready pentest reports

Use Cases

Application Security Testing - Detect and validate critical vulnerabilities in your applications
Rapid Penetration Testing - Get penetration tests done in hours, not weeks, with compliance reports
Bug Bounty Automation - Automate bug bounty research and generate PoCs for faster reporting
CI/CD Integration - Run tests in CI/CD to block vulnerabilities before reaching production

🚀 Quick Start

Prerequisites:

Docker (running)
An LLM API key from any supported provider (OpenAI, Anthropic, Google, etc.)

Installation & First Scan

# Install Strix
curl -sSL https://strix.ai/install | bash

# Configure your AI provider
export STRIX_LLM="openai/gpt-5.4"
export LLM_API_KEY="your-api-key"

# Run your first security assessment
strix --target ./app-directory

Note

First run automatically pulls the sandbox Docker image. Results are saved to strix_runs/<run-name>

☁️ Strix Platform

Try the Strix full-stack penetration testing platform at app.strix.ai - sign up for free, connect your repos and domains, and launch a pentest in minutes.

Validated findings with PoCs - every vulnerability includes a working proof-of-concept exploit and reproduction steps
One-click autofix - AI-generated security patches as ready-to-merge pull requests
Continuous pentesting - always-on vulnerability scanning that keeps pace with your deployments
DevSecOps integrations - GitHub, GitLab, Bitbucket, Slack, Jira, Linear, and CI/CD pipelines
Continuous learning - AI that builds on past findings, adapts to your codebase, and reduces false positives over time

Start your first pentest →

🤖 Use Strix from Your Coding Agent

Strix is agent-ready. Give Claude Code, Cursor, Codex, or any SKILL.md-compatible agent the ability to run pentests, fix findings, and set up CI scanning:

npx skills add usestrix/strix

This installs four skills: penetration-testing-with-strix (run headless scans and read results), managed-pentesting-with-strix (drive the managed app.strix.ai platform via REST — no local Docker or LLM key), fix-security-vulnerabilities-with-strix (remediate + re-scan to verify), and ci-security-scanning-with-strix (PR scanning in CI). Agents can run Strix two ways with the same engine — the open-source CLI locally, or the managed cloud when there's no local infra — and read AGENTS.md for a quick reference, docs.strix.ai/llms.txt for the CLI docs, and docs.app.strix.ai for the API.

✨ Features

Agentic Pentesting Tools

Strix agents come equipped with a comprehensive offensive security toolkit - the same tools used by professional penetration testers and ethical hackers:

HTTP Interception Proxy - Full request/response manipulation and analysis with Caido
Browser Exploitation - Automated browser for testing XSS, CSRF, clickjacking, and auth bypass flows
Shell & Command Execution - Interactive terminal for exploit development and post-exploitation
Custom Exploit Runtime - Python sandbox for writing and validating proof-of-concept exploits
Reconnaissance & OSINT - Automated attack surface mapping, subdomain enumeration, and fingerprinting
Static & Dynamic Code Analysis - SAST + DAST capabilities for comprehensive application security testing
Vulnerability Knowledge Base - Structured findings with CVSS scoring and OWASP classification

Comprehensive Vulnerability Scanner

Strix identifies, validates, and exploits a wide range of security vulnerabilities across the OWASP Top 10 and beyond:

Broken Access Control - IDOR, privilege escalation, auth bypass
Injection Attacks - SQL injection, NoSQL injection, OS command injection, SSTI
Server-Side Vulnerabilities - SSRF, XXE, insecure deserialization, RCE
Client-Side Attacks - XSS (stored/reflected/DOM), prototype pollution, CSRF
Business Logic Flaws - Race conditions, payment manipulation, workflow bypass
Authentication & Session - JWT attacks, session fixation, credential stuffing vectors
Infrastructure & Cloud - Misconfigurations, exposed services, cloud security issues
API Security - Broken authentication, mass assignment, rate limiting bypass

Graph of Agents (Multi-Agent Pentesting)

Advanced multi-agent orchestration for comprehensive automated penetration testing:

Distributed Pentesting - Specialized AI agents for recon, exploitation, and post-exploitation
Scalable Security Testing - Parallel execution across multiple targets for fast, comprehensive coverage
Dynamic Coordination - Agents share discoveries, chain vulnerabilities, and collaborate like a red team

🖥️ Local Web Viewer

Every scan writes its results to disk as it runs. Bring them up in a local dashboard with a single command:

# Open the most recent run
strix view

# ...or open a specific run by name
strix view my-run-name

strix view starts a lightweight local server (bound to 127.0.0.1 on a random port) and opens your browser to a private, tokened link. Nothing leaves your machine: the dashboard reads the run's files straight off disk, with no cloud account or upload required. The UI ships prebuilt with Strix, so there is no extra install and no JS build step.

What's in the dashboard

Overview: run status, target, and a severity breakdown of everything found so far.
Vulnerabilities: each validated finding with its severity, details, and reproduction steps.
Agent graph: a live map of the multi-agent team, showing which agent is doing what.
Steering: send instructions to a live scan from the browser to redirect the agents mid-run.
History: browse past runs on this machine and jump between them.
Reports: generate a shareable report and email it to yourself or your team.

Usage Examples

Basic Usage

# Scan a local codebase
strix --target ./app-directory

# Security review of a GitHub repository
strix --target https://github.com/org/repo

# Black-box web application assessment
strix --target https://your-app.com

API Testing (OpenAPI / Swagger / Postman)

Point Strix at an API contract and it tests every declared endpoint instead of having to discover them by crawling. Pair the spec with the live base URL so the agent knows where to send traffic:

# OpenAPI / Swagger file (.json / .yaml)
strix --target ./openapi.yaml --target https://api.your-app.com

# Postman collection export
strix --target ./collection.postman_collection.json --target https://api.your-app.com

# Postman collection pulled live by id (no manual export)
export POSTMAN_API_KEY="PMAK-..."
strix --target postman://<collection-uuid>

# ...with a Postman environment to resolve {{baseUrl}} / token variables
strix --target "postman://<collection-uuid>?env=<environment-uuid>"

Advanced Testing Scenarios

# Grey-box authenticated testing
strix --target https://your-app.com --instruction "Perform authenticated testing using credentials: user:pass"

# Multi-target testing (source code + deployed app)
strix -t https://github.com/org/app -t https://your-app.com

# Targets from a file, one target per non-empty, non-comment line
strix --target-list ./targets.txt

# White-box source-aware scan (local repository)
strix --target ./app-directory --scan-mode standard

# Focused testing with custom instructions
strix --target api.your-app.com --instruction "Focus on business logic flaws and IDOR vulnerabilities"

# Provide detailed instructions through file (e.g., rules of engagement, scope, exclusions)
strix --target api.your-app.com --instruction-file ./instruction.md

# Force PR diff-scope against a specific base branch
strix -n --target ./ --scan-mode quick --scope-mode diff --diff-base origin/main

Headless Mode

Run Strix programmatically without interactive UI using the -n/--non-interactive flag - perfect for servers and automated jobs. The CLI prints real-time vulnerability findings and the final report before exiting. Exits with non-zero code when vulnerabilities are found.

strix -n --target https://your-app.com

CI/CD (GitHub Actions)

Strix can be added to your pipeline to run a security test on pull requests with a lightweight GitHub Actions workflow:

name: strix-penetration-test

on:
  pull_request:

jobs:
  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
        with:
          fetch-depth: 0

      - name: Install Strix
        run: curl -sSL https://strix.ai/install | bash

      - name: Run Strix
        env:
          STRIX_LLM: ${{ secrets.STRIX_LLM }}
          LLM_API_KEY: ${{ secrets.LLM_API_KEY }}

        run: strix -n -t ./ --scan-mode quick

Tip

In CI pull request runs, Strix automatically scopes quick reviews to changed files. If diff-scope cannot resolve, ensure checkout uses full history (fetch-depth: 0) or pass --diff-base explicitly.

Configuration

export STRIX_LLM="openai/gpt-5.4"
export LLM_API_KEY="your-api-key"

# Optional
export LLM_API_BASE="your-api-base-url"  # if using a local model, e.g. Ollama, LMStudio
export PERPLEXITY_API_KEY="your-api-key"  # for search capabilities
export STRIX_REASONING_EFFORT="high"  # control thinking effort (default: high, quick scan: medium)

Note

Strix automatically saves your configuration to ~/.strix/cli-config.json, so you don't have to re-enter it on every run.

Sign in with a ChatGPT subscription

Instead of a metered API key, you can run Strix on your ChatGPT Plus/Pro subscription:

strix auth login chatgpt      # sign in with your ChatGPT account

export STRIX_LLM="chatgpt/gpt-5.4"   # chatgpt/<model> runs on the subscription
strix --target ./app-directory

strix auth status             # show the active sign-in
strix auth logout             # forget the sign-in

Recommended models for best results:

OpenAI GPT-5.4 - openai/gpt-5.4
Anthropic Claude Sonnet 4.6 - anthropic/claude-sonnet-4-6
Google Gemini 3 Pro Preview - vertex_ai/gemini-3-pro-preview

See the LLM Providers documentation for all supported providers including Vertex AI, Bedrock, Azure, and local models.

Enterprise Pentesting

Get the same Strix experience with enterprise-grade controls: SSO (SAML/OIDC), custom compliance-ready penetration testing reports (SOC 2, ISO 27001, PCI DSS), dedicated support & SLA, custom deployment options (VPC/self-hosted), BYOK model support, and tailored AI pentesting agents optimized for your environment. Learn more.

Documentation

Full documentation is available at docs.strix.ai - including detailed guides for usage, CI/CD integrations, skills, and advanced configuration.

Contributing

We welcome contributions of code, docs, and new skills - check out our Contributing Guide to get started or open a pull request/issue.

Join Our Community

Have questions? Found a bug? Want to contribute? Join our Discord!

Support the Project

Love Strix? Give us a ⭐ on GitHub!

Acknowledgements

Strix builds on the incredible work of open-source projects like LiteLLM, Caido, Nuclei, Playwright, and Bubble Tea. Huge thanks to their maintainers!

Warning

Authorized use only. Strix actively tests the targets you point it at, so only run it against systems you own or have explicit, written permission to test, and stay within the agreed scope. Unauthorized testing is illegal in most jurisdictions. You alone are responsible for obtaining authorization and complying with the law. Strix is provided "as is" with no warranty or liability for misuse.

MoonshotAI/kimi-cli

Kimi Code CLI is your next CLI agent.

Kimi CLI

Kimi Code | Documentation | 文档

Important

Kimi CLI is evolving into Kimi Code CLI — the next-generation terminal AI agent from the same team. Installing Kimi Code CLI automatically migrates your configuration and sessions. This project will be gradually wound down; the docs and existing installations remain available.

Kimi CLI is an AI agent that runs in the terminal, helping you complete software development tasks and terminal operations. It can read and edit code, execute shell commands, search and fetch web pages, and autonomously plan and adjust actions during execution.

Getting Started

See Getting Started for how to install and start using Kimi CLI.

Key Features

Shell command mode

Kimi CLI is not only a coding agent, but also a shell. You can switch the shell command mode by pressing Ctrl-X. In this mode, you can directly run shell commands without leaving Kimi CLI.

Note

Built-in shell commands like cd are not supported yet.

VS Code extension

Kimi CLI can be integrated with Visual Studio Code via the Kimi Code VS Code Extension.

IDE integration via ACP

Kimi CLI supports Agent Client Protocol out of the box. You can use it together with any ACP-compatible editor or IDE.

To use Kimi CLI with ACP clients, make sure to run Kimi CLI in the terminal and send /login to complete the login first. Then, you can configure your ACP client to start Kimi CLI as an ACP agent server with command kimi acp.

For example, to use Kimi CLI with Zed or JetBrains, add the following configuration to your ~/.config/zed/settings.json or ~/.jetbrains/acp.json file:

{
  "agent_servers": {
    "Kimi CLI": {
      "type": "custom",
      "command": "kimi",
      "args": ["acp"],
      "env": {}
    }
  }
}

Then you can create Kimi CLI threads in IDE's agent panel.

Zsh integration

You can use Kimi CLI together with Zsh, to empower your shell experience with AI agent capabilities.

Install the zsh-kimi-cli plugin via:

git clone https://github.com/MoonshotAI/zsh-kimi-cli.git \
  ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/kimi-cli

Note

If you are using a plugin manager other than Oh My Zsh, you may need to refer to the plugin's README for installation instructions.

Then add kimi-cli to your Zsh plugin list in ~/.zshrc:

plugins=(... kimi-cli)

After restarting Zsh, you can switch to agent mode by pressing Ctrl-X.

MCP support

Kimi CLI supports MCP (Model Context Protocol) tools.

kimi mcp sub-command group

You can manage MCP servers with kimi mcp sub-command group. For example:

# Add streamable HTTP server:
kimi mcp add --transport http context7 https://mcp.context7.com/mcp --header "CONTEXT7_API_KEY: ctx7sk-your-key"

# Add streamable HTTP server with OAuth authorization:
kimi mcp add --transport http --auth oauth linear https://mcp.linear.app/mcp

# Add stdio server:
kimi mcp add --transport stdio chrome-devtools -- npx chrome-devtools-mcp@latest

# List added MCP servers:
kimi mcp list

# Remove an MCP server:
kimi mcp remove chrome-devtools

# Authorize an MCP server:
kimi mcp auth linear

Ad-hoc MCP configuration

Kimi CLI also supports ad-hoc MCP server configuration via CLI option.

Given an MCP config file in the well-known MCP config format like the following:

{
  "mcpServers": {
    "context7": {
      "url": "https://mcp.context7.com/mcp",
      "headers": {
        "CONTEXT7_API_KEY": "YOUR_API_KEY"
      }
    },
    "chrome-devtools": {
      "command": "npx",
      "args": ["-y", "chrome-devtools-mcp@latest"]
    }
  }
}

Run kimi with --mcp-config-file option to connect to the specified MCP servers:

kimi --mcp-config-file /path/to/mcp.json

See more features in the Documentation.

Development

To develop Kimi CLI, run:

git clone https://github.com/MoonshotAI/kimi-cli.git
cd kimi-cli

make prepare  # prepare the development environment

Then you can start working on Kimi CLI.

Refer to the following commands after you make changes:

uv run kimi  # run Kimi CLI

make format  # format code
make check  # run linting and type checking
make test  # run tests
make test-kimi-cli  # run Kimi CLI tests only
make test-kosong  # run kosong tests only
make test-pykaos  # run pykaos tests only
make build-web  # build the web UI and sync it into the package (requires Node.js/npm)
make build  # build python packages
make build-bin  # build standalone binary
make help  # show all make targets

Note: make build and make build-bin automatically run make build-web to embed the web UI.

blader/humanizer

Agent skill that removes signs of AI-generated writing from text

Humanizer

A portable agent skill that removes signs of AI-generated writing from text, making it sound more natural and human. It is plain Markdown, so it can run in any harness that supports skill-style instructions.

Installation

Skills CLI

Install globally with the cross-agent skills CLI so Humanizer is available in every project:

npx skills add blader/humanizer --global

Update an existing install:

npx skills update humanizer --global

To install globally into every supported agent harness:

npx skills add blader/humanizer --global --agent '*'

To target one configured harness, pass its agent name:

npx skills add blader/humanizer --global --agent <agent-name>

Omit --global for a project-local install that can be committed and shared with collaborators. Start a new agent session or reload skills after installation.

Claude Code plugin

Claude Code users can also install Humanizer as a plugin:

/plugin marketplace add blader/humanizer
/plugin install humanizer@humanizer

The skill is then invoked as /humanizer:humanizer.

Manual

Any agent harness can use the skill directly because the runtime artifact is SKILL.md. Install it wherever your harness expects skill directories, or copy SKILL.md into an existing skill folder.

For example:

git clone https://github.com/blader/humanizer.git /path/to/your/skills/humanizer

Or, if you already have this repo cloned:

mkdir -p /path/to/your/skills/humanizer
cp SKILL.md /path/to/your/skills/humanizer/

Usage

Invoke the skill however your agent harness exposes installed skills. Common forms include a slash command or a direct request:

/humanizer

[paste your text here]

Please humanize this text: [your text]

Point it at a file and the skill rewrites it in place:

Humanize the prose in docs/launch-post.md

Voice Calibration

To match your personal writing style, provide a sample of your own writing:

/humanizer

Here's a sample of my writing for voice matching:
[paste 2-3 paragraphs of your own writing]

Now humanize this text:
[paste AI text to humanize]

The skill will analyze your sentence rhythm, word choices, and quirks, then apply them to the rewrite instead of producing generic "clean" output.

Overview

Based on Wikipedia's "Signs of AI writing" guide, maintained by WikiProject AI Cleanup. This comprehensive guide comes from observations of thousands of instances of AI-generated text.

The skill also includes a final "obviously AI generated" audit pass and a second rewrite, to catch lingering AI-isms in the first draft.

Rewrites follow a no-fabrication rule: they never add facts, names, dates, or citations that aren't in the source text. Specificity has to come from the source or the author, not from the rewrite.

Key Insight from Wikipedia

"LLMs use statistical algorithms to guess what should come next. The result tends toward the most statistically likely result that applies to the widest variety of cases."

33 Patterns Detected (with Before/After Examples)

Content Patterns

#	Pattern	Before	After
1	Significance inflation	"marking a pivotal moment in the evolution of..."	"was established in 1989 as part of a wider decentralization"
2	Notability name-dropping	"cited in NYT, BBC, FT, and The Hindu"	Trim the list; keep only sourced context
3	Superficial -ing analyses	"symbolizing... reflecting... showcasing..."	Remove, or keep only what the source supports
4	Promotional language	"nestled within the breathtaking region"	"is a town in the Gonder region"
5	Vague attributions	"Experts believe it plays a crucial role"	Name a real source or cut the claim
6	Formulaic challenges	"Despite challenges... continues to thrive"	Keep the sourced facts; cut the boosterism

Language Patterns

#	Pattern	Before	After
7	AI vocabulary	"Actually... additionally... testament... landscape... showcasing"	"also... remain common"
8	Copula avoidance	"serves as... features... boasts"	"is... has"
9	Negative parallelisms / tailing negations	"It's not just X, it's Y", "..., no guessing"	State the point directly
10	Rule of three	"innovation, inspiration, and insights"	Use natural number of items
11	Synonym cycling	"protagonist... main character... central figure... hero"	"protagonist" (repeat when clearest)
12	False ranges	"from the Big Bang to dark matter"	List topics directly
13	Passive voice / subjectless fragments	"No configuration file needed"	Name the actor when it helps clarity

Style Patterns

#	Pattern	Before	After
14	Em/en dashes	"institutions—not the people—yet this continues—"	Cut them: periods, commas, colons, or parentheses
15	Boldface overuse	"OKRs, KPIs, BMC"	"OKRs, KPIs, BMC"
16	Inline-header lists	"Performance: Performance improved"	Convert to prose
17	Title Case Headings	"Strategic Negotiations And Partnerships"	"Strategic negotiations and partnerships"
18	Emojis	"🚀 Launch Phase: 💡 Key Insight:"	Remove emojis
19	Curly quotes	`said “the project”`	`said "the project"`
26	Hyphenated word pairs	“cross-functional, data-driven, client-facing”	Drop hyphens on common word pairs
27	Persuasive authority tropes	"At its core, what matters is..."	State the point directly
28	Signposting announcements	"Let's dive in", "Here's what you need to know"	Start with the content
29	Fragmented headers	"## Performance" + "Speed matters."	Let the heading do the work
30	Diff-anchored writing	"This function was added to replace..."	Describe what it does, not what changed
31	Manufactured punchlines / staccato drama	"It had no preference. No prior. No nostalgia."	Use varied sentence lengths and concrete claims
32	Aphorism formulas	"Symmetry is the language of trust"	Replace the formula with the actual claim
33	Conversational rhetorical openers	"Honestly? It depends..."	Remove the fake-candid setup

Communication Patterns

#	Pattern	Before	After
20	Chatbot artifacts	"I hope this helps! Let me know if..."	Remove entirely
21	Cutoff disclaimers	"While details are limited in available sources..."	Find sources or remove
22	Sycophantic tone	"Great question! You're absolutely right!"	Respond directly

Filler and Hedging

#	Pattern	Before	After
23	Filler phrases	"In order to", "Due to the fact that"	"To", "Because"
24	Excessive hedging	"could potentially possibly"	"may"
25	Generic conclusions	"The future looks bright"	Specific plans or facts

Full Example

(Illustration note: the rewrite below adds specifics, like the month and the neighborhoods, that stand in for details the author would supply. In a real session those come from the user; the skill asks rather than invents.)

Before (AI-sounding):

I recently spent five unforgettable days in Lisbon, and let me tell you — this city completely stole my heart. From the moment I arrived, I knew I was somewhere truly special.

Nestled along the banks of the Tagus River, Lisbon stands as a vibrant testament to Portugal's enduring spirit, where rich history and modern energy intertwine at every turn. Yes, the famous hills are challenging — my legs certainly felt it! — but every climb rewards you with breathtaking, panoramic views that make it all worthwhile.

No trip would be complete without riding the iconic Tram 28, winding through the city's most historic neighborhoods. And the food? Simply divine. The original pastéis de nata at Pastéis de Belém are a beloved national treasure, and savoring one still warm was a moment I will never forget.

But what truly makes Lisbon special isn't just the sights — it's the feeling. Wander a few steps off the main squares and you'll discover a quieter, more authentic side: sun-drenched alleys, charming tiled facades, and friendly locals going about their daily lives, inviting you to slow down and savor every moment.

Of course, the must-see São Jorge Castle offers stunning views over the rooftops below. The queues can be long, but the experience is absolutely worth it for any history buff or curious traveler.

Would I go back? Absolutely. Lisbon isn't just a place to visit — it's a place to fall in love with, again and again. If you're dreaming of your next getaway, this is one destination that promises memories to last a lifetime. ✨

After (Humanized):

I spent five days in Lisbon last October and still have mixed feelings about it. Beautiful, yes. Also harder on the knees than anyone warned me.

The hills are the whole story and somehow never make the brochures. My hotel was up in Alfama, which photographs beautifully and translates, in practice, to climbing what felt like a six-story staircase every time I wanted coffee. By the second day my calves had opinions. I started planning each walk around which way was downhill, which is a strange way to see a city but probably an honest one.

Everyone says to ride Tram 28, so I did, wedged against a stranger's backpack for forty minutes while three tour groups filmed the same corner. I would walk the route next time, or go before breakfast. The custard tarts, though, earn the fuss. I had one at a plain little place in Graça, still warm, and for about thirty seconds I understood why people build trips around pastry.

What I did not expect was how quiet the city gets away from the main squares. Two blocks uphill from any plaza it turns into laundry lines, chipped tile, open windows, and old men watching football with the sound turned up. That is the Lisbon I keep thinking about, not the castle.

The castle is fine. The view is great, the queue is long, and I spent more time shuffling toward the entrance than looking at anything once I got inside. If I had only two days, I would trade it for an afternoon of getting lost.

I would go back, but in spring and with better shoes. Lisbon does not bend over backward to make things easy for you. I think I liked that, even when my legs disagreed.

References

Wikipedia: Signs of AI writing - Primary source
WikiProject AI Cleanup - Maintaining organization

Version History

2.9.1 - Improved distribution and portability: removed nonportable frontmatter and tool preapprovals, made global installation the documented default, added package validation, and removed the duplicated long-form example from the runtime prompt. No change to the 33 patterns.
2.9.0 - Added a no-fabrication rule: rewrites may not invent facts, names, dates, or citations not present in the source, and every example that modeled invented specifics was re-cut to use only source information (fixes #187). Replaced paragraph-count parity with an information-over-shape rule, made a user's voice sample outrank the em dash ban, and added invocation modes (pasted text / file / embedded). No change to the 33 patterns.
2.8.3 - Moved the skill version from the unsupported top-level frontmatter key to metadata.version for Agent Skills and Claude compatibility. No change to the 33 patterns.
2.8.2 - Replaced the full before/after example with a first-person Lisbon trip recap. The after now keeps the same topic, perspective, and rough length as the before while removing the AI tells without becoming clipped or slogan-like. No change to the 33 patterns.
2.8.1 - Added cross-agent installation docs, optional Claude Code plugin packaging, and a compact secondhand-text false-positive guard. No change to the 33 patterns.
2.8.0 - Added style/cadence patterns #31-33 for manufactured punchlines, aphorism formulas, and conversational rhetorical openers; expanded #20 to catch offer-to-continue chatbot closers. 33 patterns total.
2.7.0 - Added pattern #30 (diff-anchored writing); made em/en dashes a hard cut rather than "overuse"; expanded #21 to cover speculative gap-filling ("maintains a low profile"). 30 patterns total.
2.6.0 - Cleanup pass: consolidated the duplicated workflow sections, gated the personality guidance to content where voice is wanted, removed the model-fingerprinting subsection, and condensed the worked example. No change to the 29 patterns.
2.5.1 - Added a passive-voice / subjectless-fragment rule, raising the total to 29 patterns
2.5.0 - Added patterns for persuasive framing, signposting, and fragmented headers; expanded negative parallelisms to cover tailing negations; tightened wording around em dash overuse; fixed frontmatter wording to use "filler phrases"
2.4.0 - Added voice calibration: match the user's personal writing style from samples
2.3.0 - Added pattern #25: hyphenated word pair overuse
2.2.0 - Added a final "obviously AI generated" audit + second-pass rewrite prompts
2.1.1 - Fixed pattern #18 example (curly quotes vs straight quotes)
2.1.0 - Added before/after examples for all 24 patterns
2.0.0 - Complete rewrite based on raw Wikipedia article content
1.0.0 - Initial release

License

MIT

donnemartin/system-design-primer

Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

English ∙ 日本語 ∙ 简体中文 ∙ 繁體中文 | العَرَبِيَّة‎ ∙ বাংলা ∙ Português do Brasil ∙ Deutsch ∙ ελληνικά ∙ עברית ∙ Italiano ∙ 한국어 ∙ فارسی ∙ Polski ∙ русский язык ∙ Español ∙ ภาษาไทย ∙ Türkçe ∙ tiếng Việt ∙ Français | Add Translation

Help translate this guide!

The System Design Primer

Motivation

Learn how to design large-scale systems.

Prep for the system design interview.

Learn how to design large-scale systems

Learning how to design scalable systems will help you become a better engineer.

System design is a broad topic. There are a vast number of resources scattered throughout the web on system design principles.

This repo is an organized collection of resources to help you learn how to build systems at scale.

Learn from the open source community

This is a continually updated, open source project.

Contributions are welcome!

Prep for the system design interview

In addition to coding interviews, system design is a required component of the technical interview process at many tech companies.

Practice common system design interview questions and compare your results with sample solutions: discussions, code, and diagrams.

Additional topics for interview prep:

Anki flashcards

The provided Anki flashcard decks use spaced repetition to help you retain key system design concepts.

Great for use while on-the-go.

Coding Resource: Interactive Coding Challenges

Looking for resources to help you prep for the Coding Interview?

Check out the sister repo Interactive Coding Challenges, which contains an additional Anki deck:

Coding deck

Contributing

Learn from the community.

Feel free to submit pull requests to help:

Fix errors
Improve sections
Add new sections
Translate

Content that needs some polishing is placed under development.

Review the Contributing Guidelines.

Index of system design topics

Summaries of various system design topics, including pros and cons. Everything is a trade-off.

Each section contains links to more in-depth resources.

Study guide

Suggested topics to review based on your interview timeline (short, medium, long).

Q: For interviews, do I need to know everything here?

A: No, you don't need to know everything here to prepare for the interview.

What you are asked in an interview depends on variables such as:

How much experience you have
What your technical background is
What positions you are interviewing for
Which companies you are interviewing with
Luck

More experienced candidates are generally expected to know more about system design. Architects or team leads might be expected to know more than individual contributors. Top tech companies are likely to have one or more design interview rounds.

Start broad and go deeper in a few areas. It helps to know a little about various key system design topics. Adjust the following guide based on your timeline, experience, what positions you are interviewing for, and which companies you are interviewing with.

Short timeline - Aim for breadth with system design topics. Practice by solving some interview questions.
Medium timeline - Aim for breadth and some depth with system design topics. Practice by solving many interview questions.
Long timeline - Aim for breadth and more depth with system design topics. Practice by solving most interview questions.

	Short	Medium	Long
Read through the System design topics to get a broad understanding of how systems work	👍	👍	👍
Read through a few articles in the Company engineering blogs for the companies you are interviewing with	👍	👍	👍
Read through a few Real world architectures	👍	👍	👍
Review How to approach a system design interview question	👍	👍	👍
Work through System design interview questions with solutions	Some	Many	Most
Work through Object-oriented design interview questions with solutions	Some	Many	Most
Review Additional system design interview questions	Some	Many	Most

How to approach a system design interview question

How to tackle a system design interview question.

The system design interview is an open-ended conversation. You are expected to lead it.

You can use the following steps to guide the discussion. To help solidify this process, work through the System design interview questions with solutions section using the following steps.

Step 1: Outline use cases, constraints, and assumptions

Gather requirements and scope the problem. Ask questions to clarify use cases and constraints. Discuss assumptions.

Who is going to use it?
How are they going to use it?
How many users are there?
What does the system do?
What are the inputs and outputs of the system?
How much data do we expect to handle?
How many requests per second do we expect?
What is the expected read to write ratio?

Step 2: Create a high level design

Outline a high level design with all important components.

Sketch the main components and connections
Justify your ideas

Step 3: Design core components

Dive into details for each core component. For example, if you were asked to design a url shortening service, discuss:

Generating and storing a hash of the full url
- MD5 and Base62
- Hash collisions
- SQL or NoSQL
- Database schema
Translating a hashed url to the full url
- Database lookup
API and object-oriented design

Step 4: Scale the design

Identify and address bottlenecks, given the constraints. For example, do you need the following to address scalability issues?

Load balancer
Horizontal scaling
Caching
Database sharding

Discuss potential solutions and trade-offs. Everything is a trade-off. Address bottlenecks using principles of scalable system design.

Back-of-the-envelope calculations

You might be asked to do some estimates by hand. Refer to the Appendix for the following resources:

Source(s) and further reading

Check out the following links to get a better idea of what to expect:

System design interview questions with solutions

Common system design interview questions with sample discussions, code, and diagrams.

Solutions linked to content in the solutions/ folder.

Question
Design Pastebin.com (or Bit.ly)	Solution
Design the Twitter timeline and search (or Facebook feed and search)	Solution
Design a web crawler	Solution
Design Mint.com	Solution
Design the data structures for a social network	Solution
Design a key-value store for a search engine	Solution
Design Amazon's sales ranking by category feature	Solution
Design a system that scales to millions of users on AWS	Solution
Add a system design question	Contribute

Object-oriented design interview questions with solutions

Common object-oriented design interview questions with sample discussions, code, and diagrams.

Solutions linked to content in the solutions/ folder.

Note: This section is under development

Question
Design a hash map	Solution
Design a least recently used cache	Solution
Design a call center	Solution
Design a deck of cards	Solution
Design a parking lot	Solution
Design a chat server	Solution
Design a circular array	Contribute
Add an object-oriented design question	Contribute

System design topics: start here

New to system design?

First, you'll need a basic understanding of common principles, learning about what they are, how they are used, and their pros and cons.

Step 1: Review the scalability video lecture

Scalability Lecture at Harvard

Topics covered:
- Vertical scaling
- Horizontal scaling
- Caching
- Load balancing
- Database replication
- Database partitioning

Step 2: Review the scalability article

Scalability

Topics covered:
- Clones
- Databases
- Caches
- Asynchronism

Next steps

Next, we'll look at high-level trade-offs:

Performance vs scalability
Latency vs throughput
Availability vs consistency

Keep in mind that everything is a trade-off.

Then we'll dive into more specific topics such as DNS, CDNs, and load balancers.

Performance vs scalability

A service is scalable if it results in increased performance in a manner proportional to resources added. Generally, increasing performance means serving more units of work, but it can also be to handle larger units of work, such as when datasets grow.¹

Another way to look at performance vs scalability:

If you have a performance problem, your system is slow for a single user.
If you have a scalability problem, your system is fast for a single user but slow under heavy load.

Source(s) and further reading

Latency vs throughput

Latency is the time to perform some action or to produce some result.

Throughput is the number of such actions or results per unit of time.

Generally, you should aim for maximal throughput with acceptable latency.

Source(s) and further reading

Understanding latency vs throughput

Availability vs consistency

CAP theorem

Source: CAP theorem revisited

In a distributed computer system, you can only support two of the following guarantees:

Consistency - Every read receives the most recent write or an error
Availability - Every request receives a response, without guarantee that it contains the most recent version of the information
Partition Tolerance - The system continues to operate despite arbitrary partitioning due to network failures

Networks aren't reliable, so you'll need to support partition tolerance. You'll need to make a software tradeoff between consistency and availability.

CP - consistency and partition tolerance

Waiting for a response from the partitioned node might result in a timeout error. CP is a good choice if your business needs require atomic reads and writes.

AP - availability and partition tolerance

Responses return the most readily available version of the data available on any node, which might not be the latest. Writes might take some time to propagate when the partition is resolved.

AP is a good choice if the business needs to allow for eventual consistency or when the system needs to continue working despite external errors.

Source(s) and further reading

Consistency patterns

With multiple copies of the same data, we are faced with options on how to synchronize them so clients have a consistent view of the data. Recall the definition of consistency from the CAP theorem - Every read receives the most recent write or an error.

Weak consistency

After a write, reads may or may not see it. A best effort approach is taken.

This approach is seen in systems such as memcached. Weak consistency works well in real time use cases such as VoIP, video chat, and realtime multiplayer games. For example, if you are on a phone call and lose reception for a few seconds, when you regain connection you do not hear what was spoken during connection loss.

Eventual consistency

After a write, reads will eventually see it (typically within milliseconds). Data is replicated asynchronously.

This approach is seen in systems such as DNS and email. Eventual consistency works well in highly available systems.

Strong consistency

After a write, reads will see it. Data is replicated synchronously.

This approach is seen in file systems and RDBMSes. Strong consistency works well in systems that need transactions.

Source(s) and further reading

Transactions across data centers

Availability patterns

There are two complementary patterns to support high availability: fail-over and replication.

Fail-over

Active-passive

With active-passive fail-over, heartbeats are sent between the active and the passive server on standby. If the heartbeat is interrupted, the passive server takes over the active's IP address and resumes service.

The length of downtime is determined by whether the passive server is already running in 'hot' standby or whether it needs to start up from 'cold' standby. Only the active server handles traffic.

Active-passive failover can also be referred to as master-slave failover.

Active-active

In active-active, both servers are managing traffic, spreading the load between them.

If the servers are public-facing, the DNS would need to know about the public IPs of both servers. If the servers are internal-facing, application logic would need to know about both servers.

Active-active failover can also be referred to as master-master failover.

Disadvantage(s): failover

Fail-over adds more hardware and additional complexity.
There is a potential for loss of data if the active system fails before any newly written data can be replicated to the passive.

Replication

Master-slave and master-master

This topic is further discussed in the Database section:

Availability in numbers

Availability is often quantified by uptime (or downtime) as a percentage of time the service is available. Availability is generally measured in number of 9s--a service with 99.99% availability is described as having four 9s.

99.9% availability - three 9s

Duration	Acceptable downtime
Downtime per year	8h 45min 57s
Downtime per month	43m 49.7s
Downtime per week	10m 4.8s
Downtime per day	1m 26.4s

99.99% availability - four 9s

Duration	Acceptable downtime
Downtime per year	52min 35.7s
Downtime per month	4m 23s
Downtime per week	1m 5s
Downtime per day	8.6s

Availability in parallel vs in sequence

If a service consists of multiple components prone to failure, the service's overall availability depends on whether the components are in sequence or in parallel.

In sequence

Overall availability decreases when two components with availability < 100% are in sequence:

Availability (Total) = Availability (Foo) * Availability (Bar)

If both Foo and Bar each had 99.9% availability, their total availability in sequence would be 99.8%.

In parallel

Overall availability increases when two components with availability < 100% are in parallel:

Availability (Total) = 1 - (1 - Availability (Foo)) * (1 - Availability (Bar))

If both Foo and Bar each had 99.9% availability, their total availability in parallel would be 99.9999%.

Domain name system

Source: DNS security presentation

A Domain Name System (DNS) translates a domain name such as www.example.com to an IP address.

DNS is hierarchical, with a few authoritative servers at the top level. Your router or ISP provides information about which DNS server(s) to contact when doing a lookup. Lower level DNS servers cache mappings, which could become stale due to DNS propagation delays. DNS results can also be cached by your browser or OS for a certain period of time, determined by the time to live (TTL).

NS record (name server) - Specifies the DNS servers for your domain/subdomain.
MX record (mail exchange) - Specifies the mail servers for accepting messages.
A record (address) - Points a name to an IP address.
CNAME (canonical) - Points a name to another name or CNAME (example.com to www.example.com) or to an A record.

Services such as CloudFlare and Route 53 provide managed DNS services. Some DNS services can route traffic through various methods:

Weighted round robin
- Prevent traffic from going to servers under maintenance
- Balance between varying cluster sizes
- A/B testing
Latency-based
Geolocation-based

Disadvantage(s): DNS

Accessing a DNS server introduces a slight delay, although mitigated by caching described above.
DNS server management could be complex and is generally managed by governments, ISPs, and large companies.
DNS services have recently come under DDoS attack, preventing users from accessing websites such as Twitter without knowing Twitter's IP address(es).

Source(s) and further reading

Content delivery network

Source: Why use a CDN

A content delivery network (CDN) is a globally distributed network of proxy servers, serving content from locations closer to the user. Generally, static files such as HTML/CSS/JS, photos, and videos are served from CDN, although some CDNs such as Amazon's CloudFront support dynamic content. The site's DNS resolution will tell clients which server to contact.

Serving content from CDNs can significantly improve performance in two ways:

Users receive content from data centers close to them
Your servers do not have to serve requests that the CDN fulfills

Push CDNs

Push CDNs receive new content whenever changes occur on your server. You take full responsibility for providing content, uploading directly to the CDN and rewriting URLs to point to the CDN. You can configure when content expires and when it is updated. Content is uploaded only when it is new or changed, minimizing traffic, but maximizing storage.

Sites with a small amount of traffic or sites with content that isn't often updated work well with push CDNs. Content is placed on the CDNs once, instead of being re-pulled at regular intervals.

Pull CDNs

Pull CDNs grab new content from your server when the first user requests the content. You leave the content on your server and rewrite URLs to point to the CDN. This results in a slower request until the content is cached on the CDN.

A time-to-live (TTL) determines how long content is cached. Pull CDNs minimize storage space on the CDN, but can create redundant traffic if files expire and are pulled before they have actually changed.

Sites with heavy traffic work well with pull CDNs, as traffic is spread out more evenly with only recently-requested content remaining on the CDN.

Disadvantage(s): CDN

CDN costs could be significant depending on traffic, although this should be weighed with additional costs you would incur not using a CDN.
Content might be stale if it is updated before the TTL expires it.
CDNs require changing URLs for static content to point to the CDN.

Source(s) and further reading

Load balancer

Source: Scalable system design patterns

Load balancers distribute incoming client requests to computing resources such as application servers and databases. In each case, the load balancer returns the response from the computing resource to the appropriate client. Load balancers are effective at:

Preventing requests from going to unhealthy servers
Preventing overloading resources
Helping to eliminate a single point of failure

Load balancers can be implemented with hardware (expensive) or with software such as HAProxy.

Additional benefits include:

SSL termination - Decrypt incoming requests and encrypt server responses so backend servers do not have to perform these potentially expensive operations
- Removes the need to install X.509 certificates on each server
Session persistence - Issue cookies and route a specific client's requests to same instance if the web apps do not keep track of sessions

To protect against failures, it's common to set up multiple load balancers, either in active-passive or active-active mode.

Load balancers can route traffic based on various metrics, including:

Random
Least loaded
Session/cookies
Round robin or weighted round robin
Layer 4
Layer 7

Layer 4 load balancing

Layer 4 load balancers look at info at the transport layer to decide how to distribute requests. Generally, this involves the source, destination IP addresses, and ports in the header, but not the contents of the packet. Layer 4 load balancers forward network packets to and from the upstream server, performing Network Address Translation (NAT).

Layer 7 load balancing

Layer 7 load balancers look at the application layer to decide how to distribute requests. This can involve contents of the header, message, and cookies. Layer 7 load balancers terminate network traffic, reads the message, makes a load-balancing decision, then opens a connection to the selected server. For example, a layer 7 load balancer can direct video traffic to servers that host videos while directing more sensitive user billing traffic to security-hardened servers.

At the cost of flexibility, layer 4 load balancing requires less time and computing resources than Layer 7, although the performance impact can be minimal on modern commodity hardware.

Horizontal scaling

Load balancers can also help with horizontal scaling, improving performance and availability. Scaling out using commodity machines is more cost efficient and results in higher availability than scaling up a single server on more expensive hardware, called Vertical Scaling. It is also easier to hire for talent working on commodity hardware than it is for specialized enterprise systems.

Disadvantage(s): horizontal scaling

Scaling horizontally introduces complexity and involves cloning servers
- Servers should be stateless: they should not contain any user-related data like sessions or profile pictures
- Sessions can be stored in a centralized data store such as a database (SQL, NoSQL) or a persistent cache (Redis, Memcached)
Downstream servers such as caches and databases need to handle more simultaneous connections as upstream servers scale out

Disadvantage(s): load balancer

The load balancer can become a performance bottleneck if it does not have enough resources or if it is not configured properly.
Introducing a load balancer to help eliminate a single point of failure results in increased complexity.
A single load balancer is a single point of failure, configuring multiple load balancers further increases complexity.

Source(s) and further reading

Reverse proxy (web server)

Source: Wikipedia

A reverse proxy is a web server that centralizes internal services and provides unified interfaces to the public. Requests from clients are forwarded to a server that can fulfill it before the reverse proxy returns the server's response to the client.

Additional benefits include:

Increased security - Hide information about backend servers, blacklist IPs, limit number of connections per client
Increased scalability and flexibility - Clients only see the reverse proxy's IP, allowing you to scale servers or change their configuration
SSL termination - Decrypt incoming requests and encrypt server responses so backend servers do not have to perform these potentially expensive operations
- Removes the need to install X.509 certificates on each server
Compression - Compress server responses
Caching - Return the response for cached requests
Static content - Serve static content directly
- HTML/CSS/JS
- Photos
- Videos
- Etc

Load balancer vs reverse proxy

Deploying a load balancer is useful when you have multiple servers. Often, load balancers route traffic to a set of servers serving the same function.
Reverse proxies can be useful even with just one web server or application server, opening up the benefits described in the previous section.
Solutions such as NGINX and HAProxy can support both layer 7 reverse proxying and load balancing.

Disadvantage(s): reverse proxy

Introducing a reverse proxy results in increased complexity.
A single reverse proxy is a single point of failure, configuring multiple reverse proxies (ie a failover) further increases complexity.

Source(s) and further reading

Application layer

Source: Intro to architecting systems for scale

Separating out the web layer from the application layer (also known as platform layer) allows you to scale and configure both layers independently. Adding a new API results in adding application servers without necessarily adding additional web servers. The single responsibility principle advocates for small and autonomous services that work together. Small teams with small services can plan more aggressively for rapid growth.

Workers in the application layer also help enable asynchronism.

Microservices

Related to this discussion are microservices, which can be described as a suite of independently deployable, small, modular services. Each service runs a unique process and communicates through a well-defined, lightweight mechanism to serve a business goal. ¹

Pinterest, for example, could have the following microservices: user profile, follower, feed, search, photo upload, etc.

Service Discovery

Systems such as Consul, Etcd, and Zookeeper can help services find each other by keeping track of registered names, addresses, and ports. Health checks help verify service integrity and are often done using an HTTP endpoint. Both Consul and Etcd have a built in key-value store that can be useful for storing config values and other shared data.

Disadvantage(s): application layer

Adding an application layer with loosely coupled services requires a different approach from an architectural, operations, and process viewpoint (vs a monolithic system).
Microservices can add complexity in terms of deployments and operations.

Source(s) and further reading

Database

Source: Scaling up to your first 10 million users

Relational database management system (RDBMS)

A relational database like SQL is a collection of data items organized in tables.

ACID is a set of properties of relational database transactions.

Atomicity - Each transaction is all or nothing
Consistency - Any transaction will bring the database from one valid state to another
Isolation - Executing transactions concurrently has the same results as if the transactions were executed serially
Durability - Once a transaction has been committed, it will remain so

There are many techniques to scale a relational database: master-slave replication, master-master replication, federation, sharding, denormalization, and SQL tuning.

Master-slave replication

The master serves reads and writes, replicating writes to one or more slaves, which serve only reads. Slaves can also replicate to additional slaves in a tree-like fashion. If the master goes offline, the system can continue to operate in read-only mode until a slave is promoted to a master or a new master is provisioned.

Source: Scalability, availability, stability, patterns

Disadvantage(s): master-slave replication

Additional logic is needed to promote a slave to a master.
See Disadvantage(s): replication for points related to both master-slave and master-master.

Master-master replication

Both masters serve reads and writes and coordinate with each other on writes. If either master goes down, the system can continue to operate with both reads and writes.

Source: Scalability, availability, stability, patterns

Disadvantage(s): master-master replication

You'll need a load balancer or you'll need to make changes to your application logic to determine where to write.
Most master-master systems are either loosely consistent (violating ACID) or have increased write latency due to synchronization.
Conflict resolution comes more into play as more write nodes are added and as latency increases.
See Disadvantage(s): replication for points related to both master-slave and master-master.

Disadvantage(s): replication

There is a potential for loss of data if the master fails before any newly written data can be replicated to other nodes.
Writes are replayed to the read replicas. If there are a lot of writes, the read replicas can get bogged down with replaying writes and can't do as many reads.
The more read slaves, the more you have to replicate, which leads to greater replication lag.
On some systems, writing to the master can spawn multiple threads to write in parallel, whereas read replicas only support writing sequentially with a single thread.
Replication adds more hardware and additional complexity.

Source(s) and further reading: replication

Federation

Source: Scaling up to your first 10 million users

Federation (or functional partitioning) splits up databases by function. For example, instead of a single, monolithic database, you could have three databases: forums, users, and products, resulting in less read and write traffic to each database and therefore less replication lag. Smaller databases result in more data that can fit in memory, which in turn results in more cache hits due to improved cache locality. With no single central master serializing writes you can write in parallel, increasing throughput.

Disadvantage(s): federation

Federation is not effective if your schema requires huge functions or tables.
You'll need to update your application logic to determine which database to read and write.
Joining data from two databases is more complex with a server link.
Federation adds more hardware and additional complexity.

Source(s) and further reading: federation

Scaling up to your first 10 million users

Sharding

Source: Scalability, availability, stability, patterns

Sharding distributes data across different databases such that each database can only manage a subset of the data. Taking a users database as an example, as the number of users increases, more shards are added to the cluster.

Similar to the advantages of federation, sharding results in less read and write traffic, less replication, and more cache hits. Index size is also reduced, which generally improves performance with faster queries. If one shard goes down, the other shards are still operational, although you'll want to add some form of replication to avoid data loss. Like federation, there is no single central master serializing writes, allowing you to write in parallel with increased throughput.

Common ways to shard a table of users is either through the user's last name initial or the user's geographic location.

Disadvantage(s): sharding

You'll need to update your application logic to work with shards, which could result in complex SQL queries.
Data distribution can become lopsided in a shard. For example, a set of power users on a shard could result in increased load to that shard compared to others.
- Rebalancing adds additional complexity. A sharding function based on consistent hashing can reduce the amount of transferred data.
Joining data from multiple shards is more complex.
Sharding adds more hardware and additional complexity.

Source(s) and further reading: sharding

Denormalization

Denormalization attempts to improve read performance at the expense of some write performance. Redundant copies of the data are written in multiple tables to avoid expensive joins. Some RDBMS such as PostgreSQL and Oracle support materialized views which handle the work of storing redundant information and keeping redundant copies consistent.

Once data becomes distributed with techniques such as federation and sharding, managing joins across data centers further increases complexity. Denormalization might circumvent the need for such complex joins.

In most systems, reads can heavily outnumber writes 100:1 or even 1000:1. A read resulting in a complex database join can be very expensive, spending a significant amount of time on disk operations.

Disadvantage(s): denormalization

Data is duplicated.
Constraints can help redundant copies of information stay in sync, which increases complexity of the database design.
A denormalized database under heavy write load might perform worse than its normalized counterpart.

Source(s) and further reading: denormalization

Denormalization

SQL tuning

SQL tuning is a broad topic and many books have been written as reference.

It's important to benchmark and profile to simulate and uncover bottlenecks.

Benchmark - Simulate high-load situations with tools such as ab.
Profile - Enable tools such as the slow query log to help track performance issues.

Benchmarking and profiling might point you to the following optimizations.

Tighten up the schema

MySQL dumps to disk in contiguous blocks for fast access.
Use CHAR instead of VARCHAR for fixed-length fields.
- CHAR effectively allows for fast, random access, whereas with VARCHAR, you must find the end of a string before moving onto the next one.
Use TEXT for large blocks of text such as blog posts. TEXT also allows for boolean searches. Using a TEXT field results in storing a pointer on disk that is used to locate the text block.
Use INT for larger numbers up to 2^32 or 4 billion.
Use DECIMAL for currency to avoid floating point representation errors.
Avoid storing large BLOBS, store the location of where to get the object instead.
VARCHAR(255) is the largest number of characters that can be counted in an 8 bit number, often maximizing the use of a byte in some RDBMS.
Set the NOT NULL constraint where applicable to improve search performance.

Use good indices

Columns that you are querying (SELECT, GROUP BY, ORDER BY, JOIN) could be faster with indices.
Indices are usually represented as self-balancing B-tree that keeps data sorted and allows searches, sequential access, insertions, and deletions in logarithmic time.
Placing an index can keep the data in memory, requiring more space.
Writes could also be slower since the index also needs to be updated.
When loading large amounts of data, it might be faster to disable indices, load the data, then rebuild the indices.

Avoid expensive joins

Denormalize where performance demands it.

Partition tables

Break up a table by putting hot spots in a separate table to help keep it in memory.

Tune the query cache

In some cases, the query cache could lead to performance issues.

Source(s) and further reading: SQL tuning

NoSQL

NoSQL is a collection of data items represented in a key-value store, document store, wide column store, or a graph database. Data is denormalized, and joins are generally done in the application code. Most NoSQL stores lack true ACID transactions and favor eventual consistency.

BASE is often used to describe the properties of NoSQL databases. In comparison with the CAP Theorem, BASE chooses availability over consistency.

Basically available - the system guarantees availability.
Soft state - the state of the system may change over time, even without input.
Eventual consistency - the system will become consistent over a period of time, given that the system doesn't receive input during that period.

In addition to choosing between SQL or NoSQL, it is helpful to understand which type of NoSQL database best fits your use case(s). We'll review key-value stores, document stores, wide column stores, and graph databases in the next section.

Key-value store

Abstraction: hash table

A key-value store generally allows for O(1) reads and writes and is often backed by memory or SSD. Data stores can maintain keys in lexicographic order, allowing efficient retrieval of key ranges. Key-value stores can allow for storing of metadata with a value.

Key-value stores provide high performance and are often used for simple data models or for rapidly-changing data, such as an in-memory cache layer. Since they offer only a limited set of operations, complexity is shifted to the application layer if additional operations are needed.

A key-value store is the basis for more complex systems such as a document store, and in some cases, a graph database.

Source(s) and further reading: key-value store

Document store

Abstraction: key-value store with documents stored as values

A document store is centered around documents (XML, JSON, binary, etc), where a document stores all information for a given object. Document stores provide APIs or a query language to query based on the internal structure of the document itself. Note, many key-value stores include features for working with a value's metadata, blurring the lines between these two storage types.

Based on the underlying implementation, documents are organized by collections, tags, metadata, or directories. Although documents can be organized or grouped together, documents may have fields that are completely different from each other.

Some document stores like MongoDB and CouchDB also provide a SQL-like language to perform complex queries. DynamoDB supports both key-values and documents.

Document stores provide high flexibility and are often used for working with occasionally changing data.

Source(s) and further reading: document store

Wide column store

Source: SQL & NoSQL, a brief history

Abstraction: nested map ColumnFamily<RowKey, Columns<ColKey, Value, Timestamp>>

A wide column store's basic unit of data is a column (name/value pair). A column can be grouped in column families (analogous to a SQL table). Super column families further group column families. You can access each column independently with a row key, and columns with the same row key form a row. Each value contains a timestamp for versioning and for conflict resolution.

Google introduced Bigtable as the first wide column store, which influenced the open-source HBase often-used in the Hadoop ecosystem, and Cassandra from Facebook. Stores such as BigTable, HBase, and Cassandra maintain keys in lexicographic order, allowing efficient retrieval of selective key ranges.

Wide column stores offer high availability and high scalability. They are often used for very large data sets.

Source(s) and further reading: wide column store

Graph database

Source: Graph database

Abstraction: graph

In a graph database, each node is a record and each arc is a relationship between two nodes. Graph databases are optimized to represent complex relationships with many foreign keys or many-to-many relationships.

Graphs databases offer high performance for data models with complex relationships, such as a social network. They are relatively new and are not yet widely-used; it might be more difficult to find development tools and resources. Many graphs can only be accessed with REST APIs.

Source(s) and further reading: graph

Source(s) and further reading: NoSQL

SQL or NoSQL

Source: Transitioning from RDBMS to NoSQL

Reasons for SQL:

Structured data
Strict schema
Relational data
Need for complex joins
Transactions
Clear patterns for scaling
More established: developers, community, code, tools, etc
Lookups by index are very fast

Reasons for NoSQL:

Semi-structured data
Dynamic or flexible schema
Non-relational data
No need for complex joins
Store many TB (or PB) of data
Very data intensive workload
Very high throughput for IOPS

Sample data well-suited for NoSQL:

Rapid ingest of clickstream and log data
Leaderboard or scoring data
Temporary data, such as a shopping cart
Frequently accessed ('hot') tables
Metadata/lookup tables

Source(s) and further reading: SQL or NoSQL

Cache

Source: Scalable system design patterns

Caching improves page load times and can reduce the load on your servers and databases. In this model, the dispatcher will first lookup if the request has been made before and try to find the previous result to return, in order to save the actual execution.

Databases often benefit from a uniform distribution of reads and writes across its partitions. Popular items can skew the distribution, causing bottlenecks. Putting a cache in front of a database can help absorb uneven loads and spikes in traffic.

Client caching

Caches can be located on the client side (OS or browser), server side, or in a distinct cache layer.

CDN caching

CDNs are considered a type of cache.

Web server caching

Reverse proxies and caches such as Varnish can serve static and dynamic content directly. Web servers can also cache requests, returning responses without having to contact application servers.

Database caching

Your database usually includes some level of caching in a default configuration, optimized for a generic use case. Tweaking these settings for specific usage patterns can further boost performance.

Application caching

In-memory caches such as Memcached and Redis are key-value stores between your application and your data storage. Since the data is held in RAM, it is much faster than typical databases where data is stored on disk. RAM is more limited than disk, so cache invalidation algorithms such as least recently used (LRU) can help invalidate 'cold' entries and keep 'hot' data in RAM.

Redis has the following additional features:

Persistence option
Built-in data structures such as sorted sets and lists

There are multiple levels you can cache that fall into two general categories: database queries and objects:

Row level
Query-level
Fully-formed serializable objects
Fully-rendered HTML

Generally, you should try to avoid file-based caching, as it makes cloning and auto-scaling more difficult.

Caching at the database query level

Whenever you query the database, hash the query as a key and store the result to the cache. This approach suffers from expiration issues:

Hard to delete a cached result with complex queries
If one piece of data changes such as a table cell, you need to delete all cached queries that might include the changed cell

Caching at the object level

See your data as an object, similar to what you do with your application code. Have your application assemble the dataset from the database into a class instance or a data structure(s):

Remove the object from cache if its underlying data has changed
Allows for asynchronous processing: workers assemble objects by consuming the latest cached object

Suggestions of what to cache:

User sessions
Fully rendered web pages
Activity streams
User graph data

When to update the cache

Since you can only store a limited amount of data in cache, you'll need to determine which cache update strategy works best for your use case.

Cache-aside

Source: From cache to in-memory data grid

The application is responsible for reading and writing from storage. The cache does not interact with storage directly. The application does the following:

Look for entry in cache, resulting in a cache miss
Load entry from the database
Add entry to cache
Return entry

def get_user(self, user_id):
    user = cache.get("user.{0}", user_id)
    if user is None:
        user = db.query("SELECT * FROM users WHERE user_id = {0}", user_id)
        if user is not None:
            key = "user.{0}".format(user_id)
            cache.set(key, json.dumps(user))
    return user

Memcached is generally used in this manner.

Subsequent reads of data added to cache are fast. Cache-aside is also referred to as lazy loading. Only requested data is cached, which avoids filling up the cache with data that isn't requested.

Disadvantage(s): cache-aside

Each cache miss results in three trips, which can cause a noticeable delay.
Data can become stale if it is updated in the database. This issue is mitigated by setting a time-to-live (TTL) which forces an update of the cache entry, or by using write-through.
When a node fails, it is replaced by a new, empty node, increasing latency.

Write-through

Source: Scalability, availability, stability, patterns

The application uses the cache as the main data store, reading and writing data to it, while the cache is responsible for reading and writing to the database:

Application adds/updates entry in cache
Cache synchronously writes entry to data store
Return

Application code:

set_user(12345, {"foo":"bar"})

Cache code:

def set_user(user_id, values):
    user = db.query("UPDATE Users WHERE id = {0}", user_id, values)
    cache.set(user_id, user)

Write-through is a slow overall operation due to the write operation, but subsequent reads of just written data are fast. Users are generally more tolerant of latency when updating data than reading data. Data in the cache is not stale.

Disadvantage(s): write through

When a new node is created due to failure or scaling, the new node will not cache entries until the entry is updated in the database. Cache-aside in conjunction with write through can mitigate this issue.
Most data written might never be read, which can be minimized with a TTL.

Write-behind (write-back)

Source: Scalability, availability, stability, patterns

In write-behind, the application does the following:

Add/update entry in cache
Asynchronously write entry to the data store, improving write performance

Disadvantage(s): write-behind

There could be data loss if the cache goes down prior to its contents hitting the data store.
It is more complex to implement write-behind than it is to implement cache-aside or write-through.

Refresh-ahead

Source: From cache to in-memory data grid

You can configure the cache to automatically refresh any recently accessed cache entry prior to its expiration.

Refresh-ahead can result in reduced latency vs read-through if the cache can accurately predict which items are likely to be needed in the future.

Disadvantage(s): refresh-ahead

Not accurately predicting which items are likely to be needed in the future can result in reduced performance than without refresh-ahead.

Disadvantage(s): cache

Need to maintain consistency between caches and the source of truth such as the database through cache invalidation.
Cache invalidation is a difficult problem, there is additional complexity associated with when to update the cache.
Need to make application changes such as adding Redis or memcached.

Source(s) and further reading

Asynchronism

Source: Intro to architecting systems for scale

Asynchronous workflows help reduce request times for expensive operations that would otherwise be performed in-line. They can also help by doing time-consuming work in advance, such as periodic aggregation of data.

Message queues

Message queues receive, hold, and deliver messages. If an operation is too slow to perform inline, you can use a message queue with the following workflow:

An application publishes a job to the queue, then notifies the user of job status
A worker picks up the job from the queue, processes it, then signals the job is complete

The user is not blocked and the job is processed in the background. During this time, the client might optionally do a small amount of processing to make it seem like the task has completed. For example, if posting a tweet, the tweet could be instantly posted to your timeline, but it could take some time before your tweet is actually delivered to all of your followers.

Redis is useful as a simple message broker but messages can be lost.

RabbitMQ is popular but requires you to adapt to the 'AMQP' protocol and manage your own nodes.

Amazon SQS is hosted but can have high latency and has the possibility of messages being delivered twice.

Task queues

Tasks queues receive tasks and their related data, runs them, then delivers their results. They can support scheduling and can be used to run computationally-intensive jobs in the background.

Celery has support for scheduling and primarily has python support.

Back pressure

If queues start to grow significantly, the queue size can become larger than memory, resulting in cache misses, disk reads, and even slower performance. Back pressure can help by limiting the queue size, thereby maintaining a high throughput rate and good response times for jobs already in the queue. Once the queue fills up, clients get a server busy or HTTP 503 status code to try again later. Clients can retry the request at a later time, perhaps with exponential backoff.

Disadvantage(s): asynchronism

Use cases such as inexpensive calculations and realtime workflows might be better suited for synchronous operations, as introducing queues can add delays and complexity.

Source(s) and further reading

Communication

Source: OSI 7 layer model

Hypertext transfer protocol (HTTP)

HTTP is a method for encoding and transporting data between a client and a server. It is a request/response protocol: clients issue requests and servers issue responses with relevant content and completion status info about the request. HTTP is self-contained, allowing requests and responses to flow through many intermediate routers and servers that perform load balancing, caching, encryption, and compression.

A basic HTTP request consists of a verb (method) and a resource (endpoint). Below are common HTTP verbs:

Verb	Description	Idempotent*	Safe	Cacheable
GET	Reads a resource	Yes	Yes	Yes
POST	Creates a resource or trigger a process that handles data	No	No	Yes if response contains freshness info
PUT	Creates or replace a resource	Yes	No	No
PATCH	Partially updates a resource	No	No	Yes if response contains freshness info
DELETE	Deletes a resource	Yes	No	No

*Can be called many times without different outcomes.

HTTP is an application layer protocol relying on lower-level protocols such as TCP and UDP.

Source(s) and further reading: HTTP

Transmission control protocol (TCP)

Source: How to make a multiplayer game

TCP is a connection-oriented protocol over an IP network. Connection is established and terminated using a handshake. All packets sent are guaranteed to reach the destination in the original order and without corruption through:

Sequence numbers and checksum fields for each packet
Acknowledgement packets and automatic retransmission

If the sender does not receive a correct response, it will resend the packets. If there are multiple timeouts, the connection is dropped. TCP also implements flow control and congestion control. These guarantees cause delays and generally result in less efficient transmission than UDP.

To ensure high throughput, web servers can keep a large number of TCP connections open, resulting in high memory usage. It can be expensive to have a large number of open connections between web server threads and say, a memcached server. Connection pooling can help in addition to switching to UDP where applicable.

TCP is useful for applications that require high reliability but are less time critical. Some examples include web servers, database info, SMTP, FTP, and SSH.

Use TCP over UDP when:

You need all of the data to arrive intact
You want to automatically make a best estimate use of the network throughput

User datagram protocol (UDP)

Source: How to make a multiplayer game

UDP is connectionless. Datagrams (analogous to packets) are guaranteed only at the datagram level. Datagrams might reach their destination out of order or not at all. UDP does not support congestion control. Without the guarantees that TCP support, UDP is generally more efficient.

UDP can broadcast, sending datagrams to all devices on the subnet. This is useful with DHCP because the client has not yet received an IP address, thus preventing a way for TCP to stream without the IP address.

UDP is less reliable but works well in real time use cases such as VoIP, video chat, streaming, and realtime multiplayer games.

Use UDP over TCP when:

You need the lowest latency
Late data is worse than loss of data
You want to implement your own error correction

Source(s) and further reading: TCP and UDP

Remote procedure call (RPC)

Source: Crack the system design interview

In an RPC, a client causes a procedure to execute on a different address space, usually a remote server. The procedure is coded as if it were a local procedure call, abstracting away the details of how to communicate with the server from the client program. Remote calls are usually slower and less reliable than local calls so it is helpful to distinguish RPC calls from local calls. Popular RPC frameworks include Protobuf, Thrift, and Avro.

RPC is a request-response protocol:

Client program - Calls the client stub procedure. The parameters are pushed onto the stack like a local procedure call.
Client stub procedure - Marshals (packs) procedure id and arguments into a request message.
Client communication module - OS sends the message from the client to the server.
Server communication module - OS passes the incoming packets to the server stub procedure.
Server stub procedure - Unmarshalls the results, calls the server procedure matching the procedure id and passes the given arguments.
The server response repeats the steps above in reverse order.

Sample RPC calls:

GET /someoperation?data=anId

POST /anotheroperation
{
  "data":"anId";
  "anotherdata": "another value"
}

RPC is focused on exposing behaviors. RPCs are often used for performance reasons with internal communications, as you can hand-craft native calls to better fit your use cases.

Choose a native library (aka SDK) when:

You know your target platform.
You want to control how your "logic" is accessed.
You want to control how error control happens off your library.
Performance and end user experience is your primary concern.

HTTP APIs following REST tend to be used more often for public APIs.

Disadvantage(s): RPC

RPC clients become tightly coupled to the service implementation.
A new API must be defined for every new operation or use case.
It can be difficult to debug RPC.
You might not be able to leverage existing technologies out of the box. For example, it might require additional effort to ensure RPC calls are properly cached on caching servers such as Squid.

Representational state transfer (REST)

REST is an architectural style enforcing a client/server model where the client acts on a set of resources managed by the server. The server provides a representation of resources and actions that can either manipulate or get a new representation of resources. All communication must be stateless and cacheable.

There are four qualities of a RESTful interface:

Identify resources (URI in HTTP) - use the same URI regardless of any operation.
Change with representations (Verbs in HTTP) - use verbs, headers, and body.
Self-descriptive error message (status response in HTTP) - Use status codes, don't reinvent the wheel.
HATEOAS (HTML interface for HTTP) - your web service should be fully accessible in a browser.

Sample REST calls:

GET /someresources/anId

PUT /someresources/anId
{"anotherdata": "another value"}

REST is focused on exposing data. It minimizes the coupling between client/server and is often used for public HTTP APIs. REST uses a more generic and uniform method of exposing resources through URIs, representation through headers, and actions through verbs such as GET, POST, PUT, DELETE, and PATCH. Being stateless, REST is great for horizontal scaling and partitioning.

Disadvantage(s): REST

With REST being focused on exposing data, it might not be a good fit if resources are not naturally organized or accessed in a simple hierarchy. For example, returning all updated records from the past hour matching a particular set of events is not easily expressed as a path. With REST, it is likely to be implemented with a combination of URI path, query parameters, and possibly the request body.
REST typically relies on a few verbs (GET, POST, PUT, DELETE, and PATCH) which sometimes doesn't fit your use case. For example, moving expired documents to the archive folder might not cleanly fit within these verbs.
Fetching complicated resources with nested hierarchies requires multiple round trips between the client and server to render single views, e.g. fetching content of a blog entry and the comments on that entry. For mobile applications operating in variable network conditions, these multiple roundtrips are highly undesirable.
Over time, more fields might be added to an API response and older clients will receive all new data fields, even those that they do not need, as a result, it bloats the payload size and leads to larger latencies.

RPC and REST calls comparison

Operation	RPC	REST
Signup	POST /signup	POST /persons
Resign	POST /resign { "personid": "1234" }	DELETE /persons/1234
Read a person	GET /readPerson?personid=1234	GET /persons/1234
Read a person’s items list	GET /readUsersItemsList?personid=1234	GET /persons/1234/items
Add an item to a person’s items	POST /addItemToUsersItemsList { "personid": "1234"; "itemid": "456" }	POST /persons/1234/items { "itemid": "456" }
Update an item	POST /modifyItem { "itemid": "456"; "key": "value" }	PUT /items/456 { "key": "value" }
Delete an item	POST /removeItem { "itemid": "456" }	DELETE /items/456

Source: Do you really know why you prefer REST over RPC

Source(s) and further reading: REST and RPC

Security

This section could use some updates. Consider contributing!

Security is a broad topic. Unless you have considerable experience, a security background, or are applying for a position that requires knowledge of security, you probably won't need to know more than the basics:

Encrypt in transit and at rest.
Sanitize all user inputs or any input parameters exposed to user to prevent XSS and SQL injection.
Use parameterized queries to prevent SQL injection.
Use the principle of least privilege.

Source(s) and further reading

Appendix

You'll sometimes be asked to do 'back-of-the-envelope' estimates. For example, you might need to determine how long it will take to generate 100 image thumbnails from disk or how much memory a data structure will take. The Powers of two table and Latency numbers every programmer should know are handy references.

Powers of two table

Power           Exact Value         Approx Value        Bytes
---------------------------------------------------------------
7                             128
8                             256
10                           1024   1 thousand           1 KB
16                         65,536                       64 KB
20                      1,048,576   1 million            1 MB
30                  1,073,741,824   1 billion            1 GB
32                  4,294,967,296                        4 GB
40              1,099,511,627,776   1 trillion           1 TB

Source(s) and further reading

Powers of two

Latency numbers every programmer should know

Latency Comparison Numbers
--------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy            10,000   ns       10 us
Send 1 KB bytes over 1 Gbps network     10,000   ns       10 us
Read 4 KB randomly from SSD*           150,000   ns      150 us          ~1GB/sec SSD
Read 1 MB sequentially from memory     250,000   ns      250 us
Round trip within same datacenter      500,000   ns      500 us
Read 1 MB sequentially from SSD*     1,000,000   ns    1,000 us    1 ms  ~1GB/sec SSD, 4X memory
HDD seek                            10,000,000   ns   10,000 us   10 ms  20x datacenter roundtrip
Read 1 MB sequentially from 1 Gbps  10,000,000   ns   10,000 us   10 ms  40x memory, 10X SSD
Read 1 MB sequentially from HDD     30,000,000   ns   30,000 us   30 ms 120x memory, 30X SSD
Send packet CA->Netherlands->CA    150,000,000   ns  150,000 us  150 ms

Notes
-----
1 ns = 10^-9 seconds
1 us = 10^-6 seconds = 1,000 ns
1 ms = 10^-3 seconds = 1,000 us = 1,000,000 ns

Handy metrics based on numbers above:

Read sequentially from HDD at 30 MB/s
Read sequentially from 1 Gbps Ethernet at 100 MB/s
Read sequentially from SSD at 1 GB/s
Read sequentially from main memory at 4 GB/s
6-7 world-wide round trips per second
2,000 round trips per second within a data center

Latency numbers visualized

Source(s) and further reading

Additional system design interview questions

Common system design interview questions, with links to resources on how to solve each.

Question	Reference(s)
Design a file sync service like Dropbox	youtube.com
Design a search engine like Google	queue.acm.org stackexchange.com ardendertat.com stanford.edu
Design a scalable web crawler like Google	quora.com
Design Google docs	code.google.com neil.fraser.name
Design a key-value store like Redis	codecapsule.com allthingsdistributed.com
Design a cache system like Memcached	slideshare.net
Design a recommendation system like Amazon's	hulu.com ijcai13.org
Design a tinyurl system like Bitly	n00tc0d3r.blogspot.com
Design a chat app like WhatsApp	highscalability.com
Design a picture sharing system like Instagram	highscalability.com highscalability.com
Design the Facebook news feed function	quora.com quora.com slideshare.net
Design the Facebook timeline function	facebook.com highscalability.com
Design the Facebook chat function	erlang-factory.com facebook.com
Design a graph search function like Facebook's	facebook.com facebook.com facebook.com
Design a content delivery network like CloudFlare	figshare.com
Design a trending topic system like Twitter's	michael-noll.com snikolov .wordpress.com
Design a random ID generation system	blog.twitter.com github.com
Return the top k requests during a time interval	cs.ucsb.edu wpi.edu
Design a system that serves data from multiple data centers	highscalability.com
Design an online multiplayer card game	indieflashblog.com buildnewgames.com
Design a garbage collection system	stuffwithstuff.com washington.edu
Design an API rate limiter	https://stripe.com/blog/
Design a Stock Exchange (like NASDAQ or Binance)	Jane Street Golang Implementation Go Implementation
Add a system design question	Contribute

Real world architectures

Articles on how real world systems are designed.

Source: Twitter timelines at scale

Don't focus on nitty gritty details for the following articles, instead:

Identify shared principles, common technologies, and patterns within these articles
Study what problems are solved by each component, where it works, where it doesn't
Review the lessons learned

Type	System	Reference(s)
Data processing	MapReduce - Distributed data processing from Google	research.google.com
Data processing	Spark - Distributed data processing from Databricks	slideshare.net
Data processing	Storm - Distributed data processing from Twitter	slideshare.net

Data store	Bigtable - Distributed column-oriented database from Google	harvard.edu
Data store	HBase - Open source implementation of Bigtable	slideshare.net
Data store	Cassandra - Distributed column-oriented database from Facebook	slideshare.net
Data store	DynamoDB - Document-oriented database from Amazon	harvard.edu
Data store	MongoDB - Document-oriented database	slideshare.net
Data store	Spanner - Globally-distributed database from Google	research.google.com
Data store	Memcached - Distributed memory caching system	slideshare.net
Data store	Redis - Distributed memory caching system with persistence and value types	slideshare.net

File system	Google File System (GFS) - Distributed file system	research.google.com
File system	Hadoop File System (HDFS) - Open source implementation of GFS	apache.org

Misc	Chubby - Lock service for loosely-coupled distributed systems from Google	research.google.com
Misc	Dapper - Distributed systems tracing infrastructure	research.google.com
Misc	Kafka - Pub/sub message queue from LinkedIn	slideshare.net
Misc	Zookeeper - Centralized infrastructure and services enabling synchronization	slideshare.net
	Add an architecture	Contribute

Company architectures

Company	Reference(s)
Amazon	Amazon architecture
Cinchcast	Producing 1,500 hours of audio every day
DataSift	Realtime datamining At 120,000 tweets per second
Dropbox	How we've scaled Dropbox
ESPN	Operating At 100,000 duh nuh nuhs per second
Google	Google architecture
Instagram	14 million users, terabytes of photos What powers Instagram
Justin.tv	Justin.Tv's live video broadcasting architecture
Facebook	Scaling memcached at Facebook TAO: Facebook’s distributed data store for the social graph Facebook’s photo storage How Facebook Live Streams To 800,000 Simultaneous Viewers
Flickr	Flickr architecture
Mailbox	From 0 to one million users in 6 weeks
Netflix	A 360 Degree View Of The Entire Netflix Stack Netflix: What Happens When You Press Play?
Pinterest	From 0 To 10s of billions of page views a month 18 million visitors, 10x growth, 12 employees
Playfish	50 million monthly users and growing
PlentyOfFish	PlentyOfFish architecture
Salesforce	How they handle 1.3 billion transactions a day
Stack Overflow	Stack Overflow architecture
TripAdvisor	40M visitors, 200M dynamic page views, 30TB data
Tumblr	15 billion page views a month
Twitter	Making Twitter 10000 percent faster Storing 250 million tweets a day using MySQL 150M active users, 300K QPS, a 22 MB/S firehose Timelines at scale Big and small data at Twitter Operations at Twitter: scaling beyond 100 million users How Twitter Handles 3,000 Images Per Second
Uber	How Uber scales their real-time market platform Lessons Learned From Scaling Uber To 2000 Engineers, 1000 Services, And 8000 Git Repositories
WhatsApp	The WhatsApp architecture Facebook bought for $19 billion
YouTube	YouTube scalability YouTube architecture

Company engineering blogs

Architectures for companies you are interviewing with.

Questions you encounter might be from the same domain.

Source(s) and further reading

Looking to add a blog? To avoid duplicating work, consider adding your company blog to the following repo:

kilimchoi/engineering-blogs

Under development

Interested in adding a section or helping complete one in-progress? Contribute!

Distributed computing with MapReduce
Consistent hashing
Scatter gather
Contribute

Credits

Credits and sources are provided throughout this repo.

Special thanks to:

Contact info

Feel free to contact me to discuss any issues, questions, or comments.

My contact info can be found on my GitHub page.

License

I am providing code and resources in this repository to you under an open source license. Because this is my personal repository, the license you receive to my code and resources is from me and not my employer (Facebook).

Copyright 2017 Donne Martin

Creative Commons Attribution 4.0 International License (CC BY 4.0)

http://creativecommons.org/licenses/by/4.0/

Robbyant/lingbot-map

A feed-forward 3D foundation model for reconstructing scenes from streaming data

LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstruction

Robbyant Team

https://github.com/user-attachments/assets/fe39e095-af2c-4ec9-b68d-a8ba97e505ab

🗺️ Meet LingBot-Map! We've built a feed-forward 3D foundation model for streaming 3D reconstruction! 🏗️🌍

LingBot-Map has focused on:

Geometric Context Transformer: Architecturally unifies coordinate grounding, dense geometric cues, and long-range drift correction within a single streaming framework through anchor context, pose-reference window, and trajectory memory.
High-Efficiency Streaming Inference: A feed-forward architecture with paged KV cache attention, enabling stable inference at ~20 FPS on 518×378 resolution over long sequences exceeding 10,000 frames.
State-of-the-Art Reconstruction: Superior performance on diverse benchmarks compared to both existing streaming and iterative optimization-based approaches.

📑 Table of Contents

Click to expand

📰 News

2026-06-28 — Fixed an SDPA KV cache bug. The SDPA backend now performs better on long sequences. We still recommend the FlashInfer backend for the best performance.
2026-05-25 — 📊 Evaluation benchmark released. We released the evaluation scripts for KITTI and Oxford Spires — see benchmark/ for the pipeline, and run preprocess/oxford.py to prepare Oxford Spires data before evaluation.
2026-04-29 — 📹 Long-video demo released. We released a very-long-video example (~25 000 frames, 13-minute indoor walkthrough) rendered with the offline pipeline — see Worked Example for the command, flag rationale, and rendered output.
2026-04-27 — 🚀 LingBot-Map accelerated. Pull the latest main and run python demo.py --compile ... or python gct_profile.py --backend flashinfer --dtype bf16 --compile to verify on your hardware.
2026-04-24 — Fixed a FlashInfer KV cache bug where --keyframe_interval > 1 silently cached non-keyframes. You should now see better pose and reconstruction quality when running with more than 320 frames.

📋 TODO

✅ Release evaluation benchmark
- ✅ Oxford Spires dataset
- ✅ KITTI dataset
- ✅ VBR dataset
- ✅ Droid-W dataset
- ✅ TUM-D dataset
- ✅ 7-scenes dataset
- ✅ ETH3D dataset
- ✅ Tanks and Temples dataset
- ✅ NRGBD dataset
✅ Release demo scripts
- ✅ Indoor long-video demo (Featured indoor walkthrough)
- ✅ Outdoor long-video demo
- ✅ LingBot-World demo (Worked example)
- ✅ Aerial long-video demo

⚙️ Installation

1. Create conda environment

conda create -n lingbot-map python=3.10 -y
conda activate lingbot-map

2. Install PyTorch (CUDA 12.8)

pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128

PyTorch 2.8.0 is the recommended version because NVIDIA Kaolin (required by the batch rendering pipeline) has prebuilt wheels for torch-2.8.0_cu128. If you only need demo.py you may use a newer PyTorch, but the batch renderer then requires building Kaolin from source. For other CUDA versions, see PyTorch Get Started.

3. Install lingbot-map

pip install -e .

4. Install FlashInfer (recommended)

FlashInfer provides paged KV cache attention for efficient streaming inference. It is a pure-Python package that JIT-compiles CUDA kernels on first use, so a single wheel works across CUDA/PyTorch versions:

pip install --index-url https://pypi.org/simple flashinfer-python

--index-url https://pypi.org/simple is only needed if your default pip index is an internal mirror that doesn't have flashinfer-python. (Optional) For faster first-use, you can additionally install a CUDA-specific JIT cache: pip install flashinfer-jit-cache -f https://flashinfer.ai/whl/cu128/flashinfer-jit-cache/. See FlashInfer installation for details. If FlashInfer is not installed, the model falls back to SDPA (PyTorch native attention) via --use_sdpa.

5. Visualization dependencies (optional)

pip install -e ".[vis]"

📦 Model Download

Model Name	Huggingface Repository	ModelScope Repository	Description
lingbot-map-long	robbyant/lingbot-map	Robbyant/lingbot-map	Better suited for long sequences and large scale scenes.
lingbot-map	robbyant/lingbot-map	Robbyant/lingbot-map	Balanced checkpoint (used in paper, benchmark and offline demo) — trade off all-around performance across short and long sequences.
lingbot-map-stage1	robbyant/lingbot-map	Robbyant/lingbot-map	Stage-1 training checkpoint of lingbot-map — can be loaded into the VGGT model for bidirectional inference (c2w).

🚧 Coming soon: we're training an stronger model that supports longer sequences — stay tuned.

🚀 Quick Start

After installation, run your first scene with one command:

python demo.py --model_path /path/to/lingbot-map.pt \
    --image_folder example/courthouse --mask_sky

This launches an interactive viser viewer at http://localhost:8080. See Interactive Demo below for the full set of scenes and flags, or jump to Offline Rendering Pipeline for long-sequence batch rendering.

🎬 Interactive Demo (`demo.py`)

Run demo.py for interactive 3D visualization via a browser-based viser viewer (default http://localhost:8080).

Try the Example Scenes

We provide three example scenes in example/ that you can run out of the box:

# courthouse scene
python demo.py --model_path /path/to/lingbot-map.pt \
    --image_folder example/courthouse --mask_sky

https://github.com/user-attachments/assets/aa10f7ab-8024-43c7-92f8-d56159ec85c8

# University scene
python demo.py --model_path /path/to/lingbot-map.pt \
    --image_folder example/university --mask_sky

https://github.com/user-attachments/assets/212a1744-6ff5-4ccf-9bd4-728608248b57

# Loop scene (loop closure trajectory)
python demo.py --model_path /path/to/lingbot-map.pt \
    --image_folder example/loop

https://github.com/user-attachments/assets/5ae0a292-b081-40c6-838c-b7c1a0538d75

🎯 Featured: indoor walkthrough (~25 000 frames, 13 minutes)

Sequence is too long for the interactive viser viewer — this clip was rendered with the Offline Rendering Pipeline. See that section for the full command.

We will provide more examples in the follow-up.

Dynamic Demo (From Droid-W)

Dataset: Download the demo sequences from robbyant/lingbot-map-demo on Hugging Face.

Example run on the dynamic sequence from the dataset above (sky masking on, 4 camera optimization iterations, keyframe every 2 frames):

Run the dynamic sequence with sky masking, 4 camera optimization iterations, and an input stride of 2:

python demo.py \
    --image_folder /path/to/dynamic\
    --model_path ../../Lingbot-Map/lingbot-map.pt \
    --camera_num_iterations 4 \
    --mask_sky \
    --stride 2

https://github.com/user-attachments/assets/567b6e9b-1cbf-402a-96be-9bab70715ec3

Streaming with Keyframe Interval

Use --keyframe_interval to reduce KV cache memory by only keeping every N-th frame as a keyframe. Non-keyframe frames still produce predictions but are not stored in the cache. This is useful for long sequences which exceed 320 frames (We train with video RoPE on 320 views, so performance degrades when the KV cache stores more than 320 views. Using a keyframe strategy allows inference over longer sequences.). In demo.py, the keyframe interval is calculated automatically.

Note on inference range. Our method does not perform state resetting by default, so the maximum inference range is bounded by the longest distance seen during training on the dataset. Beyond that distance, state resetting becomes necessary. If you observe pose collapse, switch to windowed mode (--mode windowed) — in most cases tuning --keyframe_interval alone is enough and the rest of the windowed parameters can stay at their defaults.

Windowed Inference (for long sequences, >3000 frames)

python demo.py --model_path /path/to/lingbot-map.pt \
    --video_path video.mp4 --fps 10 \
    --mode windowed --window_size 128 --overlap_keyframes 16 --keyframe_interval 2

Sky Masking

Sky masking uses an ONNX sky segmentation model to filter out sky points from the reconstructed point cloud, which improves visualization quality for outdoor scenes.

Setup:

# Install onnxruntime (required)
pip install onnxruntime        # CPU
# or
pip install onnxruntime-gpu    # GPU (faster for large image sets)

The sky segmentation model (skyseg.onnx) will be automatically downloaded from HuggingFace on first use. If the download fails or does not produce a regular file, sky masking stops with a RuntimeError that reports the model path, download URL, cause, and manual setup guidance; it never silently continues without masking. For manual recovery, download the model as skyseg.onnx in the directory from which you run the command, because root demo.py resolves its default model path relative to the current working directory:

wget -O skyseg.onnx https://huggingface.co/JianyuanWang/skyseg/resolve/main/skyseg.onnx
python demo.py --model_path /path/to/checkpoint.pt \
    --image_folder /path/to/images/ --mask_sky

Usage:

python demo.py --model_path /path/to/checkpoint.pt \
    --image_folder /path/to/images/ --mask_sky

Sky masks are cached in <image_folder>_sky_masks/ so subsequent runs skip regeneration. You can also specify a custom cache directory with --sky_mask_dir, or save side-by-side mask visualizations with --sky_mask_visualization_dir:

python demo.py --model_path /path/to/checkpoint.pt \
    --image_folder /path/to/images/ --mask_sky \
    --sky_mask_dir /path/to/cached_masks/ \
    --sky_mask_visualization_dir /path/to/mask_viz/

Visualization Options

Argument	Default	Description
`--port`	`8080`	Viser viewer port
`--conf_threshold`	`1.5`	Visibility threshold for filtering low-confidence points
`--point_size`	`0.00001`	Point cloud point size
`--downsample_factor`	`10`	Spatial downsampling for point cloud display

Performance & Memory

Without FlashInfer (SDPA fallback)

python demo.py --model_path /path/to/checkpoint.pt \
    --image_folder /path/to/images/ --use_sdpa

Running on Limited GPU Memory

If you run into out-of-memory issues, try one (or both) of the following:

--offload_to_cpu — offload per-frame predictions to CPU during inference (on by default; use --no-offload_to_cpu only if you have memory to spare).
--num_scale_frames 2 — reduce the number of bidirectional scale frames from the default 8 down to 2, which shrinks the activation peak of the initial scale phase.

Faster Inference

Lower the number of iterative refinement steps in the camera head to trade a small amount of pose accuracy for wall-clock speed:

python demo.py --model_path /path/to/checkpoint.pt \
    --image_folder /path/to/images/ --camera_num_iterations 1

--camera_num_iterations defaults to 4; setting it to 1 skips three refinement passes in the camera head (and shrinks its KV cache by 4×).

🎥 Offline Rendering Pipeline (`demo_render/batch_demo.py`)

Use this pipeline when your sequence is too long for the interactive viser viewer — for example, the indoor walkthrough featured above. demo_render/batch_demo.py is the all-in-one offline entry point: feed it a video or a folder of images and it will run model inference and produce a headless point-cloud flythrough MP4 in a single command. It shares the same PyTorch / FlashInfer / checkpoint stack as demo.py.

For those constrained by limited VRAM or GPU usage, you may also refer to the implementation at: https://github.com/ureeey/lingbot-map-rtx4060-8g/commit/eeee84a89cc97c1e39b736b46df4ee315275700b

Install (extends the main install)

1. Rendering Python dependencies

pip install -e ".[vis,render]"

render pulls in open3d>=0.19 and pyyaml (the core numpy<2 constraint comes from the base lingbot-map install). Sky masking in this pipeline uses onnxruntime-gpu for batched segmentation; install it if you don't already have the CPU onnxruntime:

pip install onnxruntime-gpu

The offline pipeline has the same first-use model download and fatal failure behavior described in Sky Masking. Use --skyseg_model_path /absolute/path/to/skyseg.onnx with demo_render/batch_demo.py; for standalone demo_render/rgbd_scan_render.py, use --sky_model /absolute/path/to/skyseg.onnx on the command line or set preprocess.sky_model in YAML.

2. Kaolin — matches the PyTorch 2.8.0 + CUDA 12.8 recommended above:

pip install --index-url https://pypi.org/simple \
    kaolin -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.8.0_cu128.html

--index-url https://pypi.org/simple bypasses any internal mirror that might otherwise serve the PyPI placeholder wheel (which raises ImportError on import). NVIDIA Kaolin does not publish prebuilt wheels for PyTorch 2.9.x — if you're on 2.9 for other reasons, build Kaolin from source (pip install --no-build-isolation git+https://github.com/NVIDIAGameWorks/kaolin.git, needs local CUDA toolkit). For other torch/CUDA combinations see NVIDIA Kaolin installation.

3. ffmpeg

sudo apt install ffmpeg    # or: brew install ffmpeg

4. CUDA extensions (required before first run)

cd demo_render/render_cuda_ext && python setup.py build_ext --inplace && cd ../..

This builds voxel_morton_ext and frustum_cull_ext in place — both are imported by rgbd_render for GPU voxelization and frustum culling.

Worked Example — long indoor walkthrough (~25 000 frames, 13 minutes)

Dataset: Download the example video from robbyant/lingbot-map-demo on Hugging Face.

    python demo_render/batch_demo.py \
    --video_path /data/demo_videos/indoor_travel.MP4 \
    --output_folder /data/outputs/indoor_travel/ \
    --model_path /path/to/lingbot-map.pt \
    --config demo_render/config/indoor.yaml \
    --mode windowed --window_size 128 \
    --keyframe_interval 10 --overlap_keyframes 8 \
    --sky_mask_dir /data/outputs/sky_masks \
    --sky_mask_visualization_dir /data/outputs/sky_mask_viz \
    --camera_vis default --keyframes_only_points \
    --frame_tag --frame_tag_position top_right \
    --save_predictions

Flag-by-flag rationale:

Flag	Why it's there
`--mode windowed --window_size 128`	Sliding-window inference is required once the sequence exceeds the ~320-frame RoPE training range; each window resets the KV cache. `window_size` counts KV-cache slots, not actual frames — the first `num_scale_frames` (=8) slots hold the scale frames and the remaining `128 − 8 = 120` slots hold keyframes. With `keyframe_interval = 13`, one window therefore covers `8 + 120 × 13 = 1568` actual frames.
`--keyframe_interval 10`	Cache only every 10th frame as a keyframe. Non-keyframes still emit per-frame predictions but don't grow the KV cache
`--overlap_keyframes 8`	Adjacent windows share 8 keyframes of context, resolved internally to `max(num_scale_frames, 8 × keyframe_interval) = 8 × 13 = 104` actual frames of overlap. Recommended whenever `keyframe_interval > 1`, to keep cross-window pose alignment stable.
`--config demo_render/config/indoor.yaml`	Seed render/scene/camera/overlay defaults from the indoor preset (short depth, tighter follow cam). Any CLI flag the user explicitly passes still overrides the YAML value.
`--sky_mask_dir` / `--sky_mask_visualization_dir`	Persist sky masks and their side-by-side visualizations to disk so subsequent reruns reuse them instead of re-running ONNX segmentation. (The render pipeline only consumes them when sky masking is enabled — by the YAML preset or by `--mask_sky`.)
`--camera_vis default`	Overlay the trajectory trail + recent-frame points on the rendered video.
`--keyframes_only_points`	Only unproject keyframe depth into the point cloud; non-keyframes still contribute their pose to the trajectory/frustum overlay. Keeps the cloud sparse for very long sequences.
`--frame_tag --frame_tag_position top_right`	Stamp a `<i> / <N> Frames` counter in the top-right corner of the MP4.
`--save_predictions`	Persist per-frame NPZs alongside the MP4. Useful for inspection or for re-rendering with different camera/overlay settings later.

Replacing keyframe_interval = 10 with image_stride = 10 speeds up rendering. Then, uncomment the camera follow section in demo_render/config/indoor.yaml and set the birdeye's ranges to [2000, 2500] to reproduce the indoor fly-through effect shown in the demo:

https://github.com/user-attachments/assets/21b444ea-e6b6-48f0-8b34-3acad41166ac

Worked Example — outdoor drive scene

Dataset: Download the example video from robbyant/lingbot-map-demo on Hugging Face.

    python demo_render/batch_demo.py \
    --video_path /data/demo_videos/drive_frames.mp4 \
    --output_folder /data/outputs/drive/ \
    --model_path /path/to/lingbot-map.pt \
    --config demo_render/config/outdoor_drive.yaml \
    --mode windowed --window_size 128 \
    --max_non_keyframe_gap 100 --overlap_keyframes 8 \
    --image_stride 1 \
    --sky_mask_dir /data/outputs/sky_masks \
    --sky_mask_visualization_dir /data/outputs/sky_mask_viz \
    --camera_vis default --keyframes_only_points \
    --frame_tag --frame_tag_position top_right \
    --save_predictions

What differs from the indoor walkthrough above:

Flag	Why it's there
`--config demo_render/config/outdoor_drive.yaml`	Seed defaults from the outdoor preset: sky masking enabled, deeper render range (`max_depth: 250`), and a follow cam tuned for vehicle trajectories with a final birdeye reveal.
`--image_stride 1`	Use every video frame. Increase it to subsample long or high-FPS drive footage.
`--max_non_keyframe_gap 100`	Upper bound on consecutive non-keyframes before a keyframe is forced. Only active with flow-based keyframe selection (`--flow_threshold > 0`); in the default fixed-interval mode it has no effect.

The remaining flags (--mode windowed --window_size 128, --overlap_keyframes 8, sky-mask caching, overlays, --save_predictions) carry over unchanged from the indoor example — see the flag-by-flag table above.

Worked Example — LingBot-World scenes

Reconstruct videos generated by LingBot-World, our world model — the same pipeline works on generated footage out of the box.

Dataset: Download the example videos (lingbo_world_frames.mp4, lingbo_world2_frames.mp4) from robbyant/lingbot-map-demo on Hugging Face.

    python demo_render/batch_demo.py \
    --video_path /data/demo_videos/lingbo_world_frames.mp4 \
    --output_folder /data/outputs/lingbo_world/ \
    --model_path /path/to/lingbot-map.pt \
    --config demo_render/config/outdoor_drive.yaml \
    --mode windowed --window_size 128 \
    --max_non_keyframe_gap 100 --overlap_keyframes 8 \
    --image_stride 1 \
    --sky_mask_dir /data/outputs/sky_masks \
    --sky_mask_visualization_dir /data/outputs/sky_mask_viz \
    --camera_vis default --keyframes_only_points \
    --frame_tag --frame_tag_position top_right \
    --save_predictions

For the second clip, run the same command with --video_path /data/demo_videos/lingbo_world2_frames.mp4 --output_folder /data/outputs/lingbo_world2/ (and separate --sky_mask_dir / --sky_mask_visualization_dir folders if you want to keep the cached masks apart).

All flags are identical to the outdoor drive scene above — only the input video and output folder change. See the drive scene and indoor walkthrough tables for the flag-by-flag rationale.

Camera Path (YAML)

The virtual camera path is described by the camera.segments list in the YAML preset passed via --config. Edit the YAML to design your own shot — no need to touch CLI flags.

Built-in presets live in demo_render/config/: default.yaml, indoor.yaml, outdoor_drive.yaml. Copy one and edit the camera: block.

YAML structure

camera:
  fov: 60.0          # camera field of view in degrees
  transition: 30     # frames blended between adjacent segments
  segments:
    - mode: follow            # chase cam following the input trajectory
      frames: [0, 1500]       # rendered-frame range this segment covers (-1 = end)
      back_offset: 0.3        # how far behind the input camera (fraction of scene scale)
      up_offset: 0.08         # vertical lift above the input camera
      look_offset: 0.4        # how far ahead the lookat target points
      smooth_window: 30       # trajectory smoothing window in frames
    - mode: birdeye           # rise up for a top-down reveal of the whole scene
      frames: [1500, 1800]
      reveal_height_mult: 2.5 # birdeye height = scene scale × this factor
    - mode: follow            # drop back into chase cam
      frames: [1800, -1]
      back_offset: 0.3
      up_offset: 0.08
      look_offset: 0.4

transition controls how many frames are blended between adjacent segments; frames: [0, -1] means "the whole sequence".

Available modes

`mode`	Behavior	Tunable fields
`follow`	Chase cam tracks the input trajectory with smooth offsets. The most cinematic option for walkthroughs.	`back_offset`, `up_offset`, `look_offset`, `smooth_window`, `scale_frames`
`birdeye`	Top-down reveal of the whole scene. Useful for hero / overview shots.	`reveal_height_mult`
`static`	Fixed eye + lookat, auto-derived from the segment's start frame.	—
`pivot`	Fixed eye, lookat sweeps along the trajectory.	—

Single-shot YAML examples

Pure follow (most common):

camera:
  fov: 60.0
  segments:
    - mode: follow
      frames: [0, -1]
      back_offset: 0.3
      up_offset: 0.08
      look_offset: 0.4
      smooth_window: 30

Full birdeye (good for overview / hero shots):

camera:
  fov: 60.0
  segments:
    - mode: birdeye
      frames: [0, -1]
      reveal_height_mult: 2.5

Follow with birdeye inserts: just list multiple segments in order under segments: — adjacent segments are interpolated using transition frames.

Caveat: when --config loads a YAML preset, passing any segment-shaping CLI flag (--camera_mode, --back_offset, --up_offset, --look_offset, --smooth_window, --follow_scale_frames, --birdeye_start, --birdeye_duration, --reveal_height_mult) discards the YAML's segments and rebuilds the camera path from those flags instead. To stay fully YAML-driven, don't pass any of them on the command line.

Output files

For a given output name (e.g. <scene> or <video_name>):

File	Description
`<name>_pointcloud.mp4`	Rendered point-cloud flythrough
`<name>_pointcloud_rgb.mp4`	Original RGB frames encoded as video
`<name>_pointcloud_config.yaml`	Full config snapshot of this run
`batch_results.json`	Per-scene success / duration summary

📜 License

This project is released under the Apache License 2.0. See LICENSE file for details.

📖 Citation

@article{chen2026geometric,
  title={Geometric Context Transformer for Streaming 3D Reconstruction},
  author={Chen, Lin-Zhuo and Gao, Jian and Chen, Yihang and Cheng, Ka Leong and Sun, Yipengjing and Hu, Liangxiao and Xue, Nan and Zhu, Xing and Shen, Yujun and Yao, Yao and Xu, Yinghao},
  journal={arXiv preprint arXiv:2604.14141},
  year={2026}
}

✨ Acknowledgments

We thank Shangzhan Zhang, Jianyuan Wang, Yudong Jin, Christian Rupprecht, and Xun Cao for their helpful discussions and support.

This work builds upon several excellent open-source projects:

unclecode/crawl4ai

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper.

🚀 Crawl4AI Cloud API — Closed Beta (Launching Soon)

Reliable, large-scale web extraction, now built to be drastically more cost-effective than any of the existing solutions.

👉 Apply here for early access
We’ll be onboarding in phases and working closely with early users. Limited slots.

Crawl4AI turns the web into clean, LLM ready Markdown for RAG, agents, and data pipelines. Fast, controllable, battle tested by a 50k+ star community.

✨ Check out latest update v0.9.2

✨ New in v0.9.2: Maintenance patch release. Fixes a MemoryAdaptiveDispatcher task/page leak when a streaming crawl is closed, Docker Playground "Advanced Config" and Monitor WebSocket auth, Playwright headless-shell packaging, and GPU (ENABLE_GPU=true) Docker builds. Release notes →

✨ Recent v0.9.0: Major secure-by-default release of the Docker API server. Auth is on by default, the server binds loopback unless given a token, and the request body is now an untrusted trust boundary. Release notes →

✨ Recent v0.8.7: Security-hardening release. Fixes critical Docker API vulnerabilities (RCE, SSRF, auth bypass, file write, XSS, hardcoded JWT secret), adds DomainMapper, and ships scraping, deep-crawl, and LLM fixes. Release notes →

✨ Previous v0.8.0: Crash Recovery & Prefetch Mode! Deep crawl crash recovery with resume_state and on_state_change callbacks for long-running crawls. New prefetch=True mode for 5-10x faster URL discovery. Release notes →

✨ Previous v0.7.8: Stability & Bug Fix Release! 11 bug fixes addressing Docker API issues, LLM extraction improvements, URL handling fixes, and dependency updates. Release notes →

🤓 My Personal Story

I grew up on an Amstrad, thanks to my dad, and never stopped building. In grad school I specialized in NLP and built crawlers for research. That’s where I learned how much extraction matters.

In 2023, I needed web-to-Markdown. The “open source” option wanted an account, API token, and $16, and still under-delivered. I went turbo anger mode, built Crawl4AI in days, and it went viral. Now it’s the most-starred crawler on GitHub.

I made it open source for availability, anyone can use it without a gate. Now I’m building the platform for affordability, anyone can run serious crawls without breaking the bank. If that resonates, join in, send feedback, or just crawl something amazing.

Why developers pick Crawl4AI

LLM ready output, smart Markdown with headings, tables, code, citation hints
Fast in practice, async browser pool, caching, minimal hops
Full control, sessions, proxies, cookies, user scripts, hooks
Adaptive intelligence, learns site patterns, explores only what matters
Deploy anywhere, zero keys, CLI and Docker, cloud friendly

🚀 Quick Start

Install Crawl4AI:

# Install the package
pip install -U crawl4ai

# For pre release versions
pip install crawl4ai --pre

# Run post-installation setup
crawl4ai-setup

# Verify your installation
crawl4ai-doctor

If you encounter any browser-related issues, you can install them manually:

python -m playwright install --with-deps chromium

Run a simple web crawl with Python:

import asyncio
from crawl4ai import *

async def main():
    async with AsyncWebCrawler() as crawler:
        result = await crawler.arun(
            url="https://www.nbcnews.com/business",
        )
        print(result.markdown)

if __name__ == "__main__":
    asyncio.run(main())

Or use the new command-line interface:

# Basic crawl with markdown output
crwl https://www.nbcnews.com/business -o markdown

# Deep crawl with BFS strategy, max 10 pages
crwl https://docs.crawl4ai.com --deep-crawl bfs --max-pages 10

# Use LLM extraction with a specific question
crwl https://www.example.com/products -q "Extract all product prices"

💖 Support Crawl4AI

🎉 Sponsorship Program Now Open! After powering 51K+ developers and 1 year of growth, Crawl4AI is launching dedicated support for startups and enterprises. Be among the first 50 Founding Sponsors for permanent recognition in our Hall of Fame.

Crawl4AI is the #1 trending open-source web crawler on GitHub. Your support keeps it independent, innovative, and free for the community — while giving you direct access to premium benefits.

🤝 Sponsorship Tiers

🌱 Believer ($5/mo) — Join the movement for data democratization
🚀 Builder ($50/mo) — Priority support & early access to features
💼 Growing Team ($500/mo) — Bi-weekly syncs & optimization help
🏢 Data Infrastructure Partner ($2000/mo) — Full partnership with dedicated support
Custom arrangements available - see SPONSORS.md for details & contact

Why sponsor?
No rate-limited APIs. No lock-in. Build and own your data pipeline with direct guidance from the creator of Crawl4AI.

See All Tiers & Benefits →

✨ Features

📝 Markdown Generation

🧹 Clean Markdown: Generates clean, structured Markdown with accurate formatting.
🎯 Fit Markdown: Heuristic-based filtering to remove noise and irrelevant parts for AI-friendly processing.
🔗 Citations and References: Converts page links into a numbered reference list with clean citations.
🛠️ Custom Strategies: Users can create their own Markdown generation strategies tailored to specific needs.
📚 BM25 Algorithm: Employs BM25-based filtering for extracting core information and removing irrelevant content.

📊 Structured Data Extraction

🤖 LLM-Driven Extraction: Supports all LLMs (open-source and proprietary) for structured data extraction.
🧱 Chunking Strategies: Implements chunking (topic-based, regex, sentence-level) for targeted content processing.
🌌 Cosine Similarity: Find relevant content chunks based on user queries for semantic extraction.
🔎 CSS-Based Extraction: Fast schema-based data extraction using XPath and CSS selectors.
🔧 Schema Definition: Define custom schemas for extracting structured JSON from repetitive patterns.

🌐 Browser Integration

🖥️ Managed Browser: Use user-owned browsers with full control, avoiding bot detection.
🔄 Remote Browser Control: Connect to Chrome Developer Tools Protocol for remote, large-scale data extraction.
👤 Browser Profiler: Create and manage persistent profiles with saved authentication states, cookies, and settings.
🔒 Session Management: Preserve browser states and reuse them for multi-step crawling.
🧩 Proxy Support: Seamlessly connect to proxies with authentication for secure access.
⚙️ Full Browser Control: Modify headers, cookies, user agents, and more for tailored crawling setups.
🌍 Multi-Browser Support: Compatible with Chromium, Firefox, and WebKit.
📐 Dynamic Viewport Adjustment: Automatically adjusts the browser viewport to match page content, ensuring complete rendering and capturing of all elements.

🔎 Crawling & Scraping

🖼️ Media Support: Extract images, audio, videos, and responsive image formats like srcset and picture.
🚀 Dynamic Crawling: Execute JS and wait for async or sync for dynamic content extraction.
📸 Screenshots: Capture page screenshots during crawling for debugging or analysis.
📂 Raw Data Crawling: Directly process raw HTML (raw:) or local files (file://).
🔗 Comprehensive Link Extraction: Extracts internal, external links, and embedded iframe content.
🛠️ Customizable Hooks: Define hooks at every step to customize crawling behavior (supports both string and function-based APIs).
💾 Caching: Cache data for improved speed and to avoid redundant fetches.
📄 Metadata Extraction: Retrieve structured metadata from web pages.
📡 IFrame Content Extraction: Seamless extraction from embedded iframe content.
🕵️ Lazy Load Handling: Waits for images to fully load, ensuring no content is missed due to lazy loading.
🔄 Full-Page Scanning: Simulates scrolling to load and capture all dynamic content, perfect for infinite scroll pages.

🚀 Deployment

🐳 Dockerized Setup: Optimized Docker image with FastAPI server for easy deployment.
🔑 Secure Authentication: Built-in JWT token authentication for API security.
🔄 API Gateway: One-click deployment with secure token authentication for API-based workflows.
🌐 Scalable Architecture: Designed for mass-scale production and optimized server performance.
☁️ Cloud Deployment: Ready-to-deploy configurations for major cloud platforms.

🎯 Additional Features

🕶️ Stealth Mode: Avoid bot detection by mimicking real users.
🏷️ Tag-Based Content Extraction: Refine crawling based on custom tags, headers, or metadata.
🔗 Link Analysis: Extract and analyze all links for detailed data exploration.
🛡️ Error Handling: Robust error management for seamless execution.
🔐 CORS & Static Serving: Supports filesystem-based caching and cross-origin requests.
📖 Clear Documentation: Simplified and updated guides for onboarding and advanced usage.
🙌 Community Recognition: Acknowledges contributors and pull requests for transparency.

Try it Now!

✨ Play around with this

✨ Visit our Documentation Website

Installation 🛠️

Crawl4AI offers flexible installation options to suit various use cases. You can install it as a Python package or use Docker.

🐍 Using pip

Choose the installation option that best fits your needs:

Basic Installation

For basic web crawling and scraping tasks:

pip install crawl4ai
crawl4ai-setup # Setup the browser

By default, this will install the asynchronous version of Crawl4AI, using Playwright for web crawling.

👉 Note: When you install Crawl4AI, the crawl4ai-setup should automatically install and set up Playwright. However, if you encounter any Playwright-related errors, you can manually install it using one of these methods:

Through the command line:
```
playwright install
```
If the above doesn't work, try this more specific command:
```
python -m playwright install chromium
```

This second method has proven to be more reliable in some cases.

Installation with Synchronous Version

The sync version is deprecated and will be removed in future versions. If you need the synchronous version using Selenium:

pip install crawl4ai[sync]

Development Installation

For contributors who plan to modify the source code:

git clone https://github.com/unclecode/crawl4ai.git
cd crawl4ai
pip install -e .                    # Basic installation in editable mode

Install optional features:

pip install -e ".[torch]"           # With PyTorch features
pip install -e ".[transformer]"     # With Transformer features
pip install -e ".[cosine]"          # With cosine similarity features
pip install -e ".[sync]"            # With synchronous crawling (Selenium)
pip install -e ".[all]"             # Install all optional features

🐳 Docker Deployment

🚀 Now Available! Our completely redesigned Docker implementation is here! This new solution makes deployment more efficient and seamless than ever.

New Docker Features

The new Docker implementation includes:

Real-time Monitoring Dashboard with live system metrics and browser pool visibility
Browser pooling with page pre-warming for faster response times
Interactive playground to test and generate request code
MCP integration for direct connection to AI tools like Claude Code
Comprehensive API endpoints including HTML extraction, screenshots, PDF generation, and JavaScript execution
Multi-architecture support with automatic detection (AMD64/ARM64)
Optimized resources with improved memory management

Getting Started

# Pull and run the latest release
docker pull unclecode/crawl4ai:latest
docker run -d -p 11235:11235 --name crawl4ai --shm-size=1g unclecode/crawl4ai:latest

# Visit the monitoring dashboard at http://localhost:11235/dashboard
# Or the playground at http://localhost:11235/playground

Quick Test

Run a quick test (works for both Docker options):

import requests

# Submit a crawl job
response = requests.post(
    "http://localhost:11235/crawl",
    json={"urls": ["https://example.com"], "priority": 10}
)
if response.status_code == 200:
    print("Crawl job submitted successfully.")
    
if "results" in response.json():
    results = response.json()["results"]
    print("Crawl job completed. Results:")
    for result in results:
        print(result)
else:
    task_id = response.json()["task_id"]
    print(f"Crawl job submitted. Task ID:: {task_id}")
    result = requests.get(f"http://localhost:11235/task/{task_id}")

For more examples, see our Docker Examples. For advanced configuration, monitoring features, and production deployment, see our Self-Hosting Guide.

🔬 Advanced Usage Examples 🔬

You can check the project structure in the directory docs/examples. Over there, you can find a variety of examples; here, some popular examples are shared.

📝 Heuristic Markdown Generation with Clean and Fit Markdown

import asyncio
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode
from crawl4ai.content_filter_strategy import PruningContentFilter, BM25ContentFilter
from crawl4ai.markdown_generation_strategy import DefaultMarkdownGenerator

async def main():
    browser_config = BrowserConfig(
        headless=True,  
        verbose=True,
    )
    run_config = CrawlerRunConfig(
        cache_mode=CacheMode.ENABLED,
        markdown_generator=DefaultMarkdownGenerator(
            content_filter=PruningContentFilter(threshold=0.48, threshold_type="fixed", min_word_threshold=0)
        ),
        # markdown_generator=DefaultMarkdownGenerator(
        #     content_filter=BM25ContentFilter(user_query="WHEN_WE_FOCUS_BASED_ON_A_USER_QUERY", bm25_threshold=1.0)
        # ),
    )
    
    async with AsyncWebCrawler(config=browser_config) as crawler:
        result = await crawler.arun(
            url="https://docs.micronaut.io/4.9.9/guide/",
            config=run_config
        )
        print(len(result.markdown.raw_markdown))
        print(len(result.markdown.fit_markdown))

if __name__ == "__main__":
    asyncio.run(main())

🖥️ Executing JavaScript & Extract Structured Data without LLMs

import asyncio
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode
from crawl4ai import JsonCssExtractionStrategy
import json

async def main():
    schema = {
    "name": "KidoCode Courses",
    "baseSelector": "section.charge-methodology .w-tab-content > div",
    "fields": [
        {
            "name": "section_title",
            "selector": "h3.heading-50",
            "type": "text",
        },
        {
            "name": "section_description",
            "selector": ".charge-content",
            "type": "text",
        },
        {
            "name": "course_name",
            "selector": ".text-block-93",
            "type": "text",
        },
        {
            "name": "course_description",
            "selector": ".course-content-text",
            "type": "text",
        },
        {
            "name": "course_icon",
            "selector": ".image-92",
            "type": "attribute",
            "attribute": "src"
        }
    ]
}

    extraction_strategy = JsonCssExtractionStrategy(schema, verbose=True)

    browser_config = BrowserConfig(
        headless=False,
        verbose=True
    )
    run_config = CrawlerRunConfig(
        extraction_strategy=extraction_strategy,
        js_code=["""(async () => {const tabs = document.querySelectorAll("section.charge-methodology .tabs-menu-3 > div");for(let tab of tabs) {tab.scrollIntoView();tab.click();await new Promise(r => setTimeout(r, 500));}})();"""],
        cache_mode=CacheMode.BYPASS
    )
        
    async with AsyncWebCrawler(config=browser_config) as crawler:
        
        result = await crawler.arun(
            url="https://www.kidocode.com/degrees/technology",
            config=run_config
        )

        companies = json.loads(result.extracted_content)
        print(f"Successfully extracted {len(companies)} companies")
        print(json.dumps(companies[0], indent=2))


if __name__ == "__main__":
    asyncio.run(main())

📚 Extracting Structured Data with LLMs

import os
import asyncio
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode, LLMConfig
from crawl4ai import LLMExtractionStrategy
from pydantic import BaseModel, Field

class OpenAIModelFee(BaseModel):
    model_name: str = Field(..., description="Name of the OpenAI model.")
    input_fee: str = Field(..., description="Fee for input token for the OpenAI model.")
    output_fee: str = Field(..., description="Fee for output token for the OpenAI model.")

async def main():
    browser_config = BrowserConfig(verbose=True)
    run_config = CrawlerRunConfig(
        word_count_threshold=1,
        extraction_strategy=LLMExtractionStrategy(
            # Here you can use any provider that Litellm library supports, for instance: ollama/qwen2
            # provider="ollama/qwen2", api_token="no-token", 
            llm_config = LLMConfig(provider="openai/gpt-4o", api_token=os.getenv('OPENAI_API_KEY')), 
            schema=OpenAIModelFee.schema(),
            extraction_type="schema",
            instruction="""From the crawled content, extract all mentioned model names along with their fees for input and output tokens. 
            Do not miss any models in the entire content. One extracted model JSON format should look like this: 
            {"model_name": "GPT-4", "input_fee": "US$10.00 / 1M tokens", "output_fee": "US$30.00 / 1M tokens"}."""
        ),            
        cache_mode=CacheMode.BYPASS,
    )
    
    async with AsyncWebCrawler(config=browser_config) as crawler:
        result = await crawler.arun(
            url='https://openai.com/api/pricing/',
            config=run_config
        )
        print(result.extracted_content)

if __name__ == "__main__":
    asyncio.run(main())

🤖 Using Your own Browser with Custom User Profile

import os, sys
from pathlib import Path
import asyncio, time
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode

async def test_news_crawl():
    # Create a persistent user data directory
    user_data_dir = os.path.join(Path.home(), ".crawl4ai", "browser_profile")
    os.makedirs(user_data_dir, exist_ok=True)

    browser_config = BrowserConfig(
        verbose=True,
        headless=True,
        user_data_dir=user_data_dir,
        use_persistent_context=True,
    )
    run_config = CrawlerRunConfig(
        cache_mode=CacheMode.BYPASS
    )
    
    async with AsyncWebCrawler(config=browser_config) as crawler:
        url = "ADDRESS_OF_A_CHALLENGING_WEBSITE"
        
        result = await crawler.arun(
            url,
            config=run_config,
            magic=True,
        )
        
        print(f"Successfully crawled {url}")
        print(f"Content length: {len(result.markdown)}")

✨ Recent Updates

Version 0.9.2 Release Highlights - Maintenance Bug Fixes

A maintenance patch release with bug fixes across the dispatcher, Docker, and GPU builds. MemoryAdaptiveDispatcher no longer leaks crawl tasks and browser pages when a streaming crawl is closed. Docker fixes cover the Playground "Advanced Config" 400, the Monitor WebSocket 500 under JWT auth, and Playwright headless-shell packaging. ENABLE_GPU=true Docker builds no longer fail on the CUDA toolkit.

No new features, no breaking changes.

pip install -U crawl4ai

Full v0.9.2 Release Notes →

Version 0.9.1 Release Highlights - Bug Fixes & PruningContentFilter Whitelist

A patch release with 12 bug fixes and one new feature. The new preserve_classes / preserve_tags parameters for PruningContentFilter let you whitelist CSS classes or HTML tags that should never be pruned — useful for protecting short metadata elements like author names and timestamps.

Bug fixes span Docker (auth gate UI, supervisord/redis dirs, FastAPI compatibility, redis auth), browser (Windows channel crash, context snapshot leak), core (HTTP timeout unit mismatch, best-first ordering), and extraction (html2text table attributes).

pip install -U crawl4ai

Full v0.9.1 Release Notes →

Version 0.9.0 Release Highlights - Secure-by-Default Docker Server

A major, secure-by-default release of the Docker API server. The out-of-the-box deployment is hardened with defense in depth: authentication is on by default, the server binds loopback unless you give it a token, and the network request body is treated as an untrusted trust boundary.

pip install -U crawl4ai

Migration Guide → · Full v0.9.0 Release Notes →

Version 0.8.7 Release Highlights - Security Hardening, DomainMapper & Community Fixes

A security-hardening release. Fixes critical Docker API vulnerabilities (AST sandbox escape RCE, hook sandbox RCE, hardcoded JWT secret, SSRF on webhook and crawl endpoints, arbitrary file write, monitor auth bypass, stored XSS, and unauthenticated JS execution), adds the DomainMapper feature, and ships a batch of scraping, deep-crawl, and LLM fixes. If you self-host the Docker API, upgrade immediately.

pip install -U crawl4ai

Full v0.8.7 Release Notes →

Version 0.8.6 - Security Hotfix: litellm Supply Chain Fix

Replaced litellm dependency with unclecode-litellm due to a PyPI supply chain compromise affecting the original package. If you're on v0.8.5 or earlier, upgrade immediately.

pip install -U crawl4ai

Version 0.8.5 Release Highlights - Anti-Bot Detection, Shadow DOM & 60+ Bug Fixes

Our biggest release since v0.8.0. Anti-bot detection with proxy escalation, Shadow DOM flattening, deep crawl cancellation, and over 60 bug fixes.

🛡️ Anti-Bot Detection & Proxy Escalation:

3-tier detection: known vendors, generic block indicators, structural integrity checks
Automatic retry with proxy chain and fallback fetch function

from crawl4ai import CrawlerRunConfig
from crawl4ai.async_configs import ProxyConfig

config = CrawlerRunConfig(
    proxy_config=[ProxyConfig.DIRECT, ProxyConfig(server="http://my-proxy:8080")],
    max_retries=2,
    fallback_fetch_function=my_web_unlocker,
)

🌑 Shadow DOM Flattening:
- Extract content hidden inside shadow DOM components
```
config = CrawlerRunConfig(flatten_shadow_dom=True)
```
🛑 Deep Crawl Cancellation:
- Stop long crawls gracefully with cancel() or should_cancel callback
- Works with BFS, DFS, and BestFirst strategies
⚙️ Config Defaults API:
- set_defaults() / get_defaults() / reset_defaults() on BrowserConfig and CrawlerRunConfig
🔒 Critical Security Fixes:
- RCE via deserialization in Docker /crawl endpoint — removed eval(), added allowlist
- Redis CVE-2025-49844 (CVSS 10.0) — upgraded to 7.2.7
60+ Bug Fixes across browser management, proxy, deep crawling, extraction, CLI, and Docker

Full v0.8.5 Release Notes →

Version 0.8.0 Release Highlights - Crash Recovery & Prefetch Mode

This release introduces crash recovery for deep crawls, a new prefetch mode for fast URL discovery, and critical security fixes for Docker deployments.

🔄 Deep Crawl Crash Recovery:
- on_state_change callback fires after each URL for real-time state persistence
- resume_state parameter to continue from a saved checkpoint
- JSON-serializable state for Redis/database storage
- Works with BFS, DFS, and Best-First strategies
```
from crawl4ai.deep_crawling import BFSDeepCrawlStrategy

strategy = BFSDeepCrawlStrategy(
    max_depth=3,
    resume_state=saved_state,  # Continue from checkpoint
    on_state_change=save_to_redis,  # Called after each URL
)
```
⚡ Prefetch Mode for Fast URL Discovery:
- prefetch=True skips markdown, extraction, and media processing
- 5-10x faster than full processing
- Perfect for two-phase crawling: discover first, process selectively
```
config = CrawlerRunConfig(prefetch=True)
result = await crawler.arun("https://example.com", config=config)
# Returns HTML and links only - no markdown generation
```
🔒 Security Fixes (Docker API):
- Hooks disabled by default (CRAWL4AI_HOOKS_ENABLED=false)
- file:// URLs blocked on API endpoints to prevent LFI
- __import__ removed from hook execution sandbox

Full v0.8.0 Release Notes →

Version 0.7.8 Release Highlights - Stability & Bug Fix Release

This release focuses on stability with 11 bug fixes addressing issues reported by the community. No new features, but significant improvements to reliability.

🐳 Docker API Fixes:
- Fixed ContentRelevanceFilter deserialization in deep crawl requests (#1642)
- Fixed ProxyConfig JSON serialization in BrowserConfig.to_dict() (#1629)
- Fixed .cache folder permissions in Docker image (#1638)

🤖 LLM Extraction Improvements:

Configurable rate limiter backoff with new LLMConfig parameters (#1269):

from crawl4ai import LLMConfig

config = LLMConfig(
    provider="openai/gpt-4o-mini",
    backoff_base_delay=5,           # Wait 5s on first retry
    backoff_max_attempts=5,          # Try up to 5 times
    backoff_exponential_factor=3     # Multiply delay by 3 each attempt
)

HTML input format support for LLMExtractionStrategy (#1178):

from crawl4ai import LLMExtractionStrategy

strategy = LLMExtractionStrategy(
    llm_config=config,
    instruction="Extract table data",
    input_format="html"  # Now supports: "html", "markdown", "fit_markdown"
)

Fixed raw HTML URL variable - extraction strategies now receive "Raw HTML" instead of HTML blob (#1116)

🔗 URL Handling:
- Fixed relative URL resolution after JavaScript redirects (#1268)
- Fixed import statement formatting in extracted code (#1181)
📦 Dependency Updates:
- Replaced deprecated PyPDF2 with pypdf (#1412)
- Pydantic v2 ConfigDict compatibility - no more deprecation warnings (#678)
🧠 AdaptiveCrawler:
- Fixed query expansion to actually use LLM instead of hardcoded mock data (#1621)

Full v0.7.8 Release Notes →

Version 0.7.7 Release Highlights - The Self-Hosting & Monitoring Update

📊 Real-time Monitoring Dashboard: Interactive web UI with live system metrics and browser pool visibility

# Access the monitoring dashboard
# Visit: http://localhost:11235/dashboard

# Real-time metrics include:
# - System health (CPU, memory, network, uptime)
# - Active and completed request tracking
# - Browser pool management (permanent/hot/cold)
# - Janitor cleanup events
# - Error monitoring with full context

🔌 Comprehensive Monitor API: Complete REST API for programmatic access to all monitoring data

import httpx

async with httpx.AsyncClient() as client:
    # System health
    health = await client.get("http://localhost:11235/monitor/health")

    # Request tracking
    requests = await client.get("http://localhost:11235/monitor/requests")

    # Browser pool status
    browsers = await client.get("http://localhost:11235/monitor/browsers")

    # Endpoint statistics
    stats = await client.get("http://localhost:11235/monitor/endpoints/stats")

⚡ WebSocket Streaming: Real-time updates every 2 seconds for custom dashboards
🔥 Smart Browser Pool: 3-tier architecture (permanent/hot/cold) with automatic promotion and cleanup
🧹 Janitor System: Automatic resource management with event logging
🎮 Control Actions: Manual browser management (kill, restart, cleanup) via API
📈 Production Metrics: 6 critical metrics for operational excellence with Prometheus integration
🐛 Critical Bug Fixes:
- Fixed async LLM extraction blocking issue (#1055)
- Enhanced DFS deep crawl strategy (#1607)
- Fixed sitemap parsing in AsyncUrlSeeder (#1598)
- Resolved browser viewport configuration (#1495)
- Fixed CDP timing with exponential backoff (#1528)
- Security update for pyOpenSSL (>=25.3.0)

Full v0.7.7 Release Notes →

Version 0.7.5 Release Highlights - The Docker Hooks & Security Update

🔧 Docker Hooks System: Complete pipeline customization with user-provided Python functions at 8 key points

✨ Function-Based Hooks API (NEW): Write hooks as regular Python functions with full IDE support:

from crawl4ai import hooks_to_string
from crawl4ai.docker_client import Crawl4aiDockerClient

# Define hooks as regular Python functions
async def on_page_context_created(page, context, **kwargs):
    """Block images to speed up crawling"""
    await context.route("**/*.{png,jpg,jpeg,gif,webp}", lambda route: route.abort())
    await page.set_viewport_size({"width": 1920, "height": 1080})
    return page

async def before_goto(page, context, url, **kwargs):
    """Add custom headers"""
    await page.set_extra_http_headers({'X-Crawl4AI': 'v0.7.5'})
    return page

# Option 1: Use hooks_to_string() utility for REST API
hooks_code = hooks_to_string({
    "on_page_context_created": on_page_context_created,
    "before_goto": before_goto
})

# Option 2: Docker client with automatic conversion (Recommended)
client = Crawl4aiDockerClient(base_url="http://localhost:11235")
results = await client.crawl(
    urls=["https://httpbin.org/html"],
    hooks={
        "on_page_context_created": on_page_context_created,
        "before_goto": before_goto
    }
)
# ✓ Full IDE support, type checking, and reusability!

🤖 Enhanced LLM Integration: Custom providers with temperature control and base_url configuration
🔒 HTTPS Preservation: Secure internal link handling with preserve_https_for_internal_links=True
🐍 Python 3.10+ Support: Modern language features and enhanced performance
🛠️ Bug Fixes: Resolved multiple community-reported issues including URL processing, JWT authentication, and proxy configuration

Full v0.7.5 Release Notes →

Version 0.7.4 Release Highlights - The Intelligent Table Extraction & Performance Update

🚀 LLMTableExtraction: Revolutionary table extraction with intelligent chunking for massive tables:

from crawl4ai import LLMTableExtraction, LLMConfig

# Configure intelligent table extraction
table_strategy = LLMTableExtraction(
    llm_config=LLMConfig(provider="openai/gpt-4.1-mini"),
    enable_chunking=True,           # Handle massive tables
    chunk_token_threshold=5000,     # Smart chunking threshold
    overlap_threshold=100,          # Maintain context between chunks
    extraction_type="structured"    # Get structured data output
)

config = CrawlerRunConfig(table_extraction_strategy=table_strategy)
result = await crawler.arun("https://complex-tables-site.com", config=config)

# Tables are automatically chunked, processed, and merged
for table in result.tables:
    print(f"Extracted table: {len(table['data'])} rows")

⚡ Dispatcher Bug Fix: Fixed sequential processing bottleneck in arun_many for fast-completing tasks
🧹 Memory Management Refactor: Consolidated memory utilities into main utils module for cleaner architecture
🔧 Browser Manager Fixes: Resolved race conditions in concurrent page creation with thread-safe locking
🔗 Advanced URL Processing: Better handling of raw:// URLs and base tag link resolution
🛡️ Enhanced Proxy Support: Flexible proxy configuration supporting both dict and string formats

Full v0.7.4 Release Notes →

Version 0.7.3 Release Highlights - The Multi-Config Intelligence Update

🕵️ Undetected Browser Support: Bypass sophisticated bot detection systems:

from crawl4ai import AsyncWebCrawler, BrowserConfig

browser_config = BrowserConfig(
    browser_type="undetected",  # Use undetected Chrome
    headless=True,              # Can run headless with stealth
    extra_args=[
        "--disable-blink-features=AutomationControlled",
        "--disable-web-security"
    ]
)

async with AsyncWebCrawler(config=browser_config) as crawler:
    result = await crawler.arun("https://protected-site.com")
# Successfully bypass Cloudflare, Akamai, and custom bot detection

🎨 Multi-URL Configuration: Different strategies for different URL patterns in one batch:

from crawl4ai import CrawlerRunConfig, MatchMode, CacheMode
  
  configs = [
      # Documentation sites - aggressive caching
      CrawlerRunConfig(
          url_matcher=["*docs*", "*documentation*"],
          cache_mode=CacheMode.WRITE_ONLY,
          markdown_generator_options={"include_links": True}
      ),
      
      # News/blog sites - fresh content
      CrawlerRunConfig(
          url_matcher=lambda url: 'blog' in url or 'news' in url,
          cache_mode=CacheMode.BYPASS
      ),
      
      # Fallback for everything else
      CrawlerRunConfig()
  ]
  
  results = await crawler.arun_many(urls, config=configs)
  # Each URL gets the perfect configuration automatically

🧠 Memory Monitoring: Track and optimize memory usage during crawling:

from crawl4ai.memory_utils import MemoryMonitor

monitor = MemoryMonitor()
monitor.start_monitoring()

results = await crawler.arun_many(large_url_list)

report = monitor.get_report()
print(f"Peak memory: {report['peak_mb']:.1f} MB")
print(f"Efficiency: {report['efficiency']:.1f}%")
# Get optimization recommendations

📊 Enhanced Table Extraction: Direct DataFrame conversion from web tables:

result = await crawler.arun("https://site-with-tables.com")

# New way - direct table access
if result.tables:
    import pandas as pd
    for table in result.tables:
        df = pd.DataFrame(table['data'])
        print(f"Table: {df.shape[0]} rows × {df.shape[1]} columns")

💰 GitHub Sponsors: 4-tier sponsorship system for project sustainability
🐳 Docker LLM Flexibility: Configure providers via environment variables

Full v0.7.3 Release Notes →

Version 0.7.0 Release Highlights - The Adaptive Intelligence Update

🧠 Adaptive Crawling: Your crawler now learns and adapts to website patterns automatically:

config = AdaptiveConfig(
    confidence_threshold=0.7, # Min confidence to stop crawling
    max_depth=5, # Maximum crawl depth
    max_pages=20, # Maximum number of pages to crawl
    strategy="statistical"
)

async with AsyncWebCrawler() as crawler:
    adaptive_crawler = AdaptiveCrawler(crawler, config)
    state = await adaptive_crawler.digest(
        start_url="https://news.example.com",
        query="latest news content"
    )
# Crawler learns patterns and improves extraction over time

🌊 Virtual Scroll Support: Complete content extraction from infinite scroll pages:

scroll_config = VirtualScrollConfig(
    container_selector="[data-testid='feed']",
    scroll_count=20,
    scroll_by="container_height",
    wait_after_scroll=1.0
)

result = await crawler.arun(url, config=CrawlerRunConfig(
    virtual_scroll_config=scroll_config
))

🔗 Intelligent Link Analysis: 3-layer scoring system for smart link prioritization:

link_config = LinkPreviewConfig(
    query="machine learning tutorials",
    score_threshold=0.3,
    concurrent_requests=10
)

result = await crawler.arun(url, config=CrawlerRunConfig(
    link_preview_config=link_config,
    score_links=True
))
# Links ranked by relevance and quality

🎣 Async URL Seeder: Discover thousands of URLs in seconds:

seeder = AsyncUrlSeeder(SeedingConfig(
    source="sitemap+cc",
    pattern="*/blog/*",
    query="python tutorials",
    score_threshold=0.4
))

urls = await seeder.discover("https://example.com")

⚡ Performance Boost: Up to 3x faster with optimized resource handling and memory efficiency

Read the full details in our 0.7.0 Release Notes or check the CHANGELOG.

Version Numbering in Crawl4AI

Crawl4AI follows standard Python version numbering conventions (PEP 440) to help users understand the stability and features of each release.

📈 Version Numbers Explained

Our version numbers follow this pattern: MAJOR.MINOR.PATCH (e.g., 0.4.3)

Pre-release Versions

We use different suffixes to indicate development stages:

dev (0.4.3dev1): Development versions, unstable
a (0.4.3a1): Alpha releases, experimental features
b (0.4.3b1): Beta releases, feature complete but needs testing
rc (0.4.3): Release candidates, potential final version

Installation

Regular installation (stable version):
```
pip install -U crawl4ai
```
Install pre-release versions:
```
pip install crawl4ai --pre
```
Install specific version:
```
pip install crawl4ai==0.4.3b1
```

Why Pre-releases?

We use pre-releases to:

Test new features in real-world scenarios
Gather feedback before final releases
Ensure stability for production users
Allow early adopters to try new features

For production environments, we recommend using the stable version. For testing new features, you can opt-in to pre-releases using the --pre flag.

📖 Documentation & Roadmap

🚨 Documentation Update Alert: We're undertaking a major documentation overhaul next week to reflect recent updates and improvements. Stay tuned for a more comprehensive and up-to-date guide!

For current documentation, including installation instructions, advanced features, and API reference, visit our Documentation Website.

To check our development plans and upcoming features, visit our Roadmap.

📈 Development TODOs

🤝 Contributing

We welcome contributions from the open-source community. Check out our contribution guidelines for more information.

I'll help modify the license section with badges. For the halftone effect, here's a version with it:

Here's the updated license section:

📄 License & Attribution

This project is licensed under the Apache License 2.0, attribution is recommended via the badges below. See the Apache 2.0 License file for details.

Attribution Requirements

When using Crawl4AI, you must include one of the following attribution methods:

📈 1. Badge Attribution (Recommended)

Add one of these badges to your README, documentation, or website:

Theme	Badge
Disco Theme (Animated)
Night Theme (Dark with Neon)
Dark Theme (Classic)
Light Theme (Classic)

HTML code for adding the badges:

<!-- Disco Theme (Animated) -->
<a href="https://github.com/unclecode/crawl4ai">
  <img src="https://raw.githubusercontent.com/unclecode/crawl4ai/main/docs/assets/powered-by-disco.svg" alt="Powered by Crawl4AI" width="200"/>
</a>

<!-- Night Theme (Dark with Neon) -->
<a href="https://github.com/unclecode/crawl4ai">
  <img src="https://raw.githubusercontent.com/unclecode/crawl4ai/main/docs/assets/powered-by-night.svg" alt="Powered by Crawl4AI" width="200"/>
</a>

<!-- Dark Theme (Classic) -->
<a href="https://github.com/unclecode/crawl4ai">
  <img src="https://raw.githubusercontent.com/unclecode/crawl4ai/main/docs/assets/powered-by-dark.svg" alt="Powered by Crawl4AI" width="200"/>
</a>

<!-- Light Theme (Classic) -->
<a href="https://github.com/unclecode/crawl4ai">
  <img src="https://raw.githubusercontent.com/unclecode/crawl4ai/main/docs/assets/powered-by-light.svg" alt="Powered by Crawl4AI" width="200"/>
</a>

<!-- Simple Shield Badge -->
<a href="https://github.com/unclecode/crawl4ai">
  <img src="https://img.shields.io/badge/Powered%20by-Crawl4AI-blue?style=flat-square" alt="Powered by Crawl4AI"/>
</a>

📖 2. Text Attribution

Add this line to your documentation: ``` This project uses Crawl4AI (https://github.com/unclecode/crawl4ai) for web data extraction. ```

📚 Citation

If you use Crawl4AI in your research or project, please cite:

@software{crawl4ai2024,
  author = {UncleCode},
  title = {Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub Repository},
  howpublished = {\url{https://github.com/unclecode/crawl4ai}},
  commit = {Please use the commit hash you're working with}
}

Text citation format:

UncleCode. (2024). Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper [Computer software]. 
GitHub. https://github.com/unclecode/crawl4ai

📧 Contact

For questions, suggestions, or feedback, feel free to reach out:

GitHub: unclecode
Twitter: @unclecode
Website: crawl4ai.com

Happy Crawling! 🕸️🚀

🗾 Mission

Our mission is to unlock the value of personal and enterprise data by transforming digital footprints into structured, tradeable assets. Crawl4AI empowers individuals and organizations with open-source tools to extract and structure data, fostering a shared data economy.

We envision a future where AI is powered by real human knowledge, ensuring data creators directly benefit from their contributions. By democratizing data and enabling ethical sharing, we are laying the foundation for authentic AI advancement.

🔑 Key Opportunities

Data Capitalization: Transform digital footprints into measurable, valuable assets.
Authentic AI Data: Provide AI systems with real human insights.
Shared Economy: Create a fair data marketplace that benefits data creators.

🚀 Development Pathway

Open-Source Tools: Community-driven platforms for transparent data extraction.
Digital Asset Structuring: Tools to organize and value digital knowledge.
Ethical Data Marketplace: A secure, fair platform for exchanging structured data.

For more details, see our full mission statement.

🌟 Current Sponsors

🤝 Strategic Partners

These companies provide core infrastructure and technology that power Crawl4AI’s capabilities — from web access and proxy networks to AI tooling and data pipelines.

Company	About
	Massive is a web access API backed by millions of volunteer devices in 195+ countries. AI agents, models, and data pipelines use it to reach any site on the internet, reliably, in real time, and at scale.

🏢 Enterprise Sponsors

Our enterprise sponsors support Crawl4AI and help scale it to power production-grade data pipelines.

Company	About	Sponsorship Tier
	Helps engineers and buyers find, compare, and source electronic & industrial parts in seconds, with specs, pricing, lead times & alternatives.	🥇 Gold
	Kidocode is a hybrid technology and entrepreneurship school for kids aged 5–18, offering both online and on-campus education.	🥇 Gold
	Singapore-based Aleph Null is Asia’s leading edtech hub, dedicated to student-centric, AI-driven education—empowering learners with the tools to thrive in a fast-changing world.	🥇 Gold

💼 Become a Strategic Partner or Sponsor

Interested in partnering with Crawl4AI?

Whether you’re a proxy provider, AI infrastructure company, cloud platform, or an organization looking to support the Crawl4AI ecosystem, we’d love to hear from you.

📩 Contact: hello@crawl4ai.com

🧑‍🤝 Individual Sponsors

A heartfelt thanks to our individual supporters! Every contribution helps us keep our opensource mission alive and thriving!

Want to join them? Sponsor Crawl4AI →

Star History

thinking-machines-lab/tinker-cookbook

Post-training with Tinker

Tinker Cookbook

We provide two libraries for the broader community to customize their language models: tinker and tinker-cookbook.

tinker is a training SDK for researchers and developers to fine-tune language models. You send API requests to us and we handle the complexities of distributed training.
tinker-cookbook includes realistic examples of fine-tuning language models. It builds on the Tinker API and provides common abstractions to fine-tune language models.

Installation

Sign up for Tinker here.
Once you have access, create an API key from the console and export it as environment variable TINKER_API_KEY.

Install tinker-cookbook (includes the tinker SDK as a dependency):

# Latest stable release from PyPI
uv pip install tinker-cookbook

# Or install the nightly build
uv pip install 'tinker-cookbook @ git+https://github.com/thinking-machines-lab/tinker-cookbook.git@nightly'

Using Inkling and Inkling-Small

Inkling is Thinking Machines Lab's model tailored for Tinker. It is a general-purpose model that can code, reason, call tools, and process image and audio inputs. Inkling-Small is a smaller, lower-cost model in the same family that shares Inkling's rendering path and reasoning-effort interface. Tinker Cookbook supports sampling and post-training both through the standalone tml-renderers library, included in the default installation together with the required torch>=2.10.

Pass model_name="thinkingmachines/Inkling" or model_name="thinkingmachines/Inkling-Small" anywhere the cookbook accepts a model name. The appropriate renderer and tokenizer are selected automatically for any thinkingmachines/Inkling* model, including the :peft: long-context variants.

Both models support controllable reasoning effort as well as multimodal audio and image inputs. See the Inkling documentation for setup and usage details, and Models & Pricing for per-model context lengths and pricing.

Tinker

Here we introduce a few Tinker primitives — the basic components to fine-tune LLMs (see the quickstart guide for more details):

import tinker
service_client = tinker.ServiceClient()
training_client = service_client.create_lora_training_client(
  base_model="meta-llama/Llama-3.2-1B", rank=32,
)
training_client.forward_backward(...)
training_client.optim_step(...)
training_client.save_state(...)
training_client.load_state(...)

sampling_client = training_client.save_weights_and_get_sampling_client()
sampling_client.sample(...)

See tinker_cookbook/recipes/sl_loop.py and tinker_cookbook/recipes/rl_loop.py for minimal examples of using these primitives to fine-tune LLMs.

Tutorials

New to Tinker? The tutorials/ directory contains 20+ progressive marimo notebooks that walk through core concepts — rendering, loss functions, completers, weight management — and advanced topics such as custom RL environments, DPO, RLHF, and weight export. Run any tutorial with marimo edit tutorials/101_hello_tinker.py. See the tutorials README for the full list, or browse rendered versions on the Tinker docs site.

To download the weights of any model:

rest_client = service_client.create_rest_client()
future = rest_client.get_checkpoint_archive_url_from_tinker_path(sampling_client.model_path)
with open(f"model-checkpoint.tar.gz", "wb") as f:
    f.write(future.result())

Tinker Cookbook

Besides these primitives, we also offer Tinker Cookbook (a.k.a. this repo), a library of a wide range of abstractions to help you customize training environments. tinker_cookbook/recipes/sl_basic.py and tinker_cookbook/recipes/rl_basic.py contain minimal examples to configure supervised learning and reinforcement learning.

We also include more complete examples in the tinker_cookbook/recipes/ folder:

Chat SFT: supervised fine-tuning on conversational datasets (e.g., Tulu3).
Math RL: reinforcement learning for mathematical reasoning with verifiable rewards.
Code RL: RL on competitive programming with sandboxed code execution (DeepCoder replication).
Preference learning: DPO and a three-stage RLHF pipeline (SFT, reward model, RL).
Distillation: on-policy and off-policy knowledge distillation with single- and multi-teacher configurations.
Tool use: RL for retrieval-augmented generation (Search-R1 replication).
Multi-agent: multi-agent RL with self-play and cross-play.
Audio: SFT and RL for Inkling audio inputs, including speech recognition and speaking-style classification.
VLM image classification: supervised training and evaluation for vision-language models.

The recipes README covers all available recipes, including Harbor RL, rubric-based grading, VLM classification, and SDFT. Each recipe includes a README.md with implementation details, launch commands, and expected results.

Evaluation (experimental)

Tinker Cookbook includes a benchmark framework for evaluating trained models:

from tinker_cookbook.eval.benchmarks import run_benchmarks, BenchmarkConfig

results = await run_benchmarks(
    ["gsm8k", "mmlu_pro", "ifeval"],
    sampling_client, renderer,
    BenchmarkConfig(save_dir="evals/step500"),
)

The framework currently supports 12 benchmarks (GSM8K, MATH-500, MMLU-Pro, MMLU-Redux, GPQA, IFEval, MBPP, C-Eval, SuperGPQA, IFBench, AIME 2025, AIME 2026) with verified scores against published results, plus experimental benchmarks such as LiveCodeBench, Terminal Bench, and SWE-bench. Benchmarks can also serve as inline training evaluators via BenchmarkEvaluator.

Note: Benchmark scores are sensitive to evaluation configuration — system prompts, max_tokens, temperature, and timeout settings can shift results significantly. We document our exact settings alongside all reported scores. This framework is under active development; feedback and contributions are welcome. See the eval README for verified scores, configuration details, and instructions for adding new benchmarks.

Documentation

For the full Tinker documentation, visit tinker-docs.thinkingmachines.ai.

Utilities

Tinker Cookbook also provides reusable building blocks:

renderers — bidirectional conversion between token sequences and structured chat messages
hyperparam_utils — learning rate and hyperparameter scaling for LoRA training
eval — benchmark framework and inline training evaluators (see Evaluation above)

Claude Code Skills

Tinker Cookbook ships with Claude Code skills that teach Claude how to use the Tinker API. Install them so Claude can help you write training code in any project:

/plugin marketplace add thinking-machines-lab/tinker-cookbook

Then install the tinker plugin from the Discover tab (/plugin → Discover). Once installed, three skills are available:

Command	What it does
`/tinker:research`	Plan and run post-training experiments — SFT, RL, DPO, distillation, evaluation, hyperparameters, model selection, and more
`/tinker:debug`	Diagnose slow training, hangs, output mismatches, renderer issues, and errors
`/tinker:inkling`	Work with Inkling and Inkling-Small — thinking effort, rendering, post-training, evaluation, and multimodal input

Skills also trigger automatically based on context — ask Claude to "set up SFT training" and it will load the right skill without a slash command. Skills update automatically when the repo is updated.

Development Setup

uv sync --extra dev
pre-commit install

This installs dev dependencies and registers pre-commit hooks that run ruff formatting and linting on every commit. CI enforces these checks on all pull requests.

Contributing

This project is built in the spirit of open science and collaborative development. We believe that the best tools emerge through community involvement and shared learning.

We welcome PR contributions after our private beta is over. If you have any feedback, please email us at tinker@thinkingmachines.ai.

Citation

If you use Tinker for your research, please cite it as:

Thinking Machines Lab, 2026. Tinker. https://thinkingmachines.ai/tinker/.

Or use this BibTeX citation:

@misc{tml2026tinker,
  author = {Thinking Machines Lab},
  title = {Tinker},
  year = {2026},
  url = {https://thinkingmachines.ai/tinker/},
}

paperless-ngx/paperless-ngx

A community-supported supercharged document management system: scan, index and archive all your documents

Paperless-ngx

Paperless-ngx is a document management system that transforms your physical documents into a searchable online archive so you can keep, well, less paper.

Paperless-ngx is the official successor to the original Paperless & Paperless-ng projects and is designed to distribute the responsibility of advancing and supporting the project among a team of people. Consider joining us!

Thanks to the generous folks at DigitalOcean, a demo is available at demo.paperless-ngx.com using login demo / demo. Note: demo content is reset frequently and confidential information should not be uploaded.

This project is supported by:

Features

A full list of features and screenshots are available in the documentation.

Getting started

The easiest way to deploy paperless is docker compose. The files in the /docker/compose directory are configured to pull the image from the GitHub container registry.

If you'd like to jump right in, you can configure a docker compose environment with our install script:

bash -c "$(curl -L https://raw.githubusercontent.com/paperless-ngx/paperless-ngx/main/install-paperless-ngx.sh)"

More details and step-by-step guides for alternative installation methods can be found in the documentation.

Migrating from Paperless-ng is easy, just drop in the new docker image! See the documentation on migrating for more details.

Documentation

The documentation for Paperless-ngx is available at https://docs.paperless-ngx.com.

Contributing

If you feel like contributing to the project, please do! Bug fixes, enhancements, visual fixes etc. are always welcome. If you want to implement something big: Please start a discussion about that! The documentation has some basic information on how to get started.

Community Support

People interested in continuing the work on paperless-ngx are encouraged to reach out here on github and in the Matrix Room. If you would like to contribute to the project on an ongoing basis there are multiple teams (frontend, ci/cd, etc) that could use your help so please reach out!

Translation

Paperless-ngx is available in many languages that are coordinated on Crowdin. If you want to help out by translating paperless-ngx into your language, please head over to https://crowdin.com/project/paperless-ngx, and thank you! More details can be found in CONTRIBUTING.md.

Feature Requests

Feature requests can be submitted via GitHub Discussions, you can search for existing ideas, add your own and vote for the ones you care about.

Bugs

For bugs please open an issue or start a discussion if you have questions.

Related Projects

Please see the wiki for a user-maintained list of related projects and software that is compatible with Paperless-ngx.

Important Note

Document scanners are typically used to scan sensitive documents like your social insurance number, tax records, invoices, etc. Paperless-ngx should never be run on an untrusted host because information is stored in clear text without encryption. No guarantees are made regarding security (but we do try!) and you use the app at your own risk. The safest way to run Paperless-ngx is on a local server in your own home with backups in place.

harbor-framework/harbor

Framework for evaluating and improving agents

Harbor

Harbor is a framework from the creators of Terminal-Bench for evaluating and optimizing agents and language models. You can use Harbor to:

Evaluate arbitrary agents like Claude Code, OpenHands, Codex CLI, and more.
Build and share your own benchmarks and environments.
Conduct experiments in thousands of environments in parallel through providers like Daytona, Modal, LangSmith, Blaxel, and Novita Sandbox.
Generate rollouts for RL optimization.

Check out the Harbor Cookbook for end-to-end examples and guides.

Installation

uv tool install harbor

pip install harbor

Example: Running Terminal-Bench-2.0

Harbor is the official harness for Terminal-Bench-2.0:

export ANTHROPIC_API_KEY=<YOUR-KEY> 
harbor run --dataset terminal-bench@2.0 \
   --agent claude-code \
   --model anthropic/claude-opus-4-1 \
   --n-concurrent 4

This will launch the benchmark locally using Docker. To run it on a cloud provider (like Daytona) pass the --env flag as below:

export ANTHROPIC_API_KEY=<YOUR-KEY> 
export DAYTONA_API_KEY=<YOUR-KEY>
harbor run --dataset terminal-bench@2.0 \
   --agent claude-code \
   --model anthropic/claude-opus-4-1 \
   --n-concurrent 100 \
   --env daytona

To see all supported agents, and other options run:

harbor run --help

To explore all supported third party benchmarks (like SWE-Bench and Aider Polyglot) run:

harbor datasets list

To evaluate an agent and model one of these datasets, you can use the following command:

harbor run -d "<dataset@version>" -m "<model>" -a "<agent>"

Citation

If you use Harbor in academic work, please cite it using the “Cite this repository” button on GitHub or the following BibTeX entry:

@software{Harbor_Framework,
author = {{Harbor Framework Team}},
title = {{Harbor: A framework for evaluating and optimizing agents and models in container environments}},
year = {2026},
version = {v0.16.1},
doi = {10.5281/zenodo.20953922},
url = {https://doi.org/10.5281/zenodo.20953922}
}

The DOI above is the concept DOI, which always resolves to the latest release and aggregates citations across all versions. To cite a specific version instead, use that version's DOI from the Zenodo record.

dottxt-ai/outlines

Structured Outputs

🗒️ Structured outputs for LLMs 🗒️

Made with ❤👷️ by the team at .txt
Trusted by NVIDIA, Cohere, HuggingFace, vLLM, etc.

The .txt API is currently in early access. Request access here →

🚀 Building the future of structured generation

We're working with select partners to develop new interfaces to structured generation.

Need XML, FHIR, custom schemas or grammars? Let's talk.

Audit your schema: share one schema, we show you what breaks under generation, the constraints that fix it, and compliance rates before and after. Sign up here.

Why Outlines?

LLMs are powerful but their outputs are unpredictable. Most solutions attempt to fix bad outputs after generation using parsing, regex, or fragile code that breaks easily.

Outlines guarantees structured outputs during generation — directly from any LLM.

Works with any model - Same code runs across OpenAI, Ollama, vLLM, and more
Simple integration - Just pass your desired output type: model(prompt, output_type)
Guaranteed valid structure - No more parsing headaches or broken JSON
Provider independence - Switch models without changing code

The Outlines Philosophy

Outlines follows a simple pattern that mirrors Python's own type system. Simply specify the desired output type, and Outlines will ensure your data matches that structure exactly:

For a yes/no response, use Literal["Yes", "No"]
For numerical values, use int
For complex objects, define a structure with a Pydantic model

Quickstart

Getting started with outlines is simple:

1. Install outlines

pip install outlines

2. Connect to your preferred model

import outlines
from transformers import AutoTokenizer, AutoModelForCausalLM


MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
    AutoTokenizer.from_pretrained(MODEL_NAME)
)

3. Start with simple structured outputs

from typing import Literal
from pydantic import BaseModel


# Simple classification
sentiment = model(
    "Analyze: 'This product completely changed my life!'",
    Literal["Positive", "Negative", "Neutral"]
)
print(sentiment)  # "Positive"

# Extract specific types
temperature = model("What's the boiling point of water in Celsius?", int)
print(temperature)  # 100

4. Create complex structures

from pydantic import BaseModel
from enum import Enum

class Rating(Enum):
    poor = 1
    fair = 2
    good = 3
    excellent = 4

class ProductReview(BaseModel):
    rating: Rating
    pros: list[str]
    cons: list[str]
    summary: str

review = model(
    "Review: The XPS 13 has great battery life and a stunning display, but it runs hot and the webcam is poor quality.",
    ProductReview,
    max_new_tokens=200,
)

review = ProductReview.model_validate_json(review)
print(f"Rating: {review.rating.name}")  # "Rating: good"
print(f"Pros: {review.pros}")           # "Pros: ['great battery life', 'stunning display']"
print(f"Summary: {review.summary}")     # "Summary: Good laptop with great display but thermal issues"

Real-world examples

Here are production-ready examples showing how Outlines solves common problems:

🙋‍♂️ Customer Support Triage
This example shows how to convert a free-form customer email into a structured service ticket. By parsing attributes like priority, category, and escalation flags, the code enables automated routing and handling of support issues.

import outlines
from enum import Enum
from pydantic import BaseModel
from transformers import AutoTokenizer, AutoModelForCausalLM
from typing import List


MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
    AutoTokenizer.from_pretrained(MODEL_NAME)
)


def alert_manager(ticket):
    print("Alert!", ticket)


class TicketPriority(str, Enum):
    low = "low"
    medium = "medium"
    high = "high"
    urgent = "urgent"

class ServiceTicket(BaseModel):
    priority: TicketPriority
    category: str
    requires_manager: bool
    summary: str
    action_items: List[str]


customer_email = """
Subject: URGENT - Cannot access my account after payment

I paid for the premium plan 3 hours ago and still can't access any features.
I've tried logging out and back in multiple times. This is unacceptable as I
have a client presentation in an hour and need the analytics dashboard.
Please fix this immediately or refund my payment.
"""

prompt = f"""
<|im_start|>user
Analyze this customer email:

{customer_email}
<|im_end|>
<|im_start|>assistant
"""

ticket = model(
    prompt,
    ServiceTicket,
    max_new_tokens=500
)

# Use structured data to route the ticket
ticket = ServiceTicket.model_validate_json(ticket)
if ticket.priority == "urgent" or ticket.requires_manager:
    alert_manager(ticket)

📦 E-commerce product categorization
This use case demonstrates how outlines can transform product descriptions into structured categorization data (e.g., main category, sub-category, and attributes) to streamline tasks such as inventory management. Each product description is processed automatically, reducing manual categorization overhead.

import outlines
from pydantic import BaseModel
from transformers import AutoTokenizer, AutoModelForCausalLM
from typing import List, Optional


MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
    AutoTokenizer.from_pretrained(MODEL_NAME)
)


def update_inventory(product, category, sub_category):
    print(f"Updated {product.split(',')[0]} in category {category}/{sub_category}")


class ProductCategory(BaseModel):
    main_category: str
    sub_category: str
    attributes: List[str]
    brand_match: Optional[str]

# Process product descriptions in batches
product_descriptions = [
    "Apple iPhone 15 Pro Max 256GB Titanium, 6.7-inch Super Retina XDR display with ProMotion",
    "Organic Cotton T-Shirt, Men's Medium, Navy Blue, 100% Sustainable Materials",
    "KitchenAid Stand Mixer, 5 Quart, Red, 10-Speed Settings with Dough Hook Attachment"
]

template = outlines.Template.from_string("""
<|im_start|>user
Categorize this product:

{{ description }}
<|im_end|>
<|im_start|>assistant
""")

# Get structured categorization for all products
categories = model(
    [template(description=desc) for desc in product_descriptions],
    ProductCategory,
    max_new_tokens=200
)

# Use categorization for inventory management
categories = [
    ProductCategory.model_validate_json(category) for category in categories
]
for product, category in zip(product_descriptions, categories):
    update_inventory(product, category.main_category, category.sub_category)

📊 Parse event details with incomplete data
This example uses outlines to parse event descriptions into structured information (like event name, date, location, type, and topics), even handling cases where the data is incomplete. It leverages union types to return either structured event data or a fallback “I don’t know” answer, ensuring robust extraction in varying scenarios.

import outlines
from typing import Union, List, Literal
from pydantic import BaseModel
from enum import Enum
from transformers import AutoTokenizer, AutoModelForCausalLM


MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
    AutoTokenizer.from_pretrained(MODEL_NAME)
)

class EventType(str, Enum):
    conference = "conference"
    webinar = "webinar"
    workshop = "workshop"
    meetup = "meetup"
    other = "other"


class EventInfo(BaseModel):
    """Structured information about a tech event"""
    name: str
    date: str
    location: str
    event_type: EventType
    topics: List[str]
    registration_required: bool

# Create a union type that can either be a structured EventInfo or "I don't know"
EventResponse = Union[EventInfo, Literal["I don't know"]]

# Sample event descriptions
event_descriptions = [
    # Complete information
    """
    Join us for DevCon 2023, the premier developer conference happening on November 15-17, 2023
    at the San Francisco Convention Center. Topics include AI/ML, cloud infrastructure, and web3.
    Registration is required.
    """,

    # Insufficient information
    """
    Tech event next week. More details coming soon!
    """
]

# Process events
results = []
for description in event_descriptions:
    prompt = f"""
<|im_start>system
You are a helpful assistant
<|im_end|>
<|im_start>user
Extract structured information about this tech event:

{description}

If there is enough information, return a JSON object with the following fields:

- name: The name of the event
- date: The date where the event is taking place
- location: Where the event is taking place
- event_type: either 'conference', 'webinar', 'workshop', 'meetup' or 'other'
- topics: a list of topics of the conference
- registration_required: a boolean that indicates whether registration is required

If the information available does not allow you to fill this JSON, and only then, answer 'I don't know'.
<|im_end|>
<|im_start|>assistant
"""
    # Union type allows the model to return structured data or "I don't know"
    result = model(prompt, EventResponse, max_new_tokens=200)
    results.append(result)

# Display results
for i, result in enumerate(results):
    print(f"Event {i+1}:")
    if isinstance(result, str):
        print(f"  {result}")
    else:
        # It's an EventInfo object
        print(f"  Name: {result.name}")
        print(f"  Type: {result.event_type}")
        print(f"  Date: {result.date}")
        print(f"  Topics: {', '.join(result.topics)}")
    print()

# Use structured data in downstream processing
structured_count = sum(1 for r in results if isinstance(r, EventInfo))
print(f"Successfully extracted data for {structured_count} of {len(results)} events")

🗂️ Categorize documents into predefined types
In this case, outlines classifies documents into predefined categories (e.g., “Financial Report,” “Legal Contract”) using a literal type specification. The resulting classifications are displayed in both a table format and through a category distribution summary, illustrating how structured outputs can simplify content management.

import outlines
from typing import Literal, List
import pandas as pd
from transformers import AutoTokenizer, AutoModelForCausalLM


MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
    AutoTokenizer.from_pretrained(MODEL_NAME)
)


# Define classification categories using Literal
DocumentCategory = Literal[
    "Financial Report",
    "Legal Contract",
    "Technical Documentation",
    "Marketing Material",
    "Personal Correspondence"
]

# Sample documents to classify
documents = [
    "Q3 Financial Summary: Revenue increased by 15% year-over-year to $12.4M. EBITDA margin improved to 23% compared to 19% in Q3 last year. Operating expenses...",

    "This agreement is made between Party A and Party B, hereinafter referred to as 'the Parties', on this day of...",

    "The API accepts POST requests with JSON payloads. Required parameters include 'user_id' and 'transaction_type'. The endpoint returns a 200 status code on success."
]

template = outlines.Template.from_string("""
<|im_start|>user
Classify the following document into exactly one category among the following categories:
- Financial Report
- Legal Contract
- Technical Documentation
- Marketing Material
- Personal Correspondence

Document:
{{ document }}
<|im_end|>
<|im_start|>assistant
""")

# Classify documents
def classify_documents(texts: List[str]) -> List[DocumentCategory]:
    results = []

    for text in texts:
        prompt = template(document=text)
        # The model must return one of the predefined categories
        category = model(prompt, DocumentCategory, max_new_tokens=200)
        results.append(category)

    return results

# Perform classification
classifications = classify_documents(documents)

# Create a simple results table
results_df = pd.DataFrame({
    "Document": [doc[:50] + "..." for doc in documents],
    "Classification": classifications
})

print(results_df)

# Count documents by category
category_counts = pd.Series(classifications).value_counts()
print("\nCategory Distribution:")
print(category_counts)

📅 Schedule a meeting from requests with Function Calling
This example demonstrates how outlines can interpret a natural language meeting request and translate it into a structured format matching a predefined function’s parameters. Once the meeting details are extracted (e.g., title, date, duration, attendees), they are used to automatically schedule the meeting.

import outlines
import json
from typing import List, Optional
from datetime import date
from transformers import AutoTokenizer, AutoModelForCausalLM


MODEL_NAME = "microsoft/phi-4"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
    AutoTokenizer.from_pretrained(MODEL_NAME)
)


# Define a function with typed parameters
def schedule_meeting(
    title: str,
    date: date,
    duration_minutes: int,
    attendees: List[str],
    location: Optional[str] = None,
    agenda_items: Optional[List[str]] = None
):
    """Schedule a meeting with the specified details"""
    # In a real app, this would create the meeting
    meeting = {
        "title": title,
        "date": date,
        "duration_minutes": duration_minutes,
        "attendees": attendees,
        "location": location,
        "agenda_items": agenda_items
    }
    return f"Meeting '{title}' scheduled for {date} with {len(attendees)} attendees"

# Natural language request
user_request = """
I need to set up a product roadmap review with the engineering team for next
Tuesday at 2pm. It should last 90 minutes. Please invite john@example.com,
sarah@example.com, and the product team at product@example.com.
"""

# Outlines automatically infers the required structure from the function signature
prompt = f"""
<|im_start|>user
Extract the meeting details from this request:

{user_request}
<|im_end|>
<|im_start|>assistant
"""
meeting_params = model(prompt, schedule_meeting, max_new_tokens=200)

# The result is a dictionary matching the function parameters
meeting_params = json.loads(meeting_params)
print(meeting_params)

# Call the function with the extracted parameters
result = schedule_meeting(**meeting_params)
print(result)
# "Meeting 'Product Roadmap Review' scheduled for 2023-10-17 with 3 attendees"

📝 Dynamically generate prompts with re-usable templates
Using Jinja-based templates, this example shows how to generate dynamic prompts for tasks like sentiment analysis. It illustrates how to easily re-use and customize prompts—including few-shot learning strategies—for different content types while ensuring the outputs remain structured.

import outlines
from typing import List, Literal
from transformers import AutoTokenizer, AutoModelForCausalLM


MODEL_NAME = "microsoft/phi-4"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
    AutoTokenizer.from_pretrained(MODEL_NAME)
)


# 1. Create a reusable template with Jinja syntax
sentiment_template = outlines.Template.from_string("""
<|im_start>user
Analyze the sentiment of the following {{ content_type }}:

{{ text }}

Provide your analysis as either "Positive", "Negative", or "Neutral".
<|im_end>
<|im_start>assistant
""")

# 2. Generate prompts with different parameters
review = "This restaurant exceeded all my expectations. Fantastic service!"
prompt = sentiment_template(content_type="review", text=review)

# 3. Use the templated prompt with structured generation
result = model(prompt, Literal["Positive", "Negative", "Neutral"])
print(result)  # "Positive"

# Templates can also be loaded from files
example_template = outlines.Template.from_file("templates/few_shot.txt")

# Use with examples for few-shot learning
examples = [
    ("The food was cold", "Negative"),
    ("The staff was friendly", "Positive")
]
few_shot_prompt = example_template(examples=examples, query="Service was slow")
print(few_shot_prompt)

They use outlines

Model Integrations

Model type	Description	Documentation
Server Support	vLLM and Ollama	Server Integrations →
Local Model Support	transformers and llama.cpp	Model Integrations →
API Support	OpenAI, Gemini, and Dottxt	API Integrations →

Core Features

Feature	Description	Documentation
Multiple Choices	Constrain outputs to predefined options	Multiple Choices Guide →
Function Calls	Infer structure from function signatures	Function Guide →
JSON/Pydantic	Generate outputs matching JSON schemas	JSON Guide →
Regular Expressions	Generate text following a regex pattern	Regex Guide →
Grammars	Enforce complex output structures	Grammar Guide →

Other Features

Feature	Description	Documentation
Prompt templates	Separate complex prompts from code	Template Guide →
Custome types	Intuitive interface to build complex types	Python Types Guide →
Applications	Encapsulate templates and types into functions	Application Guide →

About .txt

Outlines is developed and maintained by .txt, a company dedicated to making LLMs more reliable for production applications.

Our focus is on advancing structured generation technology through:

🧪 Cutting-edge Research: We publish our findings on structured generation
🚀 Enterprise-grade solutions: You can license our enterprise-grade libraries.
🧩 Open Source Collaboration: We believe in building in public and contributing to the community

Community

💡 Have an idea? Come chat with us on Discord
🐞 Found a bug? Open an issue
🧩 Want to contribute? Consult our contribution guide.

Cite Outlines

@article{willard2023efficient,
  title={Efficient Guided Generation for Large Language Models},
  author={Willard, Brandon T and Louf, R{\'e}mi},
  journal={arXiv preprint arXiv:2307.09702},
  year={2023}
}

livekit/agents

A framework for building realtime voice AI agents 🤖🎙️📹

Looking for the JS/TS library? Check out AgentsJS

What is Agents?

The Agent Framework is designed for building realtime, programmable participants that run on servers. Use it to create conversational, multi-modal voice agents that can see, hear, and understand.

Features

Flexible integrations: A comprehensive ecosystem to mix and match the right STT, LLM, TTS, and Realtime API to suit your use case.
Integrated job scheduling: Built-in task scheduling and distribution with dispatch APIs to connect end users to agents.
Extensive WebRTC clients: Build client applications using LiveKit's open-source SDK ecosystem, supporting all major platforms.
Telephony integration: Works seamlessly with LiveKit's telephony stack, allowing your agent to make calls to or receive calls from phones.
Exchange data with clients: Use RPCs and other Data APIs to seamlessly exchange data with clients.
Semantic turn detection: Uses a transformer model to detect when a user is done with their turn, helps to reduce interruptions.
MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.
Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.
Open-source: Fully open-source, allowing you to run the entire stack on your own servers, including LiveKit server, one of the most widely used WebRTC media servers.

Installation

To install the core Agents library, along with plugins for popular model providers:

pip install "livekit-agents[openai,deepgram,cartesia]"

Docs and guides

Documentation on the framework and how to use it can be found here

Building with AI coding agents

If you're using an AI coding assistant to build with LiveKit Agents, we recommend the following setup for the best results:

Install the LiveKit Docs MCP server — Gives your coding agent access to up-to-date LiveKit documentation, code search across LiveKit repositories, and working examples.
Install the LiveKit Agent Skill — Provides your coding agent with architectural guidance and best practices for building voice AI applications, including workflow design, handoffs, tasks, and testing patterns.
```
npx skills add livekit/agent-skills --skill livekit-agents
```

The Agent Skill works best alongside the MCP server: the skill teaches your agent how to approach building with LiveKit, while the MCP server provides the current API details to implement it correctly.

Core concepts

Agent: An LLM-based application with defined instructions.
AgentSession: A container for agents that manages interactions with end users.
entrypoint: The starting point for an interactive session, similar to a request handler in a web server.
AgentServer: The main process that coordinates job scheduling and launches agents for user sessions.

Usage

Simple voice agent

from livekit.agents import (
    Agent,
    AgentServer,
    AgentSession,
    JobContext,
    RunContext,
    cli,
    function_tool,
    inference,
)


@function_tool
async def lookup_weather(
    context: RunContext,
    location: str,
):
    """Used to look up weather information."""

    return {"weather": "sunny", "temperature": 70}


server = AgentServer()


@server.rtc_session()
async def entrypoint(ctx: JobContext):
    session = AgentSession(
        vad=inference.VAD(),
        # any combination of STT, LLM, TTS, or realtime API can be used
        # this example shows LiveKit Inference, a unified API to access different models via LiveKit Cloud
        # to use model provider keys directly, replace with the following:
        # from livekit.plugins import deepgram, openai, cartesia
        # stt=deepgram.STT(model="nova-3"),
        # llm=openai.LLM(model="gpt-4.1-mini"),
        # tts=cartesia.TTS(model="sonic-3", voice="9626c31c-bec5-4cca-baa8-f8ba9e84c8bc"),
        stt=inference.STT("deepgram/nova-3", language="multi"),
        llm=inference.LLM("google/gemma-4-31b-it"),  # low-latency gemma, hosted on LiveKit
        tts=inference.TTS("cartesia/sonic-3", voice="9626c31c-bec5-4cca-baa8-f8ba9e84c8bc"),
    )

    agent = Agent(
        instructions="You are a friendly voice assistant built by LiveKit.",
        tools=[lookup_weather],
    )

    await session.start(agent=agent, room=ctx.room)
    await session.generate_reply(instructions="greet the user and ask about their day")


if __name__ == "__main__":
    cli.run_app(server)

You'll need the following environment variables for this example:

LIVEKIT_URL
LIVEKIT_API_KEY
LIVEKIT_API_SECRET

Multi-agent handoff

This code snippet is abbreviated. For the full example, see multi_agent.py

...
class IntroAgent(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions=f"You are a story teller. Your goal is to gather a few pieces of information from the user to make the story personalized and engaging."
            "Ask the user for their name and where they are from"
        )

    async def on_enter(self):
        self.session.generate_reply(instructions="greet the user and gather information")

    @function_tool
    async def information_gathered(
        self,
        context: RunContext,
        name: str,
        location: str,
    ):
        """Called when the user has provided the information needed to make the story personalized and engaging.

        Args:
            name: The name of the user
            location: The location of the user
        """

        context.userdata.name = name
        context.userdata.location = location

        story_agent = StoryAgent(name, location)
        return story_agent, "Let's start the story!"


class StoryAgent(Agent):
    def __init__(self, name: str, location: str) -> None:
        super().__init__(
            instructions=f"You are a storyteller. Use the user's information in order to make the story personalized."
            f"The user's name is {name}, from {location}",
            # override the default model, switching to Realtime API from standard LLMs
            llm=openai.realtime.RealtimeModel(voice="echo"),
            chat_ctx=chat_ctx,
        )

    async def on_enter(self):
        self.session.generate_reply()


@server.rtc_session()
async def entrypoint(ctx: JobContext):
    userdata = StoryData()
    session = AgentSession[StoryData](
        vad=inference.VAD(),
        stt="deepgram/nova-3",
        llm="google/gemma-4-31b-it",  # low-latency gemma, hosted on LiveKit
        tts="cartesia/sonic-3:9626c31c-bec5-4cca-baa8-f8ba9e84c8bc",
        userdata=userdata,
    )

    await session.start(
        agent=IntroAgent(),
        room=ctx.room,
    )
...

Testing

Automated tests are essential for building reliable agents, especially with the non-deterministic behavior of LLMs. LiveKit Agents include native test integration to help you create dependable agents.

@pytest.mark.asyncio
async def test_no_availability() -> None:
    llm = google.LLM()
    async with AgentSession(llm=llm) as sess:
        await sess.start(MyAgent())
        result = await sess.run(
            user_input="Hello, I need to place an order."
        )
        result.expect.skip_next_event_if(type="message", role="assistant")
        result.expect.next_event().is_function_call(name="start_order")
        result.expect.next_event().is_function_call_output()
        await (
            result.expect.next_event()
            .is_message(role="assistant")
            .judge(llm, intent="assistant should be asking the user what they would like")
        )

Examples

For more examples and detailed setup instructions, see the examples directory. For even more examples, see the python-agents-examples repository.

🎙️ Starter Agent

A starter agent optimized for voice conversations.

Code

🔄 Multi-user push to talk

Responds to multiple users in the room via push-to-talk.

Code

🎵 Background audio

Background ambient and thinking audio to improve realism.

Code

🛠️ Dynamic tool creation

Creating function tools dynamically.

Code

☎️ Outbound caller

Agent that makes outbound phone calls

Code

📋 Structured output

Using structured output from LLM to guide TTS tone.

Code

🔌 MCP support

Use tools from MCP servers

Code

💬 Text-only agent

Skip voice altogether and use the same code for text-only integrations

Code

📝 Multi-user transcriber

Produce transcriptions from all users in the room

Code

🎥 Video avatars

Add an AI avatar with Tavus, Bithuman, LemonSlice, and more

Code

🍽️ Restaurant ordering and reservations

Full example of an agent that handles calls for a restaurant.

Code

👁️ Gemini Live vision

Full example (including iOS app) of Gemini Live agent that can see.

Code

Running your agent

Testing in terminal

python myagent.py console

Runs your agent in terminal mode, enabling local audio input and output for testing. This mode doesn't require external servers or dependencies and is useful for quickly validating behavior.

Developing with LiveKit clients

python myagent.py dev

Starts the agent server and enables hot reloading when files change. This mode allows each process to host multiple concurrent agents efficiently.

The agent connects to LiveKit Cloud or your self-hosted server. Set the following environment variables:

LIVEKIT_URL
LIVEKIT_API_KEY
LIVEKIT_API_SECRET

You can connect using any LiveKit client SDK or telephony integration. To get started quickly, try the Agents Playground.

Running for production

python myagent.py start

Runs the agent with production-ready optimizations.

License

The Agents framework is licensed under Apache-2.0. The LiveKit turn detection models are licensed under the LiveKit Model License.

Contributing

The Agents framework is under active development in a rapidly evolving field. We welcome and appreciate contributions of any kind, be it feedback, bugfixes, features, new plugins and tools, or better documentation. You can file issues under this repo, open a PR, or chat with us in the LiveKit community.

Development setup

This project uses uv for package management. To install dependencies for development:

uv sync --all-extras --dev

Examples

This project includes many examples in the examples directory. To run them, create the file examples/.env with credentials for LiveKit Server and any necessary model providers (see examples/.env.example), then run:

uv run examples/voice_agents/basic_agent.py dev

For more information, see the examples README.

Tests

Unit tests are in the tests directory and can be run with:

uv run pytest --unit

Integration tests for each plugin require various API credentials and run automatically in GitHub CI for PRs submitted by project maintainers. See the tests workflow for details.

Formatting

This project uses ruff for formatting and linting:

uv run ruff format
uv run ruff check --fix

Documentation

To generate docs locally with pdoc:

uv sync --all-extras --group docs
uv run --active pdoc --skip-errors --html --output-dir=docs livekit

LiveKit Ecosystem
Agents SDKs	Python · Node.js
LiveKit SDKs	Browser · Swift · Android · Flutter · React Native · Rust · Node.js · Python · Unity · Unity (WebGL) · ESP32 · C++
Starter Apps	Python Agent · TypeScript Agent · React App · SwiftUI App · Android App · Flutter App · React Native App · Web Embed
UI Components	React · Android Compose · SwiftUI · Flutter
Server APIs	Node.js · Golang · Ruby · Java/Kotlin · Python · Rust · PHP (community) · .NET (community)
Resources	Docs · Docs MCP Server · CLI · LiveKit Cloud
LiveKit Server OSS	LiveKit server · Egress · Ingress · SIP
Community	Developer Community · Slack · X · YouTube

PostHog/posthog

🦔 PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

Docs - Community - Roadmap - Why PostHog? - Changelog - Bug reports

PostHog is the open source platform for building self-driving products

PostHog provides every tool you need to build a successful product, and captures all the context agents need to proactively diagnose problems, uncover opportunities, and ship fixes:

Self-driving mode: Turn signals in your product data (errors, rage clicks, failed queries, and more) into researched reports and pull requests you review and merge.
Product analytics: Autocapture or manually instrument event-based analytics to understand user behavior and analyze data with visualization or SQL.
Web analytics: Monitor web traffic and user sessions with a GA-like dashboard. Easily monitor conversion, web vitals, and revenue.
Session replays: Watch real user sessions of interactions with your website or mobile app to diagnose issues and understand user behavior.
Feature flags: Safely roll out features to select users or cohorts with feature flags.
Experiments: Test changes and measure their statistical impact on goal metrics. Set up experiments with no-code too.
Error tracking: Track errors, get alerts, and resolve issues to improve your product.
Logs: Ingest, search, and analyze log data alongside the rest of your product data.
Surveys: Ask anything with our collection of no-code survey templates, or build custom surveys with our survey builder.
Data warehouse: Sync data from external tools like Stripe, Hubspot, your data warehouse, and more. Query it alongside your product data.
Data pipelines: Run custom filters and transformations on your incoming data. Send it to 25+ tools or any webhook in real time or batch export large amounts to your warehouse.
AI observability: Capture traces, generations, latency, and cost for your LLM-powered app.
Workflows: Create workflows that automate actions or send messages to your users.

You can steer it all from Slack, web, desktop (PostHog Desktop), or your own editor via the MCP.

Best of all, all of this is free to use with a generous monthly free tier for each tool. Get started by signing up for PostHog Cloud US or PostHog Cloud EU.

Getting started with PostHog

PostHog Cloud (Recommended)

The fastest and most reliable way to get started with PostHog is signing up for free to PostHog Cloud or PostHog Cloud EU. Your first 1 million events, 5k recordings, 1M flag requests, 100k exceptions, and 1500 survey responses are free every month, after which you pay based on usage.

Self-hosting the open-source hobby deploy (Advanced)

If you want to self-host PostHog, you can deploy a hobby instance in one line on Linux with Docker (recommended 4GB memory):

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/posthog/posthog/HEAD/bin/deploy-hobby)"

Open source deployments should scale to approximately 100k events per month, after which we recommend migrating to a PostHog Cloud.

We do not provide customer support or offer guarantees for open source deployments. See our self-hosting docs, troubleshooting guide, and disclaimer for more info.

Setting up PostHog

Once you've got a PostHog instance, you can set it up by installing our JavaScript web snippet, one of our SDKs, or by using our API. You can also connect the MCP to bring PostHog into Claude Code, Cursor, or any MCP-compatible agent.

We have SDKs and libraries for popular languages and frameworks like:

Frontend	Mobile	Backend
JavaScript	React Native	Python
Next.js	Android	Node
React	iOS	PHP
Vue	Flutter	Ruby

Beyond this, we have docs and guides for Go, .NET/C#, Django, Angular, WordPress, Webflow, and more.

Once you've installed PostHog, see our product docs for more information on how to set up product analytics, web analytics, session replays, feature flags, experiments, error tracking, surveys, data warehouse, and more.

Learning more about PostHog

Our code isn't the only thing that's open source 😳. We also open source our company handbook which details our strategy, ways of working, and processes.

Curious about how to make the most of PostHog? We wrote a guide to winning with PostHog which walks you through the basics of measuring activation, tracking retention, and capturing revenue.

Contributing

We ❤️ contributions big and small:

Vote on features or get early access to beta functionality in our roadmap
Open a PR (see our instructions on developing PostHog locally)
Submit a feature request or bug report

For an overview of the codebase structure, see monorepo layout and products.

Open-source vs. paid

This repo is available under the MIT expat license, except for the ee directory (which has its license here) if applicable.

Need absolutely 💯% FOSS? Check out our posthog-foss repository, which is purged of all proprietary code and features.

The pricing for our paid plan is completely transparent and available on our pricing page.

We're hiring!

Hey! If you're reading this, you've proven yourself as a dedicated README reader.

You might also make a great addition to our team. We're growing fast and would love for you to join us.

AgriciDaniel/claude-seo

Universal SEO skill for Claude Code. 25 sub-skills + 18 sub-agents covering technical SEO, E-E-A-T, schema, GEO/AEO, backlinks, local SEO, maps intelligence, semantic clustering, e-commerce SEO, international SEO, Google APIs, and PDF/Excel reporting. Optional DataForSEO, Firecrawl, and Banana extensions.

Claude SEO: SEO Skill for Claude Code

Claude SEO is an open-source SEO analysis plugin for Claude Code. It runs 25 sub-skills and 18 specialist agents in parallel across technical SEO, content quality (E-E-A-T), Schema.org markup, AI search optimization (GEO), local SEO, e-commerce, and international SEO. Every audit produces a prioritized action plan with testable recommendations grounded in primary-source guidance from Google.

Two versions of this skill.

🌐 Public open-source → AgriciDaniel/claude-seo: MIT, public releases, no membership. Use this if you want stable + downloadable.

🔒 Community private mirror → AI-Marketing-Hub/claude-seo: early access to upcoming features and direct collaboration with the AI Marketing Hub Pro community. Requires membership.

Why Claude SEO

AI-search first. Aligned with Google's AI Optimization Guide. Question-based citability scoring, primary-source evidence on llms.txt, IPTC TrainedAlgorithmicMedia for AI-generated product images, agent-friendly page checks per web.dev.
Parallel execution. Full site audits spawn up to 15 specialist agents simultaneously. Site-level audits complete in minutes rather than hours.
Falsifiable, not promotional. Every recommendation carries the first-principle observation it rests on, its dependency relationships, an explicit "how would we know this failed?" check, and a leading indicator. See Methodology.

Real results

Google Search Console for a site started 23 March 2026 and run on this workflow: total clicks and impressions across its first three months, through 12 June 2026.

Using Codex instead of Claude Code? Use Codex SEO, the Codex-first port with TOML agents, plugin packaging, deterministic runners, and the same SEO workflow surface.

Who this is for

SEO agencies running 5+ client sites. Replace quarterly deep audits with weekly automated runs. Same team capacity, 4× audit cadence, every recommendation comes with a falsifiability check the client can verify.
In-house SEO leads at SaaS / publisher / e-commerce companies. Second-pair-of-eyes before executive reviews. Catches what GSC and Lighthouse hide: schema deprecation, AI-citability gaps, expired-domain heritage risk, parasite-SEO exposure, machine-translation drift.
Freelance SEO consultants. Anchor day-one client scope with a 15-minute audit and a real 0-100 score. Win the engagement with concrete proof of value before you spend an hour writing the proposal.

Run a full audit and watch parallel agents fan out across the site:

Watch the full demo on YouTube

Installation

ℹ️ Which version are you installing?

Public open-source (default). The commands below install from AgriciDaniel/claude-seo — MIT, public releases, no membership required.

AI Marketing Hub Pro member? Install the community version with early access instead: swap AgriciDaniel/claude-seo for AI-Marketing-Hub/claude-seo and the plugin slug claude-seo@agricidaniel-claude-seo for claude-seo@ai-marketing-hub-claude-seo. Requires gh auth login (or PAT) with access to the AI-Marketing-Hub org. If /plugin marketplace add 404s, DM in the Skool community to get added.

Plugin Install (Claude Code 1.0.33+)

The fastest path. One-time marketplace add, then plugin install:

/plugin marketplace add AgriciDaniel/claude-seo
/plugin install claude-seo@agricidaniel-claude-seo
/seo setup

The explicit setup step creates an isolated Python environment in Claude's persistent plugin data and installs Playwright Chromium. Check it at any time with /seo doctor. No global Python packages or PATH shims are created.

Manual Install (Unix / macOS / Linux)

git clone --depth 1 https://github.com/AgriciDaniel/claude-seo.git
bash claude-seo/install.sh

One-liner (curl, review then run)

curl -fsSL https://raw.githubusercontent.com/AgriciDaniel/claude-seo/main/install.sh > install.sh
cat install.sh        # review before running
bash install.sh
rm install.sh

Windows (PowerShell)

git clone --depth 1 https://github.com/AgriciDaniel/claude-seo.git
powershell -ExecutionPolicy Bypass -File claude-seo\install.ps1

Why git clone instead of irm | iex? Claude Code's own security guardrails flag irm ... | iex as a supply chain risk: downloading and executing remote code without verification. The git clone approach lets you inspect claude-seo\install.ps1 before running it.

Quick Start

# Start Claude Code
claude

# Full site audit: parallel sub-agents produce a prioritized action plan
/seo audit https://example.com

# Deep single-page analysis: on-page elements, content quality, schema
/seo page https://example.com/about

# Schema markup audit: detect, validate, generate
/seo schema https://example.com

# AI search optimization: passage citability + primary-source-aligned recommendations
/seo geo https://example.com

# Generate a sitemap with industry templates
/seo sitemap generate

Commands

32 user-invocable /seo commands across the orchestrator, its sub-skills, and 8 MCP extensions. Full reference in docs/COMMANDS.md.

Command	Description
`/seo setup`	Create or refresh the isolated Python runtime and Chromium
`/seo doctor`	Check runtime readiness without changing the system
`/seo audit <url>`	Full website audit with parallel sub-agent delegation
`/seo page <url>`	Deep single-page analysis
`/seo technical <url>`	Technical SEO audit across 9 categories
`/seo content <url>`	E-E-A-T and content quality analysis
`/seo content-brief <topic>`	Detailed content brief: target keywords, outline, internal links
`/seo schema <url>`	Detect, validate, and generate Schema.org markup
`/seo geo <url>`	AI Overviews / Generative Engine Optimization
`/seo sitemap <url \| generate>`	Analyze or generate XML sitemaps
`/seo images <url>`	Image optimization analysis
`/seo plan <type>`	Strategic SEO planning (saas, local, ecommerce, publisher, agency)
`/seo programmatic <url>`	Programmatic SEO analysis and planning
`/seo competitor-pages <url>`	Competitor comparison page generation
`/seo local <url>`	Local SEO analysis (GBP, citations, reviews, map pack)
`/seo maps [command]`	Maps intelligence (geo-grid, GBP audit, reviews, competitors)
`/seo hreflang <url>`	Hreflang / i18n SEO audit and generation
`/seo google [command]`	Google SEO APIs (GSC, PageSpeed, CrUX, Indexing, GA4, PDF reports)
`/seo backlinks <url>`	Backlink profile analysis (Moz, Bing, Common Crawl)
`/seo cluster <keyword>`	SERP-based semantic clustering
`/seo sxo <url>`	Search Experience Optimization (page-type, user stories, personas)
`/seo drift baseline \| compare \| history <url>`	SEO drift monitoring with SQLite snapshots
`/seo ecommerce <url>`	E-commerce SEO and marketplace intelligence
`/seo flow [stage]`	FLOW framework prompts (CC BY 4.0, evidence-led)
`/seo firecrawl [command] <url>`	Full-site crawling (extension)
`/seo dataforseo [command]`	Live SEO data (extension)
`/seo image-gen [use-case]`	AI image generation for SEO assets (extension)
`/seo ahrefs [command] <url>`	Backlinks, organic keywords, and content data via the official Ahrefs MCP (extension)
`/seo seranking [command]`	AI Share-of-Voice across ChatGPT, Gemini, Perplexity, AI Overviews, AI Mode (extension)
`/seo profound [command]`	LLM citation tracking with time-series data (extension)
`/seo bing [command] <url>`	Bing Webmaster Tools + IndexNow URL submission (extension)
`/seo unlighthouse <url>`	Multi-page Lighthouse runner, runs locally (extension)

Features

What Core Web Vitals does Claude SEO check?

Claude SEO measures the current three Core Web Vitals: LCP (Largest Contentful Paint, target under 2.5s), INP (Interaction to Next Paint, target under 200ms), and CLS (Cumulative Layout Shift, target under 0.1). INP replaced FID on March 12, 2024; FID was removed from Chrome's field-data tools (CrUX API, PageSpeed Insights) on September 9, 2024 (Lighthouse is a lab tool and never reported FID), and Claude SEO never references FID. Field data comes from the Chrome User Experience Report (CrUX) when available; lab data falls back to Lighthouse via PageSpeed Insights. LCP can be decomposed into subparts (TTFB, load delay, load duration, render delay) via the /seo google CrUX integration to localize bottlenecks. Mobile and desktop are measured separately. CrUX History (25-week trend) is included in the Tier 0 free credential set.

How does Claude SEO assess E-E-A-T?

E-E-A-T (Experience, Expertise, Authoritativeness, Trustworthiness) is evaluated against the Search Quality Rater Guidelines, last updated September 2025 with YMYL expanded to include political and social topics. Experience signals: original research, case studies, first-hand photos. Expertise: author credentials and topical depth. Authoritativeness: external citations and brand mentions. Trustworthiness, the most heavily weighted of the four: contact info, secure HTTPS, transparent corrections, date stamps. Before scoring sub-factors, Claude SEO applies Google's own Who / How / Why heuristic from the helpful-content guide. Generative AI content is fine if it meets Search Essentials; it crosses into spam when used to scale low-value pages, which seo-content humanize and seo-content verify are designed to detect.

What Schema.org types does Claude SEO support?

JSON-LD is the preferred format (Google's stated preference). Claude SEO detects, validates, and generates the active Schema.org types documented in skills/seo/references/schema-types.md, including organization, article, product, local, event, job, course, software/application, service, Q&A, and video patterns. FAQPage: Google stopped showing FAQ rich results for all sites on May 7, 2026; it has no Google rich-result benefit. Keep it only for non-Google or internal semantics if needed. Deprecated and never recommended: HowTo (rich results removed September 2023), SpecialAnnouncement (July 2025), ClaimReview, VehicleListing, EstimatedSalary, LearningVideo, CourseInfo carousel (all retired June 2025). Replacement guidance: skills/seo-schema/references/deprecated-types-2024-2026.md.

How does Claude SEO optimize for AI search?

Aligned with Google's AI Optimization Guide, which states that "AEO" and "GEO" are rebranded labels for SEO. AI Overviews and AI Mode are grounded in the same ranking systems as classic Search; pages must be indexed and eligible for snippet display to appear in any AI feature. Claude SEO scores passage citability (optimal 134-167 word self-contained answer blocks), question-based heading hierarchy, attribution density, structured data coverage, and entity presence across Wikipedia, Reddit, YouTube, and LinkedIn. The seo-geo skill includes evidence-based reframes of three popular myths: llms.txt is not currently a citation lever (primary-source evidence), content chunking is not required, and AI-specific keyword rewriting is unnecessary because synonym understanding is sufficient.

Which Google SEO APIs does Claude SEO integrate with?

A 4-tier credential system lets you start with zero keys and add data as needed. Every tier delivers real value at its level:

Tier	Credentials	APIs Unlocked
0	API key	PageSpeed Insights, CrUX, CrUX History (25-week trends)
1	+ OAuth or Service Account	+ Search Console (queries, URL Inspection, sitemap status), Indexing API
2	+ GA4 property config	+ GA4 organic traffic, top landing pages, device / country breakdown
3	+ Ads developer token	+ Keyword Planner search volume and competition data

PDF reports are generated via WeasyPrint (A4 layout) with matplotlib charts at 200 DPI. Run /seo google setup for the credential wizard. All credentials live under ~/.config/claude-seo/ with 0o600 permissions; nothing is checked into the repo.

How does Claude SEO handle local SEO?

Three layers. Google Business Profile signals: categories, hours, photos, posts, products, attributes. NAP consistency across citations: name, address, phone matched against major directories with deviation flagging. Review intelligence: rating trends, sentiment, response coverage. For multi-location businesses, Claude SEO enforces a 30-page warning threshold and a 50-page hard stop to prevent doorway-page violations (configurable). The /seo maps workflow adds geo-grid rank tracking, GBP profile auditing, and competitor radius mapping. Local schema generation covers LocalBusiness with all required and recommended properties (geo coordinates, opening hours, areaServed). Phase F (v2) added a GBP deprecation linter that detects retired chat-field references and .business.site URLs.

Compared to manual / agency / commercial tools

	Manual audit	Agency engagement	Commercial SEO audit tool	Claude SEO
Time per audit	4-8 hrs senior SEO time	1-3 weeks turnaround	10-45 min crawl + report	10-15 min
Cost	High (billable hours)	$2k-$15k+ project	$99-$999/mo subscription	Free skill + Claude Code subscription
Repeatable	Inconsistent across analysts	Inconsistent across engagements	Yes	Yes, deterministic + scriptable
Output format	Wall-of-findings PDF	Branded slide deck	Web dashboard, CSV exports	Markdown + PDF + JSON, local files
Custom benchmarks	Manual per analyst	Agency-specific frameworks	Vendor-fixed	Edit local SKILL.md
Data leaves machine?	No (your spreadsheet)	Yes (sent to agency)	Yes (uploaded to vendor)	No, fully local by default
Lock-in	None	High	High (data-exit friction)	None. MIT, your files.
AI search awareness	Depends on analyst	Depends on agency seniority	Lagging (typically 6-12 mo behind)	Google AI Optimization Guide (May 2026), Sept 2025 QRG, INP-not-FID, GEO/AEO=SEO reframe, llms.txt evidence-based posture
Falsifiability per finding	No	No	No	Yes. Every recommendation carries a "how would we know this failed?" check + leading indicator

Cost benchmarks: manual audit assumes a senior SEO consultant at typical agency billable rates; agency engagement based on common discovery/audit deliverable scopes; commercial-tool subscriptions reflect published mid-tier pricing across the SEO audit category (Ahrefs, Semrush, Sitebulb, Screaming Frog). Your numbers may differ.

Use cases

SEO agency lead running 10 client sites. Replaces the quarterly "deep audit" ritual with a weekly Monday-morning /seo audit run per site. Time to deliver a client health-score email drops from 4 hours to 12 minutes; coverage goes from quarterly to weekly without billing more hours. The drift baseline catches regressions between audits so the client conversation moves from "look at this snapshot" to "here is what changed this week."

In-house SEO lead at a 50-person SaaS company. Runs /seo audit 24 hours before each quarterly business review. Catches the items the platform UI buries (broken canonical chains on programmatic pages, schema deprecation after Google's June 2025 retirement wave, AI-citability gaps that erode SERP-to-AI-Overview pickup, expired-domain heritage on acquired blog assets) before the CMO asks why organic traffic is down in front of the board.

Freelance SEO consultant onboarding a new client. Runs /seo audit on the discovery call. Anchors the engagement scope with a real 0-100 score, 3 prioritized critical findings, and a falsifiability check on each recommendation, instead of a vague "I'll take a look and get back to you." Closes more retainers because the proof of value happens during the call, not after the proposal.

Sample Output

Claude SEO writes real markdown reports as its primary deliverable. Below is the first ~50 lines of a /seo schema https://rankenstein.pro/about audit verbatim. The actual structure, headers, and grading format the plugin produces follows.

SCHEMA-REPORT.md: first 50 lines of a real audit

# Schema Markup Report: rankenstein.pro/about

**URL:** https://rankenstein.pro/about
**Date:** 2026-02-09
**Format Detected:** JSON-LD (3 blocks) | No Microdata | No RDFa

---

## Summary

| Metric | Value |
|--------|-------|
| **JSON-LD Blocks** | 3 |
| **Schema Types** | Organization, WebSite, SoftwareApplication |
| **Critical Issues** | 2 |
| **Warnings** | 5 |
| **Passed Checks** | 18 |
| **Overall Grade** | B+ (solid foundation, actionable gaps) |

---

## Existing Schema Validation

### 1. Organization (`@id: #organization`)

| Property | Value | Status | Notes |
|----------|-------|--------|-------|
| `@context` | https://schema.org | Valid | |
| `@type` | Organization | Valid | Active type |
| `@id` | https://rankenstein.pro#organization | Good | Enables cross-referencing |
| `name` | Rankenstein | Valid | |
| `description` | Present, 200+ chars | Good | Descriptive and keyword-rich |
| `url` | https://rankenstein.pro | Valid | Absolute URL |
| `logo` | ImageObject with @id, url, width, height, caption | Excellent | Well-structured |
| `foundingDate` | "2024" | Imprecise | Year-only accepted but ISO 8601 preferred |
| `areaServed` | "Worldwide" | Text | Works but `GeoShape` is more semantic |
| `contactPoint` | email + contactType | Valid | Consider adding `telephone` |
| `founder` | 1 Person (Daniel Agrici) | Incomplete | Page describes two co-founders; second missing |
| `sameAs` | 5 social profiles | Good | GitHub, X, LinkedIn, YouTube, Reddit |
| `knowsAbout` | 6 topics | Good | Relevant topical signals |

**Critical Issue:** The `founder` property only includes Daniel Agrici. Benjamin Samar (Co-Founder & Technical Director) is displayed on the page but absent from the schema. This creates a content-schema mismatch that can confuse search engines.

Other audit outputs follow the same shape: FULL-AUDIT-REPORT.md (umbrella audit), GEO-ANALYSIS.md (AI-search readiness), LOCAL-SEO-ANALYSIS.md (GBP and citations), and a production PDF via WeasyPrint + matplotlib (cover, TOC, executive summary, data sections, recommendations, methodology, roughly 32 A4 pages for a full site audit).

Architecture

The plugin follows the Agent Skills standard with a 3-layer architecture (directive, orchestration, execution). Skills and agents are auto-discovered from skills/seo-*/ and agents/seo-*.md. The orchestrator (skills/seo/SKILL.md) handles industry detection (SaaS, local, ecommerce, publisher, agency), parallel sub-agent dispatch up to 15 simultaneously, and synthesis through the 10-principle framework before emitting the action plan. Full architecture: docs/ARCHITECTURE.md.

Methodology

Every audit walks 10 principles grouped into four phases. Each emitted recommendation carries four fields: the first-principle observation it rests on, its dependency relationship to other recommendations, a "how would we know this failed?" check, and a leading indicator to monitor.

Phase	Principles	What it does
PERCEIVE	OBSERVE (external) · OBSERVE (internal) · LISTEN	Collect raw signals; audit your own assumptions; read what the SERP, the brand voice, and the community actually say
ANALYZE	THINK · CONNECT (lateral) · CONNECT (system)	Reduce to first principles; find non-obvious cross-skill links; sequence into a dependency graph
VALIDATE	FEEL · ACCEPT	Pressure-test against UX, brand voice, operator capacity; surface falsifiability
ACT	CREATE · GROW	Ship the artifact; set the feedback loop for the next audit

Full methodology: skills/seo/references/thinking-framework.md.

What's New in v2

v2.0.0 is the largest release in the plugin's history. Six build phases, all shipped:

Phase A: Headless rendering everywhere. Shared scripts/render_page.py with Playwright Chromium plus trafilatura and htmldate. Every audit subagent gets SPA-aware fetching via --render auto (auto-detected on Next.js, React, Vue, Nuxt, Astro islands). Closes the SPA limitation that capped v1.x.
Phase B: QRG-aligned content quality gates. Filler detector and AI-pattern humanizer keyed to QRG §4.6.5 and §4.6.6, claim-verification scanner, expired-domain heritage check via WHOIS, primary-source Google updates changelog.
Phase C: Technical and CWV depth. LCP subparts via CrUX (TTFB, load delay, load duration, render delay), Speculation Rules and bfcache detection, IndexNow submitter for Bing / Yandex / Seznam / Naver, Unlighthouse multi-page Lighthouse wrapper.
Phase D: Schema completeness. Four explicit generators (Reservation, OrderAction, DiscussionForumPosting, ProfilePage), e-commerce schema validator (hasMerchantReturnPolicy, shippingDetails, MemberProgram, EU energyEfficiencyClass, ProductGroup variants), dual validator (Rich Results Test plus Schema Markup Validator).
Phase E: AI search reframing and 5 new MCP extensions. Ahrefs, SE Ranking (AI Share-of-Voice), Profound (LLM citation tracker), Bing Webmaster plus IndexNow, Unlighthouse. Plus the parasite-SEO risk scanner per Google's November 2024 site reputation abuse policy.
Phase F: Local, international, and privacy polish. Google Business Profile deprecation linter (chat field and .business.site URLs, with Q&A treated as category/region-limited), DMA consent-mode-v2 click-through diagnostic, machine-translation QA flag per January 2025 QRG.

Test coverage grew from 39 (v1.9.9) to 410 across the v2 line; the url_safety suite alone runs 91 SSRF and DNS-rebinding bypass cases, closing the obfuscated-IPv4, FQDN-trailing-dot, and redirect-rebinding bypass classes. Full migration notes and breaking changes: docs/MIGRATION-v1-to-v2.md.

Since v2.0.0

v2.1.0 (May 2026): currency refresh. May 2026 core update, Google I/O 2026 (custom version of Gemini 2.5 powers AI Mode), FAQ rich results retired 2026-05-07 (QAPage remains the type for genuine Q&A pages, FAQ markup itself just no longer yields rich results).
v2.2.0 (June 2026): security + portability. Installer credential-injection fix, SSRF authority-confusion bypass closed, Google API keys moved to the X-Goog-Api-Key header, secret-scan CI gate, Windows/macOS fixes; suite at 326.
v2.2.1 (June 2026): Google-currency reconfirmation + full command audit. Lighthouse 13.4.0 with the new Agentic Browsing category, Google Search ignores llms.txt, an internally-reweighted E-E-A-T scorecard (Trust highest, per Google's 'trust is most important'); every /seo command and subcommand audited and COMMANDS.md brought to 100% coverage.
v2.2.2 (July 2026): full-review maintenance. Corrected GBP Q&A handling, AI Mode model naming, image-model IDs, hook input behavior, and added a strict reference-graph consistency gate.
v2.2.3 (July 2026): prompt-hygiene alignment. Normalized emphasis and punctuation across the prompt surface without changing behavior, routing, or output contracts.
v2.2.4 (July 2026): community maintenance. Added the managed cross-platform runtime and safe sitemap discovery, repaired GSC pagination and totals, replaced removed Bing endpoints, fixed extension and Windows portability gaps, and reconciled every open issue and pull request.

Limitations

Two real boundaries worth being upfront about.

Heavy client-side hydration timing. Phase A's headless renderer handles most SPAs out of the box (--render auto detects empty <div id="root"> shells and switches to Playwright). Edge cases that still produce noisy findings: pages with hydration tied to scroll position past the fold, pages that fetch critical content after user interaction (modal opens, tab clicks), pages with race-condition-prone third-party widget mounts. For these, manually triggering the seo-visual subagent and comparing its Playwright snapshot to the raw-HTML subagents' findings is the recommended workflow.

Local-only without enrichment. The free tier makes no third-party API calls by default (audits still fetch the target URLs you point them at). Adding Google API credentials (Tier 0 through 3) unlocks real field data and live indexation status; without them, Core Web Vitals are lab estimates only and indexation is inferred from page-level signals. Adding MCP extensions (Ahrefs, DataForSEO, SE Ranking, Profound) similarly unlocks competitive and AI-citation data but requires their respective accounts.

Requirements

Python 3.10+
Claude Code CLI
Optional: Playwright Chromium — install.sh offers to install it (you can skip the prompt); needed only for SPA rendering and screenshots
Optional: Google API credentials for enriched CWV / GSC / GA4 data (see /seo google setup)

Uninstall

git clone --depth 1 https://github.com/AgriciDaniel/claude-seo.git
bash claude-seo/uninstall.sh

One-liner (curl)

curl -fsSL https://raw.githubusercontent.com/AgriciDaniel/claude-seo/main/uninstall.sh | bash

Extensions

Optional MCP servers add live data to the audit pipeline. Claude SEO ships extensions for 8 servers; the plugin core works without any of them.

DataForSEO

Live SERP data, keyword research, backlinks, on-page analysis, content analysis, business listings, AI visibility checks, and LLM mention tracking. 23 data commands across 9 API modules.

./extensions/dataforseo/install.sh   # requires DataForSEO account
/seo dataforseo serp best coffee shops
/seo dataforseo ai-mentions your brand

Full DataForSEO docs: extensions/dataforseo/README.md.

Firecrawl

Full-site crawling and URL discovery via the Firecrawl MCP server.

./extensions/firecrawl/install.sh
/seo firecrawl crawl https://example.com

Full Firecrawl docs: extensions/firecrawl/README.md.

Banana: AI image generation

SEO image generation (OG previews, blog heroes, product photos, infographics) via the Claude Banana Creative Director pipeline.

./extensions/banana/install.sh
/seo image-gen og "Professional SaaS dashboard"

Full Banana docs: extensions/banana/README.md.

Ahrefs, SE Ranking, Profound, Bing Webmaster, Unlighthouse (new in v2)

Five extensions added in Phase E:

Ahrefs: official @ahrefs/mcp server with backlink and organic data
SE Ranking: AI Share-of-Voice across ChatGPT, Gemini, Perplexity, AI Overviews, AI Mode
Profound: LLM citation tracker with time-series data
Bing Webmaster: Bing Webmaster Tools plus IndexNow unified
Unlighthouse: MIT-licensed multi-page Lighthouse runner

Setup walkthroughs live under extensions/<name>/docs/; integration notes: docs/MCP-INTEGRATION.md.

Ecosystem

Claude SEO is part of a family of Claude Code skills that interoperate cleanly:

Skill	What it does	How it connects
Claude SEO	SEO analysis, audits, schema, GEO	Core. Analyzes sites and generates action plans.
Claude Blog	Blog writing, optimization, scoring	Companion. Writes content optimized by SEO findings.
Claude Banana	AI image generation via Gemini	Shared. Generates images for SEO assets and blog posts.
Codex SEO	Codex-first SEO skill suite	Port. Same SEO system adapted for Codex skills, TOML agents, deterministic runners.
AI Marketing Claude	Copywriting, emails, social, ads, funnels, CRO	Community. Post-audit marketing action from SEO findings.
FLOW	Evidence-led SEO framework (41 AI prompts, CC BY 4.0)	Knowledge base. Powers `seo-flow` prompts.

Workflow example:

/seo audit https://example.com: identify content gaps and technical issues
/seo backlinks https://example.com: analyze link profile and competitor gaps
/seo geo https://example.com/blog/post: score AI-citation readiness
/blog write "target keyword": create SEO-optimized blog post (Claude Blog)
/seo image-gen hero "blog topic": generate hero image (Banana extension)

Documentation

Installation Guide
Commands Reference: every /seo command in depth
Architecture: 3-layer design, auto-discovery, parallel dispatch
Migration v1 → v2: breaking changes, six phases of work
MCP Integration: integration notes; extension setup lives under extensions/<name>/docs/
Troubleshooting
Contributors: community credits

FAQ

What is Claude SEO?

Claude SEO is an open-source SEO analysis plugin for Claude Code. It runs 25 sub-skills and 18 specialist agents in parallel across technical SEO, content quality, Schema.org markup, AI search optimization, local SEO, e-commerce, and international SEO. Audits produce a prioritized action plan where each recommendation carries the first-principle observation it rests on, its dependency relationship to other recommendations, a "how would we know this failed?" check, and a leading indicator. The plugin is MIT-licensed, ships zero proprietary tracking, and works without third-party API enrichment; audits still contact the target URLs you analyze. Aligned with Google's AI Optimization Guide and the September 2025 Quality Rater Guidelines.

How is Claude SEO different from Screaming Frog or Ahrefs Site Audit?

Different surface area, different tradeoffs. Screaming Frog crawls deeper and faster at the link-graph level; it is purpose-built as a crawler and Claude SEO does not attempt to replace it. Ahrefs Site Audit brings a proprietary backlink index and link intelligence; Claude SEO integrates with Ahrefs via its MCP extension rather than competing. Where Claude SEO leads: conversational LLM-native workflow, recommendation falsifiability (every finding carries an explicit failure-mode check), open-source MIT licensing with zero per-domain pricing, AI search optimization aligned with Google's primary-source guidance, and primary-source schema-deprecation tracking. Use Screaming Frog or Ahrefs for what they are best at; use Claude SEO when you want LLM-driven synthesis, conversational iteration, and AI-search-first audits in the same environment as your other Claude Code workflows.

Does Claude SEO work on single-page applications (Next.js, React, Vue)?

Yes. Phase A of v2 shipped a shared headless renderer (scripts/render_page.py) backed by Playwright Chromium. Audit subagents call render_page.py --mode auto, which auto-detects SPA hallmarks (empty <div id="root"> shells, single bundle script, hydration markers) and switches to a rendered fetch. The lower-level scripts/fetch_page.py wrapper supports --render auto as an opt-in wrapper mode; its default is --render never for raw HTTP. Use render_page.py --mode always or fetch_page.py --render always to force rendering. Content extraction uses trafilatura for boilerplate removal. Publication dates come from htmldate. Known nuance: pages with scroll-bound hydration or post-interaction content fetches still produce noisy findings; see the Limitations section for the recommended seo-visual cross-check workflow on those edge cases.

What Google APIs does Claude SEO use, and are they required?

None are required. Claude SEO is fully functional with zero API keys. A 4-tier credential system lets you upgrade gradually: Tier 0 (API key only) unlocks PageSpeed Insights, CrUX, and CrUX History (25-week trend data). Tier 1 (+ OAuth or service account) adds Search Console with queries, URL Inspection, sitemap status, and the Indexing API for eligible JobPosting pages or BroadcastEvent in VideoObject pages; the API does not guarantee indexing. Tier 2 (+ GA4 property config) adds organic traffic, top landing pages, and device / country breakdowns. Tier 3 (+ Ads developer token) adds Keyword Planner search volume and competition data. The credential setup wizard runs via /seo google setup. All credentials live under ~/.config/claude-seo/ with 0o600 file permissions; nothing is checked into the repo and nothing is transmitted beyond Google's own endpoints.

Is Claude SEO free?

Yes. MIT licensed, fully open source, no per-domain pricing, no telemetry, no API quotas imposed by the plugin itself. The core plugin and all 25 sub-skills work without any paid service. Some optional MCP extensions wrap paid services (DataForSEO, Ahrefs, Profound, SE Ranking) where you bring your own account credentials; their use is opt-in and the plugin works fully without them. Google APIs (PageSpeed Insights, Search Console, Indexing, GA4) are free from Google with normal account quota limits and require your own credentials. If you want commercial support or enterprise features beyond the open-source plugin, that is not part of this project.

How is Claude SEO different from regular SEO tools when it comes to AI search?

Most SEO tools treat AI search as a separate optimization discipline. Claude SEO follows Google's own position that AEO and GEO are rebranded labels for SEO. AI Overviews and AI Mode are grounded in the same ranking systems as classic Search; the eligibility floor is normal indexation. Claude SEO scores passage citability (134-167 word self-contained answer blocks), question-based heading hierarchy, attribution density, and entity presence across Wikipedia, Reddit, YouTube, and LinkedIn. It explicitly rejects three influencer myths: llms.txt as a citation lever, content chunking for AI, and AI-specific keyword rewriting. For commerce sites, Claude SEO audits the IPTC TrainedAlgorithmicMedia requirement on AI-generated product images per Google Merchant Center policy.

Community Contributors

v1.9.0 includes contributions from the AI Marketing Hub Pro Hub Challenge:

Contributor	Contribution
Lutfiya Miller (Winner)	Semantic Cluster Engine → `seo-cluster`
Florian Schmitz	SXO Skill → `seo-sxo`
Dan Colta	SEO Drift Monitor → `seo-drift`
Chris Muller	Multi-lingual SEO → `seo-hreflang` enhancements
Matej Marjanovic	E-commerce + DataForSEO Cost Config → `seo-ecommerce` + cost guardrails

See CONTRIBUTORS.md for full details and original repo links.

License

MIT License. See LICENSE for details.

Contributing

Contributions welcome. Please read CONTRIBUTING.md before submitting PRs and include the tests or checks you ran in the PR description.

Author

Built by Agrici Daniel, AI Workflow Architect. Single maintainer, open to community contributions via the Pro Skool community. Background in marketing automation, AI-assisted content workflows, and open-source tooling for Claude Code.

Blog: deep dives on AI marketing automation
AI Marketing Hub (free): open community
AI Marketing Hub Pro: Pro community, early access to this skill
YouTube: tutorials and demos
GitHub: all open-source tools

arc53/DocsGPT

Private AI platform for agents, assistants and enterprise search. Built-in Agent Builder, Deep research, Document analysis, Multi-model support, and API connectivity for agents.

DocsGPT 🦖

Private AI for agents, assistants and enterprise search

DocsGPT is an open-source AI platform for building intelligent agents and assistants. Features Agent Builder, deep research tools, document analysis (PDF, Office, web content, and audio), Multi-model support (choose your provider or run locally), and rich API connectivity for agents with actionable tools and integrations. Deploy anywhere with complete privacy control.

⚡️ Quickstart • ☁️ Cloud Version • 💬 Discord
📖 Documentation • 👫 Contribute • 🗞 Blog

Key Features:

🗂️ Wide Format Support: Reads PDF, DOCX, CSV, XLSX, EPUB, MD, RST, HTML, MDX, JSON, PPTX, images, and audio files such as MP3, WAV, M4A, OGG, and WebM.
🎙️ Speech Workflows: Record voice input into chat, transcribe audio on the backend, and ingest meeting recordings or voice notes as searchable knowledge.
🌐 Web & Data Integration: Ingests from URLs, sitemaps, Reddit, GitHub and web crawlers.
✅ Reliable Answers: Get accurate, hallucination-free responses with source citations viewable in a clean UI.
🔑 Streamlined API Keys: Generate keys linked to your settings, documents, and models, simplifying chatbot and integration setup.
🔗 Actionable Tooling: Connect to APIs, tools, and other services to enable LLM actions.
🧩 Pre-built Integrations: Use readily available HTML/React chat widgets, search tools, Discord/Telegram bots, and more.
🔌 Flexible Deployment: Works with major LLMs (OpenAI, Google, Anthropic) and local models (Ollama, llama_cpp).
🏢 Secure & Scalable: Run privately and securely with Kubernetes support, designed for enterprise-grade reliability.

Roadmap

You can find our full roadmap here. Please don't hesitate to contribute or create issues, it helps us improve DocsGPT!

Production Support / Help for Companies:

We're eager to provide personalized assistance when deploying your DocsGPT to a live environment.

Get a Demo 👋⁠

Send Email 📧

Join the Lighthouse Program 🌟

Calling all developers and GenAI innovators! The DocsGPT Lighthouse Program connects technical leaders actively deploying or extending DocsGPT in real-world scenarios. Collaborate directly with our team to shape the roadmap, access priority support, and build enterprise-ready solutions with exclusive community insights.

Learn More & Apply →

QuickStart

Note

Make sure you have Docker installed

A more detailed Quickstart is available in our documentation

Clone the repository:

git clone https://github.com/arc53/DocsGPT.git
cd DocsGPT

For macOS and Linux:

Run the setup script:
```
./setup.sh
```

For Windows:

Run the PowerShell setup script:

PowerShell -ExecutionPolicy Bypass -File .\setup.ps1

Either script will guide you through setting up DocsGPT. Five options are available: using the public API, running locally, connecting to a local inference engine, using a cloud API provider, or building the docker image locally. The scripts will automatically configure your .env file and handle necessary downloads and installations based on your chosen option.

Navigate to http://localhost:5173/

To stop DocsGPT, open a terminal in the DocsGPT directory and run:

docker compose -f deployment/docker-compose.yaml down

(or use the specific docker compose down command shown after running the setup script).

Note

For development environment setup instructions, please refer to the Development Environment Guide.

Contributing

Please refer to the CONTRIBUTING.md file for information about how to get involved. We welcome issues, questions, and pull requests.

Architecture

Project Structure

Application - Backend Flask application.
Extensions - Integrations and widgets (e.g., Chatwoot, React widget).
Frontend - Web UI built with Vite and React.
Scripts - Miscellaneous utility scripts.

Code Of Conduct

We as members, contributors, and leaders, pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. Please refer to the CODE_OF_CONDUCT.md file for more information about contributing.

Many Thanks To Our Contributors⚡

License

The source code license is MIT, as described in the LICENSE file.

GitHub Python Monthly Trending Repositories

HKUDS/DeepTutor

DeepTutor: Lifelong Personalized Tutoring

📦 Releases

📰 News

✨ Key Features

🚀 Get Started

📖 Explore DeepTutor

⌨️ DeepTutor CLI — Agent-Native Interface

🧩 Ecosystem — EduHub & the Skills Community

🌐 Community

📮 Contact

🙏 Appreciation

🗺️ Roadmap & Contribute

Shubhamsaboo/awesome-llm-apps

Awesome LLM Apps

🚀 Run one now

📂 Browse all templates

🧩 Agent Skills

🌱 Starter AI Agents

🚀 Advanced AI Agents

🛰️ Always-on Agents

🤝 Multi-agent Teams

🗣️ Voice AI Agents

🖼️ Generative UI and Agentic Frontends

🎮 Autonomous Game-Playing Agents

♾️ MCP AI Agents

📀 RAG (Retrieval Augmented Generation)

💾 LLM Apps with Memory

💬 Chat with X

🎯 LLM Optimization Tools

🔧 LLM Fine-tuning

🧑‍🏫 AI Agent Framework Crash Courses

tirth8205/code-review-graph

code-review-graph

Quick Start

How It Works

Blast-radius analysis

Incremental updates in seconds

Whole codebase or targeted answer?

Broad language coverage + Jupyter notebooks

Add your own language (no fork needed)

Risk-scored PR reviews in CI (GitHub Action)

Benchmarks

Limitations and known weaknesses

Features

Usage

Environment Variables

Tool Filtering

FAQ & how it compares

Troubleshooting

pip / pipx cannot download hatchling (or Errno 9 / Bad file descriptor to PyPI)

Windows Configuration Issues (Invalid JSON / Connection Closed)

Contributing

Licence

virgiliojr94/book-to-skill

book-to-skill

🤔 Why

📦 What it generates

🏢 Beyond books

🧾 The Discovery Loop Tax

⚙️ How it works

🚀 Usage

📥 Install

❓ FAQ

⚖️ Copyright & fair use

💖 Sponsors

License

Star History

huggingface/speech-to-speech

Speech To Speech: Build voice agents with open-source models

Quickstart

Index

How it works

Installation

CUDA Note for Qwen3-TTS

Optional Components

From Source

Supported Components

Commands

`pip` / `pipx` cannot download `hatchling` (or `Errno 9` / `Bad file descriptor` to PyPI)

Migrating from `--mode`