GitHub All Languages Daily Trending

oven-sh/bun

Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one

Bun

Documentation • Discord • Issues • Roadmap

Read the docs →

What is Bun?

Bun is an all-in-one toolkit for JavaScript and TypeScript apps. It ships as a single executable called bun.

At its core is the Bun runtime, a fast JavaScript runtime designed as a drop-in replacement for Node.js. It's written in Zig and powered by JavaScriptCore under the hood, dramatically reducing startup times and memory usage.

bun run index.tsx             # TS and JSX supported out-of-the-box

The bun command-line tool also implements a test runner, script runner, and Node.js-compatible package manager. Instead of 1,000 node_modules for development, you only need bun. Bun's built-in tools are significantly faster than existing options and usable in existing Node.js projects with little to no changes.

bun test                      # run tests
bun run start                 # run the `start` script in `package.json`
bun install <pkg>             # install a package
bunx cowsay 'Hello, world!'   # execute a package

Install

Bun supports Linux (x64 & arm64), macOS (x64 & Apple Silicon), and Windows (x64 & arm64).

Linux users — Kernel version 5.6 or higher is strongly recommended, but the minimum is 5.1.

x64 users — if you see "illegal instruction" or similar errors, check our CPU requirements

# with install script (recommended)
curl -fsSL https://bun.com/install | bash

# on windows
powershell -c "irm bun.sh/install.ps1 | iex"

# with npm
npm install -g bun

# with Homebrew
brew tap oven-sh/bun
brew install bun

# with Docker
docker pull oven/bun
docker run --rm --init --ulimit memlock=-1:-1 oven/bun

Upgrade

To upgrade to the latest version of Bun, run:

bun upgrade

Bun automatically releases a canary build on every commit to main. To upgrade to the latest canary build, run:

bun upgrade --canary

View canary build

Quick links

Guides

Contributing

Refer to the Project > Contributing guide to start contributing to Bun.

License

Refer to the Project > License page for information about Bun's licensing.

K-Dense-AI/scientific-agent-skills

A set of ready to use Agent Skills for research, science, engineering, analysis, finance and writing.

Scientific Agent Skills

🔔 Claude Scientific Skills is now Scientific Agent Skills. Same skills, broader compatibility — now works with any AI agent that supports the open Agent Skills standard, not just Claude.

New: K-Dense BYOK — A free, open-source AI co-scientist that runs on your desktop, powered by Scientific Agent Skills. Bring your own API keys, pick from 40+ models, and get a full research workspace with web search, file handling, 100+ scientific databases, and access to all 135 skills in this repo. Your data stays on your computer, and you can optionally scale to cloud compute via Modal for heavy workloads. Get started here.

A comprehensive collection of 135 ready-to-use scientific and research skills (covering cancer genomics, drug-target binding, molecular dynamics, RNA velocity, geospatial science, time series forecasting, scientific ML resource discovery via Hugging Science, 78+ scientific databases, and more) for any AI agent that supports the open Agent Skills standard, created by K-Dense. Works with Cursor, Claude Code, Codex, and more. Transform your AI agent into a research assistant capable of executing complex multi-step scientific workflows across biology, chemistry, medicine, and beyond.

These skills enable your AI agent to seamlessly work with specialized scientific libraries, databases, and tools across multiple scientific domains. While the agent can use any Python package or API on its own, these explicitly defined skills provide curated documentation and examples that make it significantly stronger and more reliable for the workflows below:

🧬 Bioinformatics & Genomics - Sequence analysis, single-cell RNA-seq, gene regulatory networks, variant annotation, phylogenetic analysis
🧪 Cheminformatics & Drug Discovery - Molecular property prediction, virtual screening, ADMET analysis, molecular docking, lead optimization
🔬 Proteomics & Mass Spectrometry - LC-MS/MS processing, peptide identification, spectral matching, protein quantification
🏥 Clinical Research & Precision Medicine - Clinical trials, pharmacogenomics, variant interpretation, drug safety, clinical decision support, treatment planning
🧠 Healthcare AI & Clinical ML - EHR analysis, physiological signal processing, medical imaging, clinical prediction models
🖼️ Medical Imaging & Digital Pathology - DICOM processing, whole slide image analysis, computational pathology, radiology workflows
🤖 Machine Learning & AI - Deep learning, reinforcement learning, time series analysis, model interpretability, Bayesian methods
🔮 Materials Science & Chemistry - Crystal structure analysis, phase diagrams, metabolic modeling, computational chemistry
🌌 Physics & Astronomy - Astronomical data analysis, coordinate transformations, cosmological calculations, symbolic mathematics, physics computations
⚙️ Engineering & Simulation - Discrete-event simulation, multi-objective optimization, metabolic engineering, systems modeling, process optimization
📊 Data Analysis & Visualization - Statistical analysis, network analysis, time series, publication-quality figures, large-scale data processing, EDA
🌍 Geospatial Science & Remote Sensing - Satellite imagery processing, GIS analysis, spatial statistics, terrain analysis, machine learning for Earth observation
🧪 Laboratory Automation - Liquid handling protocols, lab equipment control, workflow automation, LIMS integration
📚 Scientific Communication - Literature review, peer review, scientific writing, document processing, posters, slides, schematics, citation management
🔬 Multi-omics & Systems Biology - Multi-modal data integration, pathway analysis, network biology, systems-level insights
🧬 Protein Engineering & Design - Protein language models, structure prediction, sequence design, function annotation
🎓 Research Methodology - Hypothesis generation, scientific brainstorming, critical thinking, grant writing, scholar evaluation

Transform your AI coding agent into an 'AI Scientist' on your desktop!

⭐ If you find this repository useful, please consider giving it a star! It helps others discover these tools and encourages us to continue maintaining and expanding this collection.

🎬 New to Scientific Agent Skills? Watch our Getting Started with Scientific Agent Skills video for a quick walkthrough.

📦 What's Included

This repository provides 135 scientific and research skills organized into the following categories:

100+ Scientific & Financial Databases - A unified database-lookup skill provides direct access to 78 public databases (PubChem, ChEMBL, UniProt, COSMIC, ClinicalTrials.gov, FRED, USPTO, and more), plus dedicated skills for DepMap, Imaging Data Commons, PrimeKG, U.S. Treasury Fiscal Data, and Hugging Science (curated catalog of scientific datasets, models, and demos across 17 scientific domains on Hugging Face). Multi-database packages like BioServices (~40 bioinformatics services), BioPython (38 NCBI sub-databases via Entrez), and gget (20+ genomics databases) add further coverage
70+ Optimized Python Package Skills - Explicitly defined skills for RDKit, Scanpy, PyTorch Lightning, scikit-learn, BioPython, pyzotero, BioServices, PennyLane, Qiskit, OpenMM, MDAnalysis, scVelo, TimesFM, and others — with curated documentation, examples, and best practices. Note: the agent can write code using any Python package, not just these; these skills simply provide stronger, more reliable performance for the packages listed
9 Scientific Integration Skills - Explicitly defined skills for Benchling, DNAnexus, LatchBio, OMERO, Protocols.io, Open Notebook, and more. Again, the agent is not limited to these — any API or platform reachable from Python is fair game; these skills are the optimized, pre-documented paths
30+ Analysis & Communication Tools - Literature review, scientific writing, peer review, document processing, posters, slides, schematics, infographics, Mermaid diagrams, and more
10+ Research & Clinical Tools - Hypothesis generation, grant writing, clinical decision support, treatment plans, regulatory compliance, scenario analysis

Each skill includes:

✅ Comprehensive documentation (SKILL.md)
✅ Practical code examples
✅ Use cases and best practices
✅ Integration guides
✅ Reference materials

📋 Table of Contents

🚀 Why Use This?

⚡ Accelerate Your Research

Save Days of Work - Skip API documentation research and integration setup
Production-Ready Code - Tested, validated examples following scientific best practices
Multi-Step Workflows - Execute complex pipelines with a single prompt

🎯 Comprehensive Coverage

135 Skills - Extensive coverage across all major scientific domains
100+ Databases - Unified access to 78+ databases via database-lookup, plus dedicated data access skills and multi-database packages like BioServices, BioPython, and gget
70+ Optimized Python Package Skills - RDKit, Scanpy, PyTorch Lightning, scikit-learn, BioServices, PennyLane, Qiskit, OpenMM, scVelo, TimesFM, and others (the agent can use any Python package; these are the pre-documented, higher-performing paths)

🔧 Easy Integration

Simple Setup - Copy skills to your skills directory and start working
Automatic Discovery - Your agent automatically finds and uses relevant skills
Well Documented - Each skill includes examples, use cases, and best practices

🌟 Maintained & Supported

Regular Updates - Continuously maintained and expanded by K-Dense team
Community Driven - Open source with active community contributions
Enterprise Ready - Commercial support available for advanced needs

🎯 Getting Started

Option 1: npx (all platforms)

Install Scientific Agent Skills with a single command:

npx skills add K-Dense-AI/scientific-agent-skills

This is the official standard approach for installing Agent Skills across all platforms, including Claude Code, Claude Cowork, Codex, Gemini CLI, Cursor, and any other agent that supports the open Agent Skills standard.

Option 2: GitHub CLI (`gh skill`)

If you use the GitHub CLI (v2.90.0+), you can install skills with gh skill:

# Browse and install interactively
gh skill install K-Dense-AI/scientific-agent-skills

# Install a specific skill directly
gh skill install K-Dense-AI/scientific-agent-skills scanpy

# Target a specific agent host
gh skill install K-Dense-AI/scientific-agent-skills --agent cursor
gh skill install K-Dense-AI/scientific-agent-skills --agent claude-code
gh skill install K-Dense-AI/scientific-agent-skills --agent codex
gh skill install K-Dense-AI/scientific-agent-skills --agent gemini

gh skill automatically installs to the correct directory for your agent host and records provenance metadata for supply chain integrity.

Version pinning

Pin to a specific release tag or commit SHA for reproducible installs:

# Pin to a release tag
gh skill install K-Dense-AI/scientific-agent-skills --pin v1.0.0

# Pin to a commit SHA
gh skill install K-Dense-AI/scientific-agent-skills --pin abc123def

Keeping skills up to date

# Check for updates interactively
gh skill update

# Update all installed skills
gh skill update --all

That's it! Your AI agent will automatically discover the skills and use them when relevant to your scientific tasks. You can also invoke any skill manually by mentioning the skill name in your prompt.

⚠️ Security Disclaimer

Skills can execute code and influence your coding agent's behavior. Review what you install.

Agent Skills are powerful — they can instruct your AI agent to run arbitrary code, install packages, make network requests, and modify files on your system. A malicious or poorly written skill has the potential to steer your coding agent into harmful behavior.

We take security seriously. All contributions go through a review process, and we run LLM-based security scans (via Cisco AI Defense Skill Scanner) on every skill in this repository. However, as a small team with a growing number of community contributions, we cannot guarantee that every skill has been exhaustively reviewed for all possible risks.

It is ultimately your responsibility to review the skills you install and decide which ones to trust.

We recommend the following:

Do not install everything at once. Only install the skills you actually need for your work. While installing the full collection was reasonable when K-Dense created and maintained every skill, the repository now includes many community contributions that we may not have reviewed as thoroughly.
Read the SKILL.md before installing. Each skill's documentation describes what it does, what packages it uses, and what external services it connects to. If something looks suspicious, don't install it.
Check the contribution history. Skills authored by K-Dense (K-Dense-AI) have been through our internal review process. Community-contributed skills have been reviewed to the best of our ability, but with limited resources.

Run the security scanner yourself. Before installing third-party skills, scan them locally:

uv pip install cisco-ai-skill-scanner
skill-scanner scan /path/to/skill --use-behavioral

Report anything suspicious. If you find a skill that looks malicious or behaves unexpectedly, please open an issue immediately so we can investigate.

All skills are scanned on an approximately weekly basis, and SECURITY.md is updated with the latest results. We try to address security gaps as they arise.

❤️ Support the Open Source Community

Scientific Agent Skills is powered by 50+ incredible open source projects maintained by dedicated developers and research communities worldwide. Projects like Biopython, Scanpy, RDKit, scikit-learn, PyTorch Lightning, and many others form the foundation of these skills.

If you find value in this repository, please consider supporting the projects that make it possible:

⭐ Star their repositories on GitHub
💰 Sponsor maintainers via GitHub Sponsors or NumFOCUS
📝 Cite projects in your publications
💻 Contribute code, docs, or bug reports

👉 View the full list of projects to support

⚙️ Prerequisites

Python: 3.11+ (3.12+ recommended for best compatibility)
uv: Python package manager (required for installing skill dependencies)
Client: Any agent that supports the Agent Skills standard (Cursor, Claude Code, Gemini CLI, Codex, etc.)
System: macOS, Linux, or Windows with WSL2
Dependencies: Automatically handled by individual skills (check SKILL.md files for specific requirements)

Installing uv

The skills use uv as the package manager for installing Python dependencies. Install it using the instructions for your operating system:

macOS and Linux:

curl -LsSf https://astral.sh/uv/install.sh | sh

Windows:

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Alternative (via pip):

pip install uv

After installation, verify it works by running:

uv --version

For more installation options and details, visit the official uv documentation.

💡 Quick Examples

Once you've installed the skills, you can ask your AI agent to execute complex multi-step scientific workflows. Here are some example prompts:

🧪 Drug Discovery Pipeline

Goal: Find novel EGFR inhibitors for lung cancer treatment

Prompt:

Use available skills you have access to whenever possible. Query ChEMBL for EGFR inhibitors (IC50 < 50nM), analyze structure-activity relationships 
with RDKit, generate improved analogs with datamol, perform virtual screening with DiffDock 
against AlphaFold EGFR structure, search PubMed for resistance mechanisms, check COSMIC for 
mutations, and create visualizations and a comprehensive report.

Skills Used: ChEMBL, RDKit, datamol, DiffDock, AlphaFold DB, PubMed, COSMIC, scientific visualization

Need cloud GPUs and a publication-ready report at the end? Run this on K-Dense Web free.

🔬 Single-Cell RNA-seq Analysis

Goal: Comprehensive analysis of 10X Genomics data with public data integration

Prompt:

Use available skills you have access to whenever possible. Load 10X dataset with Scanpy, perform QC and doublet removal, integrate with Cellxgene 
Census data, identify cell types using NCBI Gene markers, run differential expression with 
PyDESeq2, infer gene regulatory networks with Arboreto, enrich pathways via Reactome/KEGG, 
and identify therapeutic targets with Open Targets.

Skills Used: Scanpy, Cellxgene Census, NCBI Gene, PyDESeq2, Arboreto, Reactome, KEGG, Open Targets

Want zero-setup cloud execution and shareable outputs? Try K-Dense Web free.

🧬 Multi-Omics Biomarker Discovery

Goal: Integrate RNA-seq, proteomics, and metabolomics to predict patient outcomes

Prompt:

Use available skills you have access to whenever possible. Analyze RNA-seq with PyDESeq2, process mass spec with pyOpenMS, integrate metabolites from 
HMDB/Metabolomics Workbench, map proteins to pathways (UniProt/KEGG), find interactions via 
STRING, correlate omics layers with statsmodels, build predictive model with scikit-learn, 
and search ClinicalTrials.gov for relevant trials.

Skills Used: PyDESeq2, pyOpenMS, HMDB, Metabolomics Workbench, UniProt, KEGG, STRING, statsmodels, scikit-learn, ClinicalTrials.gov

This pipeline is heavy on compute. Run it on K-Dense Web with cloud GPUs, free to start.

🎯 Virtual Screening Campaign

Goal: Discover allosteric modulators for protein-protein interactions

Prompt:

Use available skills you have access to whenever possible. Retrieve AlphaFold structures, identify interaction interface with BioPython, search ZINC 
for allosteric candidates (MW 300-500, logP 2-4), filter with RDKit, dock with DiffDock, 
rank with DeepChem, check PubChem suppliers, search USPTO patents, and optimize leads with 
MedChem/molfeat.

Skills Used: AlphaFold DB, BioPython, ZINC, RDKit, DiffDock, DeepChem, PubChem, USPTO, MedChem, molfeat

Skip the local GPU bottleneck. Run virtual screening on K-Dense Web free.

🏥 Clinical Variant Interpretation

Goal: Analyze VCF file for hereditary cancer risk assessment

Prompt:

Use available skills you have access to whenever possible. Parse VCF with pysam, annotate variants with Ensembl VEP, query ClinVar for pathogenicity, 
check COSMIC for cancer mutations, retrieve gene info from NCBI Gene, analyze protein impact 
with UniProt, search PubMed for case reports, check ClinPGx for pharmacogenomics, generate 
clinical report with document processing tools, and find matching trials on ClinicalTrials.gov.

Skills Used: pysam, Ensembl, ClinVar, COSMIC, NCBI Gene, UniProt, PubMed, ClinPGx, Document Skills, ClinicalTrials.gov

Need a polished clinical report at the end, not just code? K-Dense Web delivers publication-ready outputs. Try it free.

🌐 Systems Biology Network Analysis

Goal: Analyze gene regulatory networks from RNA-seq data

Prompt:

Use available skills you have access to whenever possible. Query NCBI Gene for annotations, retrieve sequences from UniProt, identify interactions via 
STRING, map to Reactome/KEGG pathways, analyze topology with Torch Geometric, reconstruct 
GRNs with Arboreto, assess druggability with Open Targets, model with PyMC, visualize 
networks, and search GEO for similar patterns.

Skills Used: NCBI Gene, UniProt, STRING, Reactome, KEGG, Torch Geometric, Arboreto, Open Targets, PyMC, GEO

Want end-to-end pipelines with shareable outputs and no setup? Try K-Dense Web free.

📖 Want more examples? Check out docs/examples.md for comprehensive workflow examples and detailed use cases across all scientific domains.

🚀 Want to Skip the Setup and Just Do the Science?

Recognize any of these?

You spent more time configuring environments than running analyses
Your workflow needs a GPU your local machine does not have
You need a shareable, publication-ready figure or report, not just a script
You want to run a complex multi-step pipeline right now, without reading package docs first

If so, K-Dense Web was built for you. It is the full AI co-scientist platform: everything in this repo plus cloud GPUs, 200+ skills, and outputs you can drop directly into a paper or presentation. Zero setup required.

Feature	This Repo	K-Dense Web
Scientific Skills	135 skills	200+ skills (exclusive access)
Setup	Manual installation	Zero setup, works instantly
Compute	Your machine	Cloud GPUs and HPC included
Workflows	Prompt and code	End-to-end research pipelines
Outputs	Code and analysis	Publication-ready figures, reports, and papers
Integrations	Local tools	Lab systems, ELNs, and cloud storage

"K-Dense Web took me from raw sequencing data to a draft figure in one afternoon. What used to take three days of environment setup and scripting now just works." Computational biologist, drug discovery

Try K-Dense Web

k-dense.ai | Read the full comparison

🔬 Use Cases

🧪 Drug Discovery & Medicinal Chemistry

Virtual Screening: Screen millions of compounds from PubChem/ZINC against protein targets
Lead Optimization: Analyze structure-activity relationships with RDKit, generate analogs with datamol
ADMET Prediction: Predict absorption, distribution, metabolism, excretion, and toxicity with DeepChem
Molecular Docking: Predict binding poses and affinities with DiffDock
Bioactivity Mining: Query ChEMBL for known inhibitors and analyze SAR patterns

🧬 Bioinformatics & Genomics

Sequence Analysis: Process DNA/RNA/protein sequences with BioPython and pysam
Single-Cell Analysis: Analyze 10X Genomics data with Scanpy, identify cell types, infer GRNs with Arboreto
Variant Annotation: Annotate VCF files with Ensembl VEP, query ClinVar for pathogenicity
Variant Database Management: Build scalable VCF databases with TileDB-VCF for incremental sample addition, efficient population-scale queries, and compressed storage of genomic variant data
Gene Discovery: Query NCBI Gene, UniProt, and Ensembl for comprehensive gene information
Network Analysis: Identify protein-protein interactions via STRING, map to pathways (KEGG, Reactome)

🏥 Clinical Research & Precision Medicine

Clinical Trials: Search ClinicalTrials.gov for relevant studies, analyze eligibility criteria
Variant Interpretation: Annotate variants with ClinVar, COSMIC, and ClinPGx for pharmacogenomics
Drug Safety: Query FDA databases for adverse events, drug interactions, and recalls
Precision Therapeutics: Match patient variants to targeted therapies and clinical trials

🔬 Multi-Omics & Systems Biology

Multi-Omics Integration: Combine RNA-seq, proteomics, and metabolomics data
Pathway Analysis: Enrich differentially expressed genes in KEGG/Reactome pathways
Network Biology: Reconstruct gene regulatory networks, identify hub genes
Biomarker Discovery: Integrate multi-omics layers to predict patient outcomes

📊 Data Analysis & Visualization

Statistical Analysis: Perform hypothesis testing, power analysis, and experimental design
Publication Figures: Create publication-quality visualizations with matplotlib and seaborn
Network Visualization: Visualize biological networks with NetworkX
Report Generation: Generate comprehensive PDF reports with Document Skills

🧪 Laboratory Automation

Protocol Design: Create Opentrons protocols for automated liquid handling
LIMS Integration: Integrate with Benchling and LabArchives for data management
Workflow Automation: Automate multi-step laboratory workflows

📚 Available Skills

This repository contains 135 scientific and research skills organized across multiple domains. Each skill provides comprehensive documentation, code examples, and best practices for working with scientific libraries, databases, and tools.

Skill Categories

Note: The Python package and integration skills listed below are explicitly defined skills — curated with documentation, examples, and best practices for stronger, more reliable performance. They are not a ceiling: the agent can install and use any Python package or call any API, even without a dedicated skill. The skills listed simply make common workflows faster and more dependable.

🧬 Bioinformatics & Genomics (21+ skills)

Sequence analysis: BioPython, pysam, scikit-bio, BioServices
Single-cell analysis: Scanpy, AnnData, scvi-tools, scVelo (RNA velocity), Arboreto, Cellxgene Census
Genomic tools: gget, geniml, gtars, deepTools, FlowIO, Polars-Bio, Zarr, TileDB-VCF
Differential expression: PyDESeq2
Phylogenetics: ETE Toolkit, Phylogenetics (MAFFT, IQ-TREE 2, FastTree)

🧪 Cheminformatics & Drug Discovery (10+ skills)

Molecular manipulation: RDKit, Datamol, Molfeat
Deep learning: DeepChem, TorchDrug
Docking & screening: DiffDock
Molecular dynamics: OpenMM + MDAnalysis (MD simulation & trajectory analysis)
Cloud quantum chemistry: Rowan (pKa, docking, cofolding)
Drug-likeness: MedChem
Benchmarks: PyTDC

🔬 Proteomics & Mass Spectrometry (2 skills)

Spectral processing: matchms, pyOpenMS

🏥 Clinical Research & Precision Medicine (8+ skills)

Clinical databases: via Database Lookup (ClinicalTrials.gov, ClinVar, ClinPGx, COSMIC, FDA, cBioPortal, Monarch, and more)
Cancer genomics: DepMap (cancer dependency scores, drug sensitivity)
Cancer imaging: Imaging Data Commons (NCI radiology & pathology datasets via idc-index)
Healthcare AI: PyHealth, NeuroKit2, Clinical Decision Support
Clinical documentation: Clinical Reports, Treatment Plans

🖼️ Medical Imaging & Digital Pathology (3 skills)

DICOM processing: pydicom
Whole slide imaging: histolab, PathML

🧠 Neuroscience & Electrophysiology (1 skill)

Neural recordings: Neuropixels-Analysis (extracellular spikes, silicon probes, spike sorting)

🤖 Machine Learning & AI (16+ skills)

Deep learning: PyTorch Lightning, Transformers, Stable Baselines3, PufferLib
Classical ML: scikit-learn, scikit-survival, SHAP
Time series: aeon, TimesFM (Google's zero-shot foundation model for univariate forecasting)
Bayesian methods: PyMC
Optimization: PyMOO
Graph ML: Torch Geometric
Dimensionality reduction: UMAP-learn
Statistical modeling: statsmodels

🔮 Materials Science, Chemistry & Physics (7 skills)

Materials: Pymatgen
Metabolic modeling: COBRApy
Astronomy: Astropy
Quantum computing: Cirq, PennyLane, Qiskit, QuTiP

⚙️ Engineering & Simulation (4 skills)

Numerical computing: MATLAB/Octave
Computational fluid dynamics: FluidSim
Discrete-event simulation: SimPy
Symbolic math: SymPy

📊 Data Analysis & Visualization (16+ skills)

Visualization: Matplotlib, Seaborn, Scientific Visualization
Geospatial analysis: GeoPandas, GeoMaster (remote sensing, GIS, satellite imagery, spatial ML, 500+ examples)
Data processing: Dask, Polars, Vaex
Network analysis: NetworkX
Document processing: Document Skills (PDF, DOCX, PPTX, XLSX)
Infographics: Infographics (AI-powered professional infographic creation)
Diagrams: Markdown & Mermaid Writing (text-based diagrams as default documentation standard)
Exploratory data analysis: EDA workflows
Statistical analysis: Statistical Analysis workflows

🧪 Laboratory Automation (4 skills)

Liquid handling: PyLabRobot
Cloud lab: Ginkgo Cloud Lab (cell-free protein expression, fluorescent pixel art via autonomous RAC infrastructure)
Protocol management: Protocols.io
LIMS integration: Benchling, LabArchives

🔬 Multi-omics & Systems Biology (4+ skills)

Pathway analysis: via Database Lookup (KEGG, Reactome, STRING) and PrimeKG
Multi-omics: HypoGeniC
Data management: LaminDB

🧬 Protein Engineering & Design (3 skills)

Protein language models: ESM
Glycoengineering: Glycoengineering (N/O-glycosylation prediction, therapeutic antibody optimization)
Cloud laboratory platform: Adaptyv (automated protein testing and validation)

📚 Scientific Communication (20+ skills)

Literature: Paper Lookup (PubMed, PMC, bioRxiv, medRxiv, arXiv, OpenAlex, Crossref, Semantic Scholar, CORE, Unpaywall), Literature Review
Advanced paper search: BGPT Paper Search (25+ structured fields per paper — methods, results, sample sizes, quality scores — from full text, not just abstracts)
Web search: Parallel Web (synthesized summaries with citations)
Research notebooks: Open Notebook (self-hosted NotebookLM alternative — PDFs, videos, audio, web pages; 16+ AI providers; multi-speaker podcast generation)
Writing: Scientific Writing, Peer Review
Document processing: XLSX, MarkItDown, Document Skills
Publishing: Venue Templates
Presentations: Scientific Slides, LaTeX Posters, PPTX Posters
Diagrams: Scientific Schematics, Markdown & Mermaid Writing
Infographics: Infographics (10 types, 8 styles, colorblind-safe palettes)
Citations: Citation Management
Illustration: Generate Image (AI image generation with FLUX.2 Pro and Gemini 3 Pro (Nano Banana Pro))

🔬 Scientific Databases & Data Access (6 skills → 100+ databases total)

A unified database-lookup skill provides direct REST API access to 78 public databases across all domains. Dedicated skills cover specialized data platforms. Multi-database packages like BioServices (~40 bioinformatics services), BioPython (38 NCBI sub-databases via Entrez), and gget (20+ genomics databases) add further coverage.

Unified access: Database Lookup (78 databases spanning chemistry, genomics, clinical, pathways, patents, economics, and more — PubChem, ChEMBL, UniProt, PDB, AlphaFold, KEGG, Reactome, STRING, ClinVar, COSMIC, ClinicalTrials.gov, FDA, FRED, USPTO, SEC EDGAR, and dozens more)
Cancer genomics: DepMap (cancer cell line dependencies, drug sensitivity, gene effect profiles)
Cancer imaging: Imaging Data Commons (NCI radiology & pathology datasets via idc-index)
Knowledge graph: PrimeKG (precision medicine knowledge graph — genes, drugs, diseases, phenotypes)
Fiscal data: U.S. Treasury Fiscal Data (national debt, Treasury statements, auctions, exchange rates)
Scientific ML resource catalog: Hugging Science (curated index of datasets, models, blog posts, and interactive Spaces across 17 scientific domains — astronomy, biology, chemistry, climate, genomics, materials science, medicine, physics, scientific reasoning, and more — with usage patterns for datasets, transformers, and gradio_client)

🔧 Infrastructure & Platforms (7+ skills)

Cloud compute: Modal
GPU acceleration: Optimize for GPU (CuPy, Numba CUDA, Warp, cuDF, cuML, cuGraph, KvikIO, cuCIM, cuxfilter, cuVS, cuSpatial, RAFT)
Genomics platforms: DNAnexus, LatchBio
Microscopy: OMERO
Automation: Opentrons
Resource detection: Get Available Resources

🎓 Research Methodology & Planning (12+ skills)

Ideation: Scientific Brainstorming, Hypothesis Generation
Critical analysis: Scientific Critical Thinking, Scholar Evaluation
Scenario analysis: What-If Oracle (multi-branch possibility exploration, risk analysis, strategic options)
Multi-perspective deliberation: Consciousness Council (diverse expert viewpoints, devil's advocate analysis)
Cognitive profiling: DHDNA Profiler (extract thinking patterns and cognitive signatures from any text)
Funding: Research Grants
Discovery: Research Lookup, Paper Lookup (10 academic databases)
Market analysis: Market Research Reports

⚖️ Regulatory & Standards (1 skill)

Medical device standards: ISO 13485 Certification

📖 For complete details on all skills, see docs/scientific-skills.md

💡 Looking for practical examples? Check out docs/examples.md for comprehensive workflow examples across all scientific domains.

🤝 Contributing

We welcome contributions to expand and improve this scientific skills repository!

Ways to Contribute

✨ Add New Skills

Create skills for additional scientific packages or databases
Add integrations for scientific platforms and tools

📚 Improve Existing Skills

Enhance documentation with more examples and use cases
Add new workflows and reference materials
Improve code examples and scripts
Fix bugs or update outdated information

🐛 Report Issues

Submit bug reports with detailed reproduction steps
Suggest improvements or new features

How to Contribute

Fork the repository
Create a feature branch (git checkout -b feature/amazing-skill)
Follow the existing directory structure and documentation patterns
Ensure all new skills include comprehensive SKILL.md files
Test your examples and workflows thoroughly
Commit your changes (git commit -m 'Add amazing skill')
Push to your branch (git push origin feature/amazing-skill)
Submit a pull request with a clear description of your changes

Contribution Guidelines

✅ Adhere to the Agent Skills Specification — Every skill must follow the official spec (valid SKILL.md frontmatter, naming conventions, directory structure)
✅ Maintain consistency with existing skill documentation format
✅ Ensure all code examples are tested and functional
✅ Follow scientific best practices in examples and workflows
✅ Update relevant documentation when adding new capabilities
✅ Provide clear comments and docstrings in code
✅ Include references to official documentation

Security Scanning

All skills in this repository are security-scanned using Cisco AI Defense Skill Scanner, an open-source tool that detects prompt injection, data exfiltration, and malicious code patterns in Agent Skills.

If you are contributing a new skill, we recommend running the scanner locally before submitting a pull request:

uv pip install cisco-ai-skill-scanner
skill-scanner scan /path/to/your/skill --use-behavioral

Note: A clean scan result reduces noise in review, but does not guarantee a skill is free of all risk. Contributed skills are also reviewed manually before merging.

Recognition

Contributors are recognized in our community and may be featured in:

Repository contributors list
Special mentions in release notes
K-Dense community highlights

Your contributions help make scientific computing more accessible and enable researchers to leverage AI tools more effectively!

Support Open Source

This project builds on 50+ amazing open source projects. If you find value in these skills, please consider supporting the projects we depend on.

🔧 Troubleshooting

Common Issues

Problem: Skills not loading

Verify skill folders are in the correct directory (see Getting Started)
Each skill folder must contain a SKILL.md file
Restart your agent/IDE after copying skills
In Cursor, check Settings → Rules to confirm skills are discovered

Problem: Missing Python dependencies

Solution: Check the specific SKILL.md file for required packages
Install dependencies: uv pip install package-name

Problem: API rate limits

Solution: Many databases have rate limits. Review the specific database documentation
Consider implementing caching or batch requests

Problem: Authentication errors

Solution: Some services require API keys. Check the SKILL.md for authentication setup
Verify your credentials and permissions

Problem: Outdated examples

Solution: Report the issue via GitHub Issues
Check the official package documentation for updated syntax

❓ FAQ

General Questions

Q: Is this free to use?
A: Yes! This repository is MIT licensed. However, each individual skill has its own license specified in the license metadata field within its SKILL.md file—be sure to review and comply with those terms.

Q: Why are all skills grouped together instead of separate packages?
A: We believe good science in the age of AI is inherently interdisciplinary. Bundling all skills together makes it trivial for you (and your agent) to bridge across fields—e.g., combining genomics, cheminformatics, clinical data, and machine learning in one workflow—without worrying about which individual skills to install or wire together.

Q: Can I use this for commercial projects?
A: The repository itself is MIT licensed, which allows commercial use. However, individual skills may have different licenses—check the license field in each skill's SKILL.md file to ensure compliance with your intended use.

Q: Do all skills have the same license?
A: No. Each skill has its own license specified in the license metadata field within its SKILL.md file. These licenses may differ from the repository's MIT License. Users are responsible for reviewing and adhering to the license terms of each individual skill they use.

Q: How often is this updated?
A: We regularly update skills to reflect the latest versions of packages and APIs. Major updates are announced in release notes.

Q: Can I use this with other AI models?
A: The skills follow the open Agent Skills standard and work with any compatible agent, including Cursor, Claude Code, and Codex.

Installation & Setup

Q: Do I need all the Python packages installed?
A: No! Only install the packages you need. Each skill specifies its requirements in its SKILL.md file.

Q: What if a skill doesn't work?
A: First check the Troubleshooting section. If the issue persists, file an issue on GitHub with detailed reproduction steps.

Q: Do the skills work offline?
A: Database skills require internet access to query APIs. Package skills work offline once Python dependencies are installed.

Contributing

Q: Can I contribute my own skills?
A: Absolutely! We welcome contributions. See the Contributing section for guidelines and best practices.

Q: How do I report bugs or suggest features?
A: Open an issue on GitHub with a clear description. For bugs, include reproduction steps and expected vs actual behavior.

💬 Support

Need help? Here's how to get support:

📖 Documentation: Check the relevant SKILL.md and references/ folders
🐛 Bug Reports: Open an issue
💡 Feature Requests: Submit a feature request
💼 Enterprise Support: Contact K-Dense for commercial support

📖 Citation

If you use Scientific Agent Skills in your research or project, please cite it as:

BibTeX

@software{scientific_agent_skills_2026,
  author = {{K-Dense Inc.}},
  title = {Scientific Agent Skills: A Comprehensive Collection of Scientific Tools for AI Agents},
  year = {2026},
  url = {https://github.com/K-Dense-AI/scientific-agent-skills},
  note = {135 skills covering databases, packages, integrations, and analysis tools}
}

APA

K-Dense Inc. (2026). Scientific Agent Skills: A comprehensive collection of scientific tools for AI agents [Computer software]. https://github.com/K-Dense-AI/scientific-agent-skills

MLA

K-Dense Inc. Scientific Agent Skills: A Comprehensive Collection of Scientific Tools for AI Agents. 2026, github.com/K-Dense-AI/scientific-agent-skills.

Plain Text

Scientific Agent Skills by K-Dense Inc. (2026)
Available at: https://github.com/K-Dense-AI/scientific-agent-skills

We appreciate acknowledgment in publications, presentations, or projects that benefit from these skills!

📄 License

This project is licensed under the MIT License.

Key Points:

✅ Free for any use (commercial and noncommercial)
✅ Open source - modify, distribute, and use freely
✅ Permissive - minimal restrictions on reuse
⚠️ No warranty - provided "as is" without warranty of any kind

See LICENSE.md for full terms.

Individual Skill Licenses

⚠️ Important: Each skill has its own license specified in the license metadata field within its SKILL.md file. These licenses may differ from the repository's MIT License and may include additional terms or restrictions. Users are responsible for reviewing and adhering to the license terms of each individual skill they use.

Star History

obra/superpowers

An agentic skills framework & software development methodology that works.

Superpowers

Superpowers is a complete software development methodology for your coding agents, built on top of a set of composable skills and some initial instructions that make sure your agent uses them.

Quickstart

Give your agent Superpowers: Claude Code, Codex CLI, Codex App, Factory Droid, Gemini CLI, OpenCode, Cursor, GitHub Copilot CLI.

How it works

It starts from the moment you fire up your coding agent. As soon as it sees that you're building something, it doesn't just jump into trying to write code. Instead, it steps back and asks you what you're really trying to do.

Once it's teased a spec out of the conversation, it shows it to you in chunks short enough to actually read and digest.

After you've signed off on the design, your agent puts together an implementation plan that's clear enough for an enthusiastic junior engineer with poor taste, no judgement, no project context, and an aversion to testing to follow. It emphasizes true red/green TDD, YAGNI (You Aren't Gonna Need It), and DRY.

Next up, once you say "go", it launches a subagent-driven-development process, having agents work through each engineering task, inspecting and reviewing their work, and continuing forward. It's not uncommon for Claude to be able to work autonomously for a couple hours at a time without deviating from the plan you put together.

There's a bunch more to it, but that's the core of the system. And because the skills trigger automatically, you don't need to do anything special. Your coding agent just has Superpowers.

Sponsorship

If Superpowers has helped you do stuff that makes money and you are so inclined, I'd greatly appreciate it if you'd consider sponsoring my opensource work.

Thanks!

Jesse

Installation

Installation differs by harness. If you use more than one, install Superpowers separately for each one.

Claude Code

Superpowers is available via the official Claude plugin marketplace

Official Marketplace

Install the plugin from Anthropic's official marketplace:
```
/plugin install superpowers@claude-plugins-official
```

Superpowers Marketplace

The Superpowers marketplace provides Superpowers and some other related plugins for Claude Code.

/plugin marketplace add obra/superpowers-marketplace

Install the plugin from this marketplace:

/plugin install superpowers@superpowers-marketplace

Codex CLI

Superpowers is available via the official Codex plugin marketplace.

Open the plugin search interface:
```
/plugins
```
Search for Superpowers:
```
superpowers
```
Select Install Plugin.

Codex App

Superpowers is available via the official Codex plugin marketplace.

In the Codex app, click on Plugins in the sidebar.
You should see Superpowers in the Coding section.
Click the + next to Superpowers and follow the prompts.

Factory Droid

droid plugin marketplace add https://github.com/obra/superpowers

Install the plugin:

droid plugin install superpowers@superpowers

Gemini CLI

Install the extension:

gemini extensions install https://github.com/obra/superpowers

Update later:
```
gemini extensions update superpowers
```

OpenCode

OpenCode uses its own plugin install; install Superpowers separately even if you already use it in another harness.

Tell OpenCode:

Fetch and follow instructions from https://raw.githubusercontent.com/obra/superpowers/refs/heads/main/.opencode/INSTALL.md

Detailed docs: docs/README.opencode.md

Cursor

In Cursor Agent chat, install from marketplace:
```
/add-plugin superpowers
```
Or search for "superpowers" in the plugin marketplace.

GitHub Copilot CLI

copilot plugin marketplace add obra/superpowers-marketplace

Install the plugin:

copilot plugin install superpowers@superpowers-marketplace

The Basic Workflow

brainstorming - Activates before writing code. Refines rough ideas through questions, explores alternatives, presents design in sections for validation. Saves design document.
using-git-worktrees - Activates after design approval. Creates isolated workspace on new branch, runs project setup, verifies clean test baseline.
writing-plans - Activates with approved design. Breaks work into bite-sized tasks (2-5 minutes each). Every task has exact file paths, complete code, verification steps.
subagent-driven-development or executing-plans - Activates with plan. Dispatches fresh subagent per task with two-stage review (spec compliance, then code quality), or executes in batches with human checkpoints.
test-driven-development - Activates during implementation. Enforces RED-GREEN-REFACTOR: write failing test, watch it fail, write minimal code, watch it pass, commit. Deletes code written before tests.
requesting-code-review - Activates between tasks. Reviews against plan, reports issues by severity. Critical issues block progress.
finishing-a-development-branch - Activates when tasks complete. Verifies tests, presents options (merge/PR/keep/discard), cleans up worktree.

The agent checks for relevant skills before any task. Mandatory workflows, not suggestions.

What's Inside

Skills Library

Testing

test-driven-development - RED-GREEN-REFACTOR cycle (includes testing anti-patterns reference)

Debugging

systematic-debugging - 4-phase root cause process (includes root-cause-tracing, defense-in-depth, condition-based-waiting techniques)
verification-before-completion - Ensure it's actually fixed

Collaboration

brainstorming - Socratic design refinement
writing-plans - Detailed implementation plans
executing-plans - Batch execution with checkpoints
dispatching-parallel-agents - Concurrent subagent workflows
requesting-code-review - Pre-review checklist
receiving-code-review - Responding to feedback
using-git-worktrees - Parallel development branches
finishing-a-development-branch - Merge/PR decision workflow
subagent-driven-development - Fast iteration with two-stage review (spec compliance, then code quality)

Meta

writing-skills - Create new skills following best practices (includes testing methodology)
using-superpowers - Introduction to the skills system

Philosophy

Test-Driven Development - Write tests first, always
Systematic over ad-hoc - Process over guessing
Complexity reduction - Simplicity as primary goal
Evidence over claims - Verify before declaring success

Read the original release announcement.

Contributing

The general contribution process for Superpowers is below. Keep in mind that we don't generally accept contributions of new skills and that any updates to skills must work across all of the coding agents we support.

Fork the repository
Switch to the 'dev' branch
Create a branch for your work
Follow the writing-skills skill for creating and testing new and modified skills
Submit a PR, being sure to fill in the pull request template.

See skills/writing-skills/SKILL.md for the complete guide.

Updating

Superpowers updates are somewhat coding-agent dependent, but are often automatic.

License

MIT License - see LICENSE file for details

Community

Superpowers is built by Jesse Vincent and the rest of the folks at Prime Radiant.

Discord: Join us for community support, questions, and sharing what you're building with Superpowers
Issues: https://github.com/obra/superpowers/issues
Release announcements: Sign up to get notified about new versions

Anil-matcha/Open-Generative-AI

Open-source alternative to AI video platforms — Free AI image & video generation studio with 200+ models (Flux, Midjourney, Kling, Sora, Veo). No content filters. Self-hosted, MIT licensed.

Open Generative AI — Open-Source Alternative to AI Video Platforms

The free, open-source alternative to AI Video Platforms. Generate AI images and videos using 200+ state-of-the-art models — no content filters, no closed ecosystem, no subscription fees.

Community: Join Reddit & Discord for discussions and support

🤖 Automate media generations with AI coding agents: Generative-Media-Skills — a library of skills that let agents like Claude Code, Codex, and other coding assistants drive 200+ image/video models end-to-end (prompt → generate → edit → stitch) directly from your terminal. Perfect for building automated media pipelines without touching a UI.

Related projects

Open-source Node based workflow builder -> https://github.com/SamurAIGPT/Vibe-Workflow

Open-source AI Clipping — turn any long-form YouTube video into viral-ready vertical shorts -> https://github.com/SamurAIGPT/AI-Youtube-Shorts-Generator

Open-source AI Design Agent -> https://github.com/Anil-matcha/Open-AI-Design-Agent

🌐 Try it Online — No Install Required

Hosted version: https://muapi.ai/open-generative-ai

Use all four studios (Image, Video, Lip Sync, Cinema) directly in your browser — no Node.js, no setup. Sign up for a free account to start generating. The hosted version is always up to date with the latest models.

Follow the creator for updates

⬇️ Download Desktop App

One-click installers — no Node.js or terminal required.

Platform	Download
macOS Apple Silicon (M1/M2/M3/M4)	Open Generative AI-1.0.9-arm64.dmg
macOS Intel (x64)	Open Generative AI-1.0.9.dmg
Windows (x64)	Open Generative AI Setup 1.0.9.exe
Linux (Ubuntu x64)	v1.0.9 release (`.AppImage` / `.deb`), or build locally with `npm run electron:build:linux`.

All releases: github.com/Anil-matcha/Open-Generative-AI/releases

macOS Installation Guide

Because the app is not notarized by Apple, macOS Gatekeeper will block it on first launch. Follow these steps:

Step 1 — Mount the DMG and drag the app to /Applications

Step 2 — Open Terminal and run:

xattr -cr "/Applications/Open Generative AI.app"

Step 3 — Right-click the app in /Applications → click Open → click Open again on the dialog

You only need to do this once. After that, the app opens normally.

Alternative (no Terminal):

Try to open the app — macOS will block it
Go to System Settings → Privacy & Security
Scroll down to find "Open Generative AI was blocked"
Click Open Anyway → Open

Windows Installation — SmartScreen warning fix

Windows SmartScreen may show a warning because the installer is not code-signed:

Click More info on the SmartScreen dialog
Click Run anyway

The app will install silently to %LocalAppData% with a Start Menu shortcut.

Ubuntu / Linux Installation

Linux artifacts are available when building with Electron Builder:

# Build Linux installers (AppImage + .deb)
npm run electron:build:linux

Generated files are written to the release/ folder:

AppImage — portable, run directly after making executable:

chmod +x "release/Open Generative AI-*.AppImage"
./release/Open\ Generative\ AI-*.AppImage

.deb — install on Debian/Ubuntu:

sudo apt install ./release/open-generative-ai_*_amd64.deb

If AppImage fails to start on older systems, install libfuse2:

sudo apt install libfuse2

Ubuntu 24.04+ / AppArmor sandbox restriction

Ubuntu 24.04 and later enable a kernel security policy (apparmor_restrict_unprivileged_userns) that blocks Chromium's user-namespace sandbox. If the app fails to start silently or crashes immediately, you have two options:

Option A — Recommended: install the .deb instead. The .deb package ships an AppArmor profile that grants the required permission automatically on install with no system-wide changes.

Option B — Temporary system fix (AppImage users):

sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0

This lasts until next reboot. To make it permanent:

echo 'kernel.apparmor_restrict_unprivileged_userns=0' | sudo tee /etc/sysctl.d/99-userns.conf

Open Generative AI is a free, open-source AI image, video, cinema, and lip sync studio that brings creative workflows to everyone. No content filters, no prompt rejections, no guardrails — just full creative freedom. Powered by Muapi.ai, it supports text-to-image, image-to-image, text-to-video, image-to-video, and audio-driven lip sync generation across models like Flux, Nano Banana, Midjourney, Kling, Sora, Veo, Seedream, Infinite Talk, LTX Lipsync, Wan 2.2, and more — all from a sleek, modern interface you can self-host and customize.

Why Open Generative AI instead of other AI Video Platforms?

No filters — no content filters, no nanny guardrails, no prompt rejections
Free & open-source — no subscription, no vendor lock-in
Self-hosted — your data stays on your machine, full creative control
200+ models — text-to-image, image-to-image, text-to-video, image-to-video, lip sync
Multi-image input — feed up to 14 reference images into compatible models
Lip Sync Studio — animate portraits or sync lips to any audio with 9 dedicated models
Extensible — add your own models, modify the UI, build on top of it

For a deep dive into the technical architecture and the philosophy behind the "Infinite Budget" cinema workflow, see our comprehensive guide and roadmap.

⚡ Local Model Inference (Desktop App Only)

The desktop app supports two independent local engines. Pick whichever fits the machine you actually run on:

Engine	What it is	Best for
sd.cpp (bundled)	C++ engine from stable-diffusion.cpp, runs on the same machine as the app. Metal GPU on Apple Silicon, CUDA/Vulkan/ROCm on Linux/Windows.	Image-only models. Works on Mac M-series.
Wan2GP (BYO server)	HTTP client to a user-run Wan2GP server. The server runs Python + PyTorch on a CUDA/ROCm GPU; the desktop app only sends prompts and receives results.	Video models (Wan 2.2, Hunyuan, LTX) and large image models (Flux, Qwen-Image). NVIDIA/AMD GPU required on the server; the desktop app itself can run on a Mac.

Both engines share the same UI: open Settings → Local Models to configure each.

Engine 1 — sd.cpp (bundled)

Model	Type	Size	Notes
Z-Image Turbo ⚡	Diffusion Transformer	2.5 GB + 2.7 GB aux	8-step turbo. Heavy on memory.
Z-Image Base ⚡	Diffusion Transformer	3.5 GB + 2.7 GB aux	50-step high-quality. Heavy on memory.
Dreamshaper 8	SD 1.5	2.1 GB	20-step versatile. Lightest tested option on Mac.
Realistic Vision v5.1	SD 1.5	2.1 GB	25-step photorealistic
Anything v5	SD 1.5	2.1 GB	20-step anime/illustration
SDXL Base 1.0	SDXL	6.9 GB	30-step high-res

Z-Image models require two shared auxiliary files (downloaded once, shared across both models):

Qwen3-4B Text Encoder — 2.4 GB

FLUX VAE — 335 MB

How to use:

Open Settings → Local Models in the desktop app
Install the sd.cpp inference engine (one click — auto-downloaded)
Download your chosen model (and auxiliary files for Z-Image)
In Image Studio, click the ⚡ Local toggle next to the model selector
Select your local model and generate — no API key needed

All downloads happen inside the app. Nothing is installed system-wide.

Engine 2 — Wan2GP (remote Gradio server)

The app does not bundle Python or model weights for Wan2GP. You run Wan2GP yourself on a machine with a CUDA or ROCm GPU and point the desktop app at its URL.

# On your GPU machine
git clone https://github.com/deepbeepmeep/Wan2GP
cd Wan2GP
./install.sh                          # or install.bat on Windows
python wgp.py --listen --server-name 0.0.0.0   # binds to all interfaces

Then in the desktop app: Settings → Local Models → Wan2GP server, paste the URL (e.g. http://192.168.1.42:7860), click Test, then Save. The Wan2GP models become available — image models in Image Studio, video models reachable via the same generation API (Image Studio rejects video output explicitly; full Video Studio wiring is on the roadmap).

Model	Type	Notes
Flux.1 Dev	Image	1024px, 28 steps
Qwen Image	Image	1024px, 30 steps
Wan 2.2 (T2V / I2V)	Video	Slow on consumer GPUs
Hunyuan Video	Video	High-quality T2V
LTX Video	Video	Fastest video option

Why a separate server? Wan2GP's runtime (Sage attention, flash-attn, AWQ/GGUF kernels) is CUDA-only — there is no MPS / Apple Silicon path. Treating it as a remote server lets a Mac-only user keep the desktop app while offloading inference to a Linux/Windows GPU box, a gaming PC on the LAN, or a rented RunPod/vast.ai instance.

Local inference is only available in the desktop app. The hosted web version always uses cloud APIs.

Hardware Notes

sd.cpp runs on CPU (all platforms) and Metal GPU on Apple Silicon (M1/M2/M3/M4); CUDA/Vulkan/ROCm on Linux/Windows.
Metal GPU acceleration is built into the macOS desktop binary — significantly faster than CPU-only.
Recommended for sd.cpp Z-Image: 16 GB RAM (7.4 GB weights + 2.4 GB compute buffer). On a base 8 GB M-series Mac, Z-Image is known to hang the system — stick to SD 1.5 there.
For SD 1.5 on M2: expect ~1–2 s/step with the Metal dylib active. If you see ~10 s/step instead, the binary may have fallen back to CPU — see verification below.

Verifying the SD 1.5 path (the fastest sanity test on Mac)

If you want to confirm sd.cpp is installed correctly without going through the UI, you can drive sd-cli directly. This is the same binary the app uses.

# 1. App data layout (created on first app launch)
APP_DATA="$HOME/Library/Application Support/open-generative-ai/local-ai"
ls "$APP_DATA/bin"     # sd-cli, libstable-diffusion.dylib
ls "$APP_DATA/models"  # whatever you've downloaded

# 2. Grab a small SD 1.5 model directly (Dreamshaper 8, ~2 GB)
curl -L --fail --progress-bar \
  -o "$APP_DATA/models/DreamShaper_8_pruned.safetensors" \
  "https://huggingface.co/Lykon/DreamShaper/resolve/main/DreamShaper_8_pruned.safetensors"

# 3. Run a single 512x512 / 12-step inference
DYLD_LIBRARY_PATH="$APP_DATA/bin" "$APP_DATA/bin/sd-cli" \
  -m "$APP_DATA/models/DreamShaper_8_pruned.safetensors" \
  -p "a serene mountain lake at sunrise, oil painting" \
  -o /tmp/sd15-test.png \
  --steps 12 -H 512 -W 512 --cfg-scale 7.5 --seed 42 \
  --sampling-method euler_a

A healthy run on Apple Silicon prints total params memory size = 1969.78MB (VRAM 1969.78MB, RAM 0.00MB) (Metal-backed) and produces a coherent 512×512 PNG. If VRAM is 0.00MB instead, the dylib is CPU-only — check otool -L "$APP_DATA/bin/libstable-diffusion.dylib" | grep -i metal and reinstall the engine from Settings → Local Models if Metal is missing.

✨ Features

Image Studio — Generate images from text prompts (50+ text-to-image models) or transform existing images (55+ image-to-image models). Switches model set automatically based on whether a reference image is provided. Quality and resolution controls visible for models that support them.
Local Inference — Two engines: sd.cpp (bundled, runs on Mac/Win/Linux with Metal/CUDA/Vulkan/ROCm) for SD 1.5, SDXL, and Z-Image; and Wan2GP (BYO Gradio server) for Flux, Qwen-Image, and video models (Wan 2.2, Hunyuan, LTX). Configure both in Settings → Local Models.
Multi-Image Input — Upload up to 14 reference images for compatible edit models (Nano Banana 2 Edit, Flux Kontext Dev, GPT-4o Edit, and more). Multi-select picker with order badges, batch upload, and a "Use Selected" confirmation flow.
Video Studio — Generate videos from text prompts (40+ text-to-video models) or animate a start-frame image (60+ image-to-video models). Same intelligent mode switching as Image Studio.
Lip Sync Studio — Animate portrait images or sync lips on existing videos using audio. 9 dedicated models across two modes: portrait image + audio → talking video, and video + audio → lipsync video.
Cinema Studio — Interface for photorealistic cinematic shots with pro camera controls (Lens, Focal Length, Aperture)
Workflow Studio — Build and run multi-step AI pipelines visually. Chain image, video, and audio models into automated flows. Browse community templates, create your own with a node-based editor, and run them via an interactive playground.
Upload History — Reference images are uploaded once and stored locally. A picker panel lets you reuse any previously uploaded image across sessions — no re-uploading.
Smart Controls — Dynamic aspect ratio, resolution/quality, and duration pickers that adapt to each model's capabilities (including t2i models with resolution or quality options)
Generation History — Browse, revisit, and download all past generations (persisted in browser storage)
Image & Video Download — One-click download of generated outputs in full resolution
API Key Management — Secure API key storage in browser localStorage (never sent to any server except Muapi)
Responsive Design — Works seamlessly on desktop and mobile with dark glassmorphism UI

🖼️ Image Studio — Dual Mode

The Image Studio automatically switches between two model sets:

Mode	Trigger	Models	Prompt
Text-to-Image	Default (no image)	50+ t2i models (Flux, Nano Banana 2, Seedream 5.0, Ideogram, GPT-4o, Midjourney…)	Required
Image-to-Image	Reference image uploaded	55+ i2i models (Kontext, Nano Banana 2 Edit, Seedream 5.0 Edit, Seededit, Upscaler…)	Optional

Newly Added Models

Model	Type	Key Features
Nano Banana 2	Text-to-Image	Google Gemini 3.1 Flash Image · Resolution 1K/2K/4K · Google Search enhancement · aspect ratio `auto`
Nano Banana 2 Edit	Image-to-Image	Up to 14 reference images · Resolution 1K/2K/4K · Google Search enhancement
Seedream 5.0	Text-to-Image	ByteDance · Quality basic/high · 8 aspect ratios · up to 4K
Seedream 5.0 Edit	Image-to-Image	ByteDance · Natural language style transfer · Quality basic/high
MiniMax Image 01	Text-to-Image	MiniMax · 8 aspect ratios · up to 4 images per request · 1500 char prompt

Multi-Image Input

Models that accept multiple reference images expose a multi-select picker when active:

Model	Max Images
Nano Banana 2 Edit	14
Nano Banana Edit	10
Flux Kontext Dev I2I	10
Kling O1 Edit Image	10
GPT-4o Edit / GPT Image 1.5 Edit	10
Bytedance Seedream Edit v4 / v4.5	10
Vidu Q2 Reference to Image	7
Flux 2 Flex/Pro Edit	8
Nano Banana Pro Edit	8
Flux Kontext Pro/Max I2I	2
Wan 2.5/2.6 Image Edit	2–3
Qwen Image Edit Plus / 2511	3
GPT-4o Image to Image	5
Flux 2 Klein 4b/9b Edit	4

When a multi-image model is selected the upload trigger switches to multi-select mode:

Checkboxes with order numbers — images are sent to the model in the order you select them
Batch upload — pick multiple files at once from your file dialog
Count badge on the trigger shows how many images are active; a + badge appears when more slots are available
"Use Selected" button confirms and closes the picker

🎬 Video Studio — Dual Mode

The Video Studio follows the same pattern:

Mode	Trigger	Models	Prompt
Text-to-Video	Default (no image)	40+ t2v models (Kling, Sora, Veo, Wan, Seedance 2.0, Hailuo, Runway…)	Required
Image-to-Video	Start frame uploaded	60+ i2v models (Kling I2V, Veo3 I2V, Runway I2V, Wan I2V, Seedance 2.0 I2V, Midjourney I2V…)	Optional

Newly Added Models

Model	Type	Key Features
Seedance 2.0	Text-to-Video	ByteDance · Aspect ratios 16:9 / 9:16 / 4:3 / 3:4 · Duration 5 / 10 / 15s · Quality basic/high
Seedance 2.0 I2V	Image-to-Video	ByteDance · Animate images into video · Up to 9 reference images · Aspect ratios 16:9 / 9:16 / 4:3 / 3:4 · Duration 5 / 10 / 15s · Quality basic/high
Seedance 2.0 Extend	Video Extension	ByteDance · Seamlessly continue any Seedance 2.0 generation · Preserves style, motion & audio · Optional continuation prompt · Duration 5 / 10 / 15s · Quality basic/high
Grok Imagine T2V	Text-to-Video	xAI · Duration 6 / 10 / 15s · Modes: fun / normal / spicy · Aspect ratios 9:16 / 16:9 / 2:3 / 3:2 / 1:1
Grok Imagine I2V	Image-to-Video	xAI · Duration 6 / 10 / 15s · Modes: fun / normal / spicy · Cinematic motion from still images
MiniMax Hailuo 02 / 2.3 Standard & Pro	Text-to-Video / Image-to-Video	MiniMax · Full HD video · Multiple aspect ratios · Fast variant included

🎙️ Lip Sync Studio

The Lip Sync Studio generates audio-driven talking videos using 9 models across two input modes:

Mode	Trigger	Description
Portrait Image	Default	Upload a portrait image + audio file → animated talking video
Video	Switch to Video mode	Upload an existing video + audio file → lipsync video

Image-based Models (Portrait Image + Audio → Video)

Model	Endpoint	Resolutions	Prompt
Infinite Talk	`infinitetalk-image-to-video`	480p, 720p	Optional
Wan 2.2 Speech to Video	`wan2.2-speech-to-video`	480p, 720p	Optional
LTX 2.3 Lipsync	`ltx-2.3-lipsync`	480p, 720p, 1080p	Optional
LTX 2 19B Lipsync	`ltx-2-19b-lipsync`	480p, 720p, 1080p	Optional

Video-based Models (Video + Audio → Lipsync Video)

Model	Endpoint	Resolutions	Prompt
Sync Lipsync	`sync-lipsync`	—	—
LatentSync	`latentsync-video`	—	—
Creatify Lipsync	`creatify-lipsync`	—	—
Veed Lipsync	`veed-lipsync`	—	—
Infinite Talk V2V	`infinitetalk-video-to-video`	480p, 720p	Optional

How it works:

Select Portrait Image or Video mode using the toggle
Upload your portrait image (or video) using the image/video upload button
Upload your audio file using the audio upload button
Optionally enter a prompt to guide the motion style
Select a model and resolution (where supported), then click Generate

Generation history is saved separately in lipsync_history and pending jobs resume automatically on page reload.

🔀 Workflow Studio

The Workflow Studio lets you build and run multi-step AI pipelines without writing code.

Key capabilities:

Templates — Start from pre-built workflows (image chains, video pipelines, and more)
My Workflows — Save and manage your own custom pipelines
Community — Browse and run workflows published by other users
Node-based Builder — Drag-and-drop visual editor to connect models and route outputs between steps
Playground — Run any workflow interactively with a form UI; results render inline
API execution — Every workflow is also callable via the Muapi API

💡 Want to add workflows to your own app? Check out Vibe Workflow — the open-source workflow engine powering this feature. Drop it into any project.

🎥 Cinema Studio Controls

The Cinema Studio offers precise control over the virtual camera, translating your choices into optimized prompt modifiers:

Category	Available Options
Cameras	Modular 8K Digital, Full-Frame Cine Digital, Grand Format 70mm Film, Studio Digital S35, Classic 16mm Film, Premium Large Format Digital
Lenses	Creative Tilt, Compact Anamorphic, Extreme Macro, 70s Cinema Prime, Classic Anamorphic, Premium Modern Prime, Warm Cinema Prime, Swirl Bokeh Portrait, Vintage Prime, Halation Diffusion, Clinical Sharp Prime
Focal Lengths	8mm (Ultra-Wide), 14mm, 24mm, 35mm (Human Eye), 50mm (Portrait), 85mm (Tight Portrait)
Apertures	f/1.4 (Shallow DoF), f/4 (Balanced), f/11 (Deep Focus)

📁 Upload History & Picker

Every image you upload is saved locally (URL + thumbnail) so you never upload the same file twice:

Click the upload button to open the reference image picker
Previously uploaded images appear in a 3-column grid with thumbnails
Single-image models — click a thumbnail to instantly select and close
Multi-image models — toggle multiple thumbnails (shown with order numbers), then click Use Selected
Upload new images with the Upload files button (supports multi-file selection in multi-image mode)
Remove individual images from history with the ✕ button
History persists across browser sessions (stored in localStorage)

🚀 Quick Start

Prerequisites

Node.js (v18+)
A Muapi.ai API key

Setup

Most users want the desktop app, not this dev path. If you just want to run Open Generative AI on your machine, download a prebuilt installer instead — no Node.js required. The instructions below are for contributors building from source.

Pick the entry point that matches your goal:

Desktop app (Electron) → npm run electron:dev
Hosted web version (Next.js) → npm run dev

# Clone the repository (with submodules — required for the workflow + agent packages)
git clone --recurse-submodules https://github.com/Anil-matcha/Open-Generative-AI.git
cd Open-Generative-AI

# If you already cloned without --recurse-submodules, run this once:
# git submodule update --init --recursive

# Install dependencies + build workspace packages (studio, workflow, agents).
# This step is REQUIRED — `npm install` alone is not enough; the workspaces
# need to be built before either dev script will work.
npm run setup

# Then start ONE of:
npm run electron:dev   # Desktop app (Electron + Vite) — recommended
npm run dev            # Hosted web version (Next.js) → http://localhost:3000

You'll be prompted to enter your Muapi API key on first use (skip the key if you only plan to use local models).

Troubleshooting — Couldn't find a 'pages' directory: this means Next.js can't see the app/ folder. Confirm you're running npm run dev from the repo root (the directory that contains app/, package.json, and next.config.mjs), and that you cloned with submodules. Re-run npm run setup if packages/Vibe-Workflow or packages/Open-Poe-AI are empty.

Production Build

npm run build
npm run start

Desktop App Build

Build native desktop apps with Electron:

# macOS (DMG — Intel + Apple Silicon)
npm run electron:build

# Windows (NSIS installer — x64 + ARM64)
npm run electron:build:win

# Linux (AppImage + DEB — x64)
npm run electron:build:linux

# Both platforms in one pass
npm run electron:build:all

Installers are output to the release/ folder. Pre-built binaries are also available on the Releases page.

🏗️ Architecture

The app is a Next.js monorepo with a shared packages/studio component library.

Open-Generative-AI/
├── app/                        # Next.js App Router
│   ├── layout.js               # Root layout (Tailwind, fonts)
│   ├── page.js                 # Redirects → /studio
│   └── studio/
│       └── page.js             # Studio page — renders StandaloneShell
├── components/
│   ├── StandaloneShell.js      # Tab nav + BYOK (API key from localStorage)
│   └── ApiKeyModal.js          # API key entry modal
├── packages/
│   └── studio/                 # Shared React component library
│       └── src/
│           ├── index.js        # Exports: ImageStudio, VideoStudio, LipSyncStudio, CinemaStudio, WorkflowStudio
│           ├── models.js       # 200+ model definitions (single source of truth)
│           ├── muapi.js        # API client (named exports, apiKey as first param)
│           └── components/
│               ├── ImageStudio.jsx    # Dual-mode t2i/i2i studio
│               ├── VideoStudio.jsx    # Dual-mode t2v/i2v studio
│               ├── LipSyncStudio.jsx  # Portrait/video + audio → talking video
│               ├── CinemaStudio.jsx   # Pro studio with camera controls
│               └── WorkflowStudio.jsx # Multi-step pipeline builder & playground
├── next.config.mjs             # transpilePackages: ['studio']
├── tailwind.config.js
└── package.json                # workspaces: ["packages/studio"]

The packages/studio library is also consumed by the hosted version on muapi.ai — model updates made in packages/studio/src/models.js apply to both the self-hosted app and the hosted version automatically.

🔌 API Integration

The app communicates with Muapi.ai using a two-step pattern:

Submit — POST /api/v1/{model-endpoint} with prompt and parameters
Poll — GET /api/v1/predictions/{request_id}/result until status is completed

Authentication uses the x-api-key header. During development, a Vite proxy handles CORS by routing /api requests to https://api.muapi.ai.

File uploads use POST /api/v1/upload_file (multipart/form-data) and return a hosted URL that is passed to image-conditioned models. For multi-image models the full images_list array is forwarded to the API in one request.

Lip sync jobs use the same two-step pattern: a dedicated processLipSync() method accepts image_url or video_url alongside audio_url, dispatches to the model's endpoint, and polls until the output video URL is available.

🎨 Supported Model Categories

Category	Count	Examples
Text-to-Image	50+	Flux Dev, Nano Banana 2, Seedream 5.0, Ideogram v3, Midjourney v7, GPT-4o, SDXL
Image-to-Image	55+	Nano Banana 2 Edit (×14), Flux Kontext Pro, GPT-4o Edit, Seededit v3, Upscaler, Background Remover
Text-to-Video	40+	Kling v3, Sora 2, Veo 3, Wan 2.6, Seedance 2.0, Seedance 2.0 Extend, Seedance Pro, Hailuo 2.3, Runway Gen-3
Image-to-Video	60+	Kling v2.1 I2V, Veo3 I2V, Runway I2V, Seedance 2.0 I2V, Midjourney v7 I2V, Hunyuan I2V, Wan2.2 I2V
Lip Sync	9	Infinite Talk I2V, Wan 2.2 Speech to Video, LTX 2.3 Lipsync, LTX 2 19B Lipsync, Sync, LatentSync, Creatify, Veed, Infinite Talk V2V

🛠️ Tech Stack

Next.js 14 — App Router, server components, fast dev server
React 18 — Studio UI components
Tailwind CSS v3 — Utility-first styling
npm workspaces — Monorepo with shared packages/studio library
Muapi.ai — AI model API gateway

🤔 How is this different from other AI Video Plaforms?

Open Generative AI is a community-driven, open-source alternative that provides similar creative capabilities without the closed ecosystem:

	Other providers	Open Generative AI
Cost	Subscription-based	Free (open-source)
Content filters	Yes — prompts blocked or altered	None
Restrictions	Platform guardrails enforced	Full creative freedom
Models	Proprietary	200+ open & commercial models
Multi-image input	Limited	Up to 14 images per request
Lip sync	No	9 models, image & video modes
Hosted version	Subscription	Free at muapi.ai/open-generative-ai
Self-hosting	No	Yes
Customizable	No	Fully hackable
Data privacy	Cloud-based	Your data stays local
Source code	Closed	MIT licensed

📄 License

MIT

🙏 Credits

Built with Muapi.ai — the unified API for AI image and video generation models.

Deep Dive: For more details on the "AI Influencer" engine, upcoming "Popcorn" storyboarding features, and the future of this project, read the full technical overview.

Looking for a free, open-source AI Video Platform? Open Generative AI is an open-source AI image and video generation studio — with no content filters that you can self-host, customize, and extend.

supertone-inc/supertonic

Lightning-Fast, On-Device, Multilingual TTS — running natively via ONNX.

Supertonic — Lightning Fast, On-Device, Accurate TTS

Supertonic is a lightning-fast, on-device multilingual text-to-speech system designed for local inference with minimal overhead. Powered by ONNX Runtime, it runs entirely on your device—no cloud, no API calls, no privacy concerns.

✨ Highlights

⚡ Blazingly Fast — Low-latency, real-time synthesis across desktop, browser, mobile, and edge — fast enough to turn an entire webpage into audio in under a second
🌍 31-Language Multilingual — Synthesize directly from text across 31 languages, or pass lang="na" to let Supertonic process the text language-agnostically when you don't know the input language — no separate language adapters needed
🪶 99M-Parameter Open-Weight Model — A compact, fully open-weight checkpoint — a fraction of the size of 0.7B–2B class open TTS systems — for smaller downloads, faster cold starts, and lower memory footprint
📱 Edge-Device Ready — Runs locally on desktop, mobile, browsers, and resource-constrained hardware like Raspberry Pi or e-readers, with zero network dependency, complete privacy, and no GPU required
🔊 44.1kHz High-Quality Audio — Outputs studio-grade 44.1kHz 16-bit WAV directly, ready for production playback without any external upsampler
🎭 Expression Tags — 10 inline tags (e.g. <laugh>, <breath>, <sigh>) bring natural human nuance into generated speech without prompt engineering or reference audio
🛠️ Multi-Runtime SDKs — Ready-to-use examples through ONNX Runtime across Python, Node.js, Browser (WebGPU), Java, C++, C#, Go, Swift, iOS, Rust, and Flutter

🌍 Supported Languages (31)

Arabic (ar), Bulgarian (bg), Croatian (hr), Czech (cs), Danish (da), Dutch (nl), English (en), Estonian (et), Finnish (fi), French (fr), German (de), Greek (el), Hindi (hi), Hungarian (hu), Indonesian (id), Italian (it), Japanese (ja), Korean (ko), Latvian (lv), Lithuanian (lt), Polish (pl), Portuguese (pt), Romanian (ro), Russian (ru), Slovak (sk), Slovenian (sl), Spanish (es), Swedish (sv), Turkish (tr), Ukrainian (uk), Vietnamese (vi)

Not sure which language your text is in? Pass lang="na" and Supertonic will handle the input in a language-agnostic way — no explicit language tag required.

📰 Update News

2026.04.29 - 🎉 Supertonic 3 released with 31-language support, improved reading accuracy, fewer repeat/skip failures, and v2-compatible public ONNX assets. Demo | Models
2026.01.22 - Voice Builder is now live! Turn your voice into a deployable, edge-native TTS with permanent ownership.
2026.01.06 - 🎉 Supertonic 2 released with 5-language support. The v2 code path is preserved on the release/supertonic-2 branch.
2025.12.10 - Added supertonic PyPI package! Install via pip install supertonic. For details, visit supertonic-py documentation
2025.12.10 - Added 6 new voice styles (M3, M4, M5, F3, F4, F5). See Voices for details
2025.12.08 - Optimized ONNX models via OnnxSlim now available on Hugging Face Models
2025.11.24 - Added Flutter SDK support with macOS compatibility

Quick Start

Install the Python SDK and generate speech immediately. On the first run, Supertonic downloads the model assets from Hugging Face automatically.

pip install supertonic

Python

from supertonic import TTS

# First run downloads the model from Hugging Face automatically.
tts = TTS(auto_download=True)

style = tts.get_voice_style(voice_name="M1")

text = "Supertonic is a lightning fast, on-device TTS system."

wav, duration = tts.synthesize(
    text=text,
    lang="en",                      # Language code (e.g., "en", "ko", "na" for language-agnostic)
    voice_style=style,              # Voice style object
    total_steps=8,                  # Quality: 5 (low) to 12 (high), default 8 (medium)
    speed=1.05,                     # Speed: 0.7 (slow) to 2.0 (fast)
)
# wav: numpy array of shape (1, num_samples,) with dtype=np.float32, sampled at 44100 Hz
# duration: numpy array of shape (1,) containing the duration of the generated audio in seconds

tts.save_audio(wav, "output.wav")
# import soundfile as sf
# sf.write("output.wav", wav.squeeze(), 44100)

print(f"Generated {duration[0]:.2f}s of audio")

Getting Started

First, clone the repository:

git clone https://github.com/supertone-inc/supertonic.git
cd supertonic

Prerequisites

Before running the examples, download the ONNX models and preset voices, and place them in the assets directory:

Note: The Hugging Face repository uses Git LFS. Please ensure Git LFS is installed and initialized before cloning or pulling large model files.

macOS: brew install git-lfs && git lfs install

Generic: see https://git-lfs.com for installers

git lfs install
git clone https://huggingface.co/Supertone/supertonic-3 assets

Some language examples need native runtimes:

Go: install the ONNX Runtime C library. On macOS, brew install onnxruntime is enough; the Go example auto-detects Homebrew paths.
Java: use a JDK, not just a JRE. On macOS, brew install openjdk@17 works.
C#: targets .NET 9 and allows major-version roll-forward, so .NET 9 or newer runtimes can run it.

Then run the Python example:

cd py
uv sync
uv run example_onnx.py

This generates outputs/output.wav using the default preset voice.

Other Runtime Examples

Run Supertonic in other languages and platforms

Node.js Example (Details)

cd nodejs
npm install
npm start

Browser Example (Details)

cd web
npm install
npm run dev

Java Example (Details)

cd java
mvn clean install
mvn exec:java

C++ Example (Details)

cd cpp
mkdir build && cd build
cmake .. && cmake --build . --config Release
./example_onnx

C# Example (Details)

cd csharp
dotnet restore
dotnet run

Go Example (Details)

cd go
go mod download
go run example_onnx.go helper.go

Swift Example (Details)

cd swift
swift build -c release
.build/release/example_onnx

Rust Example (Details)

cd rust
cargo build --release
./target/release/example_onnx

iOS Example (Details)

cd ios/ExampleiOSApp
xcodegen generate
open ExampleiOSApp.xcodeproj

In Xcode: Targets → ExampleiOSApp → Signing: select your Team, then choose your iPhone as run destination and build.

Technical Details

Runtime: ONNX Runtime for cross-platform inference
Browser Support: onnxruntime-web for client-side inference
Batch Processing: Supports batch inference for improved throughput
Audio Output: Outputs 44.1kHz 16-bit WAV files

Performance Highlights

Supertonic 3 is designed for practical on-device inference: compact enough to run locally, while staying competitive with much larger open TTS systems.

Reading Accuracy

Evaluated on the Minimax-MLS-test benchmark, Supertonic 3 stays within a competitive WER/CER range against much larger open TTS models such as VoxCPM2, while preserving a lightweight on-device deployment path. Asterisked languages (*) use CER; the others use WER.

📊 Detailed per-language results (WER / CER*)

Lang	VoxCPM2	OmniVoice	Qwen3-TTS	Supertonic 2	Supertonic 3
arabic*	4.14	1.74	—	—	2.14
czech	23.73	2.40	—	—	3.02
dutch	0.84	0.77	—	—	1.47
english	2.11	2.02	2.25	2.52	2.06
finnish	2.29	3.94	—	—	5.40
french	4.41	4.74	3.82	5.09	4.89
german	0.85	0.96	0.52	—	0.86
greek	3.22	2.96	—	—	3.54
hindi*	5.85	5.14	—	—	5.34
indonesian	1.25	1.67	—	—	1.34
italian	1.74	1.29	1.40	—	1.75
japanese*	3.35	3.81	3.67	—	4.61
korean*	4.70	3.22	4.07	3.65	3.26
polish	1.30	0.64	—	—	1.63
portuguese	1.74	1.40	1.21	1.52	2.48
romanian	22.39	2.29	—	—	2.19
russian	3.31	4.53	4.48	—	3.99
spanish	1.34	0.99	0.75	1.81	1.13
turkish	0.88	2.18	—	—	1.00
ukrainian	5.85	0.71	—	—	1.23
vietnamese	1.48	0.79	—	—	4.49

Lower is better. * indicates CER (character error rate); all other rows use WER (word error rate). Dashes (—) indicate the model does not officially support the language or no result is available.

Supertonic 2 to Supertonic 3

Compared with Supertonic 2, Supertonic 3 reduces repeat and skip failures, improves speaker similarity across the shared-language set, and expands language coverage from 5 to 31 languages. It keeps the v2-compatible public ONNX interface, so existing integrations can move to v3 with the same inference contract.

Runtime Footprint

Supertonic 3 runs fast on CPU, even compared with larger baselines measured on A100 GPU, and uses substantially less memory. The open-weight fixed-voice setting does not require a GPU, which makes local, browser, and edge deployment much easier.

Model Size

At about 99M parameters across the public ONNX assets, Supertonic 3 is much smaller than 0.7B to 2B class open TTS systems. The smaller model size is a practical advantage for download size, startup time, and on-device inference.

Demo

Try it now: Experience Supertonic in your browser with our Interactive Demo, or get started with pre-trained models from Hugging Face Hub

Raspberry Pi

Watch Supertonic running on a Raspberry Pi, demonstrating on-device, real-time text-to-speech synthesis:

https://github.com/user-attachments/assets/ea66f6d6-7bc5-4308-8a88-1ce3e07400d2

E-Reader

Experience Supertonic on an Onyx Boox Go 6 e-reader in airplane mode, achieving an average RTF of 0.3× with zero network dependency:

https://github.com/user-attachments/assets/64980e58-ad91-423a-9623-78c2ffc13680

Chrome Extension

Turns any webpage into audio in under one second, delivering lightning-fast, on-device text-to-speech with zero network dependency—free, private, and effortless:

https://github.com/user-attachments/assets/cc8a45fc-5c3e-4b2c-8439-a14c3d00d91c

Programming Language Support

We provide ready-to-use TTS inference examples across multiple ecosystems:

Language/Platform	Path	Description
Python	`py/`	ONNX Runtime inference
Node.js	`nodejs/`	Server-side JavaScript
Browser	`web/`	WebGPU/WASM inference
Java	`java/`	Cross-platform JVM
C++	`cpp/`	High-performance C++
C#	`csharp/`	.NET ecosystem
Go	`go/`	Go implementation
Swift	`swift/`	macOS applications
iOS	`ios/`	Native iOS apps
Rust	`rust/`	Memory-safe systems
Flutter	`flutter/`	Cross-platform apps

For detailed usage instructions, please refer to the README.md in each language directory.

Natural Text Handling

Supertonic is designed to handle complex, real-world text inputs that contain natural prose, punctuation, abbreviations, and proper nouns.

🎧 View audio samples more easily: Check out our Interactive Demo for a better viewing experience of all audio examples

Overview of Test Cases:

Category	Key Challenges	Supertonic	ElevenLabs	OpenAI	Gemini	Microsoft
Financial Expression	Decimal currency, abbreviated magnitudes (M, K), currency symbols, currency codes	✅	❌	❌	❌	❌
Phone Number	Area codes, hyphens, extensions (ext.)	✅	❌	❌	❌	❌
Technical Unit	Decimal numbers with units, abbreviated technical notations	✅	❌	❌	❌	❌

Example 1: Financial Expression

Text:

"The startup secured $5.2M in venture capital, a huge leap from their initial $450K seed round."

Challenges:

Decimal point in currency ($5.2M should be read as "five point two million")
Abbreviated magnitude units (M for million, K for thousand)
Currency symbol ($) that needs to be properly pronounced as "dollars"

Audio Samples:

System	Result	Audio Sample
Supertonic	✅	🎧 Play Audio
ElevenLabs Flash v2.5	❌	🎧 Play Audio
OpenAI TTS-1	❌	🎧 Play Audio
Gemini 2.5 Flash TTS	❌	🎧 Play Audio
VibeVoice Realtime 0.5B	❌	🎧 Play Audio

Example 2: Phone Number

Text:

"You can reach the hotel front desk at (212) 555-0142 ext. 402 anytime."

Challenges:

Area code in parentheses that should be read as separate digits
Phone number with hyphen separator (555-0142)
Abbreviated extension notation (ext.)
Extension number (402)

Audio Samples:

System	Result	Audio Sample
Supertonic	✅	🎧 Play Audio
ElevenLabs Flash v2.5	❌	🎧 Play Audio
OpenAI TTS-1	❌	🎧 Play Audio
Gemini 2.5 Flash TTS	❌	🎧 Play Audio
VibeVoice Realtime 0.5B	❌	🎧 Play Audio

Example 3: Technical Unit

Text:

"Our drone battery lasts 2.3h when flying at 30kph with full camera payload."

Challenges:

Decimal time duration with abbreviation (2.3h = two point three hours)
Speed unit with abbreviation (30kph = thirty kilometers per hour)
Technical abbreviations (h for hours, kph for kilometers per hour)
Technical/engineering context requiring proper pronunciation

Audio Samples:

System	Result	Audio Sample
Supertonic	✅	🎧 Play Audio
ElevenLabs Flash v2.5	❌	🎧 Play Audio
OpenAI TTS-1	❌	🎧 Play Audio
Gemini 2.5 Flash TTS	❌	🎧 Play Audio
VibeVoice Realtime 0.5B	❌	🎧 Play Audio

Note: These samples demonstrate how each system handles text normalization and pronunciation of complex expressions without requiring pre-processing or phonetic annotations.

Built with Supertonic

Project	Description	Links
TLDRL	Free, on-device TTS extension for reading any webpage	Chrome
Read Aloud	Open-source TTS browser extension	Chrome · Edge · GitHub
PageEcho	E-Book reader app for iOS	App Store
VoiceChat	On-device voice-to-voice LLM chatbot in the browser	Demo · GitHub
OmniAvatar	Talking avatar video generator from photo + speech	Demo
CopiloTTS	Kotlin Multiplatform TTS SDK via ONNX Runtime	GitHub
Voice Mixer	PyQt5 tool for mixing and modifying voice styles	GitHub
Supertonic MNN	Lightweight library based on MNN (fp32/fp16/int8)	GitHub · PyPI
Transformers.js	Hugging Face's JS library with Supertonic support	GitHub PR · Demo
Pinokio	1-click localhost cloud for Mac, Windows, and Linux	Pinokio · GitHub

Models & Versions

	Supertonic 3	Supertonic 2	Supertonic 1
Status	🟢 Latest	Stable	Legacy
Parameters	~99M	~66M	~66M
Languages	31	5	1 (en)
Expression Tags	✅ 10 tags	—	—
Code	main	release/supertonic-2	—
Weights	🤗 HF	🤗 HF	🤗 HF
Interactive Demo	🤗 Space	🤗 Space	🤗 Space
Audio Samples	DemoPage	—	DemoPage

Citation

The following papers describe the core technologies used in Supertonic. If you use this system in your research or find these techniques useful, please consider citing the relevant papers:

SupertonicTTS: Main Architecture

This paper introduces the overall architecture of SupertonicTTS, including the speech autoencoder, flow-matching based text-to-latent module, and efficient design choices.

@article{kim2025supertonic,
  title={SupertonicTTS: Towards Highly Efficient and Streamlined Text-to-Speech System},
  author={Kim, Hyeongju and Yang, Jinhyeok and Yu, Yechan and Ji, Seunghun and Morton, Jacob and Bous, Frederik and Byun, Joon and Lee, Juheon},
  journal={arXiv preprint arXiv:2503.23108},
  year={2025},
  url={https://arxiv.org/abs/2503.23108}
}

Length-Aware RoPE: Text-Speech Alignment

This paper presents Length-Aware Rotary Position Embedding (LARoPE), which improves text-speech alignment in cross-attention mechanisms.

@article{kim2025larope,
  title={Length-Aware Rotary Position Embedding for Text-Speech Alignment},
  author={Kim, Hyeongju and Lee, Juheon and Yang, Jinhyeok and Morton, Jacob},
  journal={arXiv preprint arXiv:2509.11084},
  year={2025},
  url={https://arxiv.org/abs/2509.11084}
}

Self-Purifying Flow Matching: Training with Noisy Labels

This paper describes the self-purification technique for training flow matching models robustly with noisy or unreliable labels.

@article{kim2025spfm,
  title={Training Flow Matching Models with Reliable Labels via Self-Purification},
  author={Kim, Hyeongju and Yu, Yechan and Yi, June Young and Lee, Juheon},
  journal={arXiv preprint arXiv:2509.19091},
  year={2025},
  url={https://arxiv.org/abs/2509.19091}
}

License

This project's sample code is released under the MIT License. - see the LICENSE for details.

The accompanying model is released under the OpenRAIL-M License. - see the LICENSE file for details.

This model was trained using PyTorch, which is licensed under the BSD 3-Clause License but is not redistributed with this project. - see the LICENSE for details.

tinyhumansai/openhuman

Your Personal AI super intelligence. Private, Simple and extremely powerful.

OpenHuman

OpenHuman is your Personal AI super intelligence. Private, Simple and extremely powerful.

Discord • Reddit • X/Twitter • Docs • Follow @senamakel (Creator)

🇺🇸 English | 🇨🇳 简体中文

Early Beta: Under active development. Expect rough edges.

To install or get started, either download from the website over at tinyhumans.ai/openhuman or run

# Download DMG, EXEs over at https://tinyhumans.ai/openhuman or run in from your terminal

# For macOS or Linux x64
curl -fsSL https://raw.githubusercontent.com/tinyhumansai/openhuman/main/scripts/install.sh | bash

# For Windows
irm https://raw.githubusercontent.com/tinyhumansai/openhuman/main/scripts/install.ps1 | iex

What is OpenHuman?

OpenHuman is an open-source agentic assistant designed to integrate with you in your daily life. Each bullet links to the deeper writeup in the docs.

Simple, UI-first & Human A clean desktop experience and short onboarding paths take you from install to a working agent in a few clicks — no config-first setup, no terminal required. The agent has a face: a desktop mascot that speaks, reacts to its surroundings, joins your Google Meets as a real participant, remembers you across weeks, and keeps thinking in the background even when you've stopped typing.
118+ third-party integrations with auto-fetch: plug into Gmail, Notion, GitHub, Slack, Stripe, Calendar, Drive, Linear, Jira and the rest of your stack with one-click OAuth. Every connection is exposed to the agent as a typed tool, and every twenty minutes the core walks each active connection and pulls fresh data into the memory tree. No prompts, no polling loops you have to write, so the agent already has tomorrow's context this morning.
Memory Tree + Obsidian Wiki: a local-first knowledge base built from your data and your activity. Everything you connect is canonicalized into ≤3k-token Markdown chunks, scored, and folded into hierarchical summary trees stored in SQLite on your machine. The same chunks land as .md files in an Obsidian-compatible vault you can open, browse and edit, inspired by Karpathy's obsidian-wiki workflow.
Batteries included: web search, a web-fetch scraper, a full coder toolset (filesystem, git, lint, test, grep), and native voice (STT in, ElevenLabs TTS out, mascot lip-sync, live Google Meet agent) are wired in by default. Model routing sends each task to the right LLM (reasoning, fast, or vision) under one subscription. No "install a plugin to read files" friction. Optional local AI via Ollama for on-device workloads.
Smart token compression (TokenJuice): every tool call, scrape result, email body, and search payload is run through a token compression layer before it touches any LLM Model. HTML is converted to Markdown, long URLs are shortened, non-ASCII characters are removed etc... You get the same information but at a fraction of the tokens. Reducing cost & latency by up to 80%.
Messaging channels and privacy & security: inbound/outbound across the channels you already use, with workflow data that stays on device, encrypted locally, treated as yours.

Contributing from source

New contributor? Start with CONTRIBUTING.md for the fork/PR workflow and local validation commands. The short path is:

Install Git, Node.js 24+, pnpm 10.10.0, Rust 1.93.0 (rustfmt + clippy), CMake, Ninja, ripgrep, and the platform desktop build prerequisites.
Fork and clone the repo, then run git submodule update --init --recursive before pnpm install so the vendored Tauri/CEF sources are present.
Use pnpm dev for web-only UI work, pnpm --filter openhuman-app dev:app for the desktop shell, and focused checks such as pnpm typecheck, pnpm format:check, and cargo check -p openhuman --lib before opening a PR.

Deeper docs: Architecture · Getting Set Up · Cloud Deploy.

Context in minutes, not weeks

OpenHuman is the first agent harness that gets to know you in minutes. Inspired by Karpathy's LLM Knowledgebase. Most agents start cold. Hermes learns by watching you work; OpenClaw waits for plugins to ferry context in. Either way, you spend days or weeks before the agent knows enough about your stack to be genuinely useful.

OpenHuman summarizes and compresses all your documents, emails & chats; and creates a memory graph that lets your agent remember everything about you.

OpenHuman skips the wait. Connect your accounts, let auto-fetch pull data locally on a 20-minute loop, and then have Memory Trees compress everything into Markdown files stored intelligently in a Karpathy-style Obsidian wiki.

In just one sync pass, the agent has full (compressed) context of your inbox, your calendar, your repos, your docs, your messages. No training period. No "give it a few weeks.". It becomes you, controlled by you.

Already self-host agentmemory across other coding agents? OpenHuman ships an optional Memory backend that proxies to it — set memory.backend = "agentmemory" in config.toml and the same durable store powers OpenHuman alongside Claude Code, Cursor, Codex, and OpenCode. See the agentmemory backend page for setup.

OpenHuman vs Other Agent Harnesses

High-level comparison (products evolve, so verify against each vendor). OpenHuman is built to minimize vendor sprawl, keep workflow knowledge on-device, and give the agent a persistent memory of your data, not only chat.

	Claude Cowork	OpenClaw	Hermes Agent	OpenHuman
Open-source	🚫 Proprietary	✅ MIT	✅ MIT	✅ GNU
Simple to start	✅ Desktop + CLI	⚠️ Terminal-first	⚠️ Terminal-first	✅ Clean UI, minutes
Cost	⚠️ Sub + add-ons	⚠️ BYO models	⚠️ BYO models	✅ One sub + TokenJuice
Memory	✅ Chat-scoped	⚠️ Plugin-reliant	✅ Self-learning	🚀 Memory Tree + Obsidian vault, optional agentmemory backend
Integrations	⚠️ Few connectors	⚠️ BYO	⚠️ BYO	🚀 118+ via OAuth
Auto-fetch	🚫 None	🚫 None	🚫 None	✅ 20-min sync into memory
API sprawl	🚫 Extra keys	🚫 BYOK	🚫 Multi-vendor	✅ One account
Model routing	🚫 Single model	⚠️ Manual	⚠️ Manual	✅ Built-in
Native tools	✅ Code-only	✅ Code-only	✅ Code-only	✅ Code + search + scraper + voice

Star us on GitHub

Building toward AGI and artificial consciousness? Star the repo and help others find the path.

Contributors Hall of Fame

Show some love and end up in the hall of fame. Contributors get free merch and special access to our Discord.

ruvnet/RuView

π RuView turns commodity WiFi signals into real-time spatial intelligence, vital sign monitoring, and presence detection — all without a single pixel of video.

π RuView

Beta Software — Under active development. APIs and firmware may change. Known limitations:

ESP32-C3 and original ESP32 are not supported (single-core, insufficient for CSI DSP)

Single ESP32 deployments have limited spatial resolution — use 2+ nodes or add a Cognitum Seed for best results

Camera-free pose accuracy is limited (PCK@20 ≈ 2.5% with proxy labels) — camera ground-truth training targets 35%+ PCK@20; the pipeline is implemented, but the data-collection and evaluation phases (ADR-079 P7–P9) are still pending, so no measured camera-supervised PCK@20 has been published yet

Contributions and bug reports welcome at Issues.

See through walls with WiFi

Turn ordinary WiFi into a spatial intelligence / sensing system. Detect people, measure breathing and heart rate, track movement, and monitor rooms — through walls, in the dark, with no cameras or wearables. Just physics.

π RuView is a WiFi sensing platform that turns radio signals into spatial intelligence.

Every WiFi router already fills your space with radio waves. When people move, breathe, or even sit still, they disturb those waves in measurable ways. RuView captures these disturbances using Channel State Information (CSI) from low-cost ESP32 sensors and turns them into actionable data: who's there, what they're doing, and whether they're okay.

What it senses:

Presence and occupancy — detect people through walls, count them, track entries and exits
Vital signs — breathing rate and heart rate, contactless, while sleeping or sitting
Activity recognition — walking, sitting, gestures, falls — from temporal CSI patterns
Environment mapping — RF fingerprinting identifies rooms, detects moved furniture, spots new objects
Sleep quality — overnight monitoring with sleep stage classification and apnea screening

Built on RuVector and Cognitum Seed, RuView runs entirely on edge hardware — an ESP32 mesh (as low as $9 per node) paired with a Cognitum Seed for persistent memory, cryptographic attestation, and AI integration. No cloud, no cameras, no internet required.

The system learns each environment locally using spiking neural networks that adapt in under 30 seconds, with multi-frequency mesh scanning across 6 WiFi channels that uses your neighbors' routers as free radar illuminators. Every measurement is cryptographically attested via an Ed25519 witness chain.

RuView also supports pose estimation (17 COCO keypoints via the WiFlow architecture), trained entirely without cameras using 10 sensor signals — a technique pioneered from the original DensePose From WiFi research at Carnegie Mellon University.

Built for low-power edge applications

Edge modules are small programs that run directly on the ESP32 sensor — no internet needed, no cloud fees, instant response.

What How Speed

🦴 Pose estimation CSI subcarrier amplitude/phase → 17 COCO keypoints 171K emb/s (M4 Pro)

🫁 Breathing detection Bandpass 0.1-0.5 Hz → zero-crossing BPM 6-30 BPM

💓 Heart rate Bandpass 0.8-2.0 Hz → zero-crossing BPM 40-120 BPM

👤 Presence sensing Trained model + PIR fusion — 100% accuracy 0.012 ms latency

🧱 Through-wall Fresnel zone geometry + multipath modeling Up to 5m depth

🧠 Edge intelligence 8-dim feature vectors + RVF store on Cognitum Seed $140 total BOM

🎯 Camera-free training 10 sensor signals, no labels needed 84s on M4 Pro

📷 Camera-supervised training MediaPipe + ESP32 CSI → 35%+ PCK@20 target (ADR-079; eval phases pending) ~19 min on laptop (pipeline)

📡 Multi-frequency mesh Channel hopping across 6 bands, neighbor APs as illuminators 3x sensing bandwidth

🌐 3D point cloud (optional fusion) Camera depth (MiDaS) + WiFi CSI + mmWave radar → unified spatial model 22 ms pipeline · 19K+ points/frame

What	How	Speed
🦴 Pose estimation	CSI subcarrier amplitude/phase → 17 COCO keypoints	171K emb/s (M4 Pro)
🫁 Breathing detection	Bandpass 0.1-0.5 Hz → zero-crossing BPM	6-30 BPM
💓 Heart rate	Bandpass 0.8-2.0 Hz → zero-crossing BPM	40-120 BPM
👤 Presence sensing	Trained model + PIR fusion — 100% accuracy	0.012 ms latency
🧱 Through-wall	Fresnel zone geometry + multipath modeling	Up to 5m depth
🧠 Edge intelligence	8-dim feature vectors + RVF store on Cognitum Seed	$140 total BOM
🎯 Camera-free training	10 sensor signals, no labels needed	84s on M4 Pro
📷 Camera-supervised training	MediaPipe + ESP32 CSI → 35%+ PCK@20 target (ADR-079; eval phases pending)	~19 min on laptop (pipeline)
📡 Multi-frequency mesh	Channel hopping across 6 bands, neighbor APs as illuminators	3x sensing bandwidth
🌐 3D point cloud (optional fusion)	Camera depth (MiDaS) + WiFi CSI + mmWave radar → unified spatial model	22 ms pipeline · 19K+ points/frame

# Option 1: Docker (simulated data, no hardware needed)
docker pull ruvnet/wifi-densepose:latest
docker run -p 3000:3000 ruvnet/wifi-densepose:latest
# Open http://localhost:3000

# Option 2: Live sensing with ESP32-S3 hardware ($9)
# Flash firmware, provision WiFi, and start sensing:
python -m esptool --chip esp32s3 --port COM9 --baud 460800 \
  write_flash 0x0 bootloader.bin 0x8000 partition-table.bin \
  0xf000 ota_data_initial.bin 0x20000 esp32-csi-node.bin
python firmware/esp32-csi-node/provision.py --port COM9 \
  --ssid "YourWiFi" --password "secret" --target-ip 192.168.1.20

# Option 3: Full system with Cognitum Seed ($140)
# ESP32 streams CSI → bridge forwards to Seed for persistent storage + kNN + witness chain
node scripts/rf-scan.js --port 5006           # Live RF room scan
node scripts/snn-csi-processor.js --port 5006  # SNN real-time learning
node scripts/mincut-person-counter.js --port 5006  # Correct person counting

Note

CSI-capable hardware recommended. Presence, vital signs, through-wall sensing, and all advanced capabilities require Channel State Information (CSI) from an ESP32-S3 ($9) or research NIC. The Docker image runs with simulated data for evaluation. Consumer WiFi laptops provide RSSI-only presence detection.

Hardware options for live CSI capture:

Option Hardware Cost Full CSI Capabilities

ESP32 + Cognitum Seed (recommended) ESP32-S3 + Cognitum Seed ~$140 Yes Pose, breathing, heartbeat, motion, presence + persistent vector store, kNN search, witness chain, MCP proxy

ESP32 Mesh 3-6x ESP32-S3 + WiFi router ~$54 Yes Pose, breathing, heartbeat, motion, presence

Research NIC Intel 5300 / Atheros AR9580 ~$50-100 Yes Full CSI with 3x3 MIMO

Any WiFi Windows, macOS, or Linux laptop $0 No RSSI-only: coarse presence and motion

No hardware? Verify the signal processing pipeline with the deterministic reference signal: python archive/v1/data/proof/verify.py

Option	Hardware	Cost	Full CSI	Capabilities
ESP32 + Cognitum Seed (recommended)	ESP32-S3 + Cognitum Seed	~$140	Yes	Pose, breathing, heartbeat, motion, presence + persistent vector store, kNN search, witness chain, MCP proxy
ESP32 Mesh	3-6x ESP32-S3 + WiFi router	~$54	Yes	Pose, breathing, heartbeat, motion, presence
Research NIC	Intel 5300 / Atheros AR9580	~$50-100	Yes	Full CSI with 3x3 MIMO
Any WiFi	Windows, macOS, or Linux laptop	$0	No	RSSI-only: coarse presence and motion

Real-time pose skeleton from WiFi CSI signals — no cameras, no wearables

▶ Live Observatory Demo | ▶ Dual-Modal Pose Fusion Demo | ▶ Live 3D Point Cloud

The server is optional for visualization and aggregation — the ESP32 runs independently for presence detection, vital signs, and fall alerts.

Live ESP32 pipeline: Connect an ESP32-S3 node → run the sensing server → open the pose fusion demo for real-time dual-modal pose estimation (webcam + WiFi CSI). See ADR-059.

🔬 How It Works

WiFi routers flood every room with radio waves. When a person moves — or even breathes — those waves scatter differently. WiFi DensePose reads that scattering pattern and reconstructs what happened:

WiFi Router → radio waves pass through room → hit human body → scatter
    ↓
ESP32 mesh (4-6 nodes) captures CSI on channels 1/6/11 via TDM protocol
    ↓
Multi-Band Fusion: 3 channels × 56 subcarriers = 168 virtual subcarriers per link
    ↓
Multistatic Fusion: N×(N-1) links → attention-weighted cross-viewpoint embedding
    ↓
Coherence Gate: accept/reject measurements → stable for days without tuning
    ↓
Signal Processing: Hampel, SpotFi, Fresnel, BVP, spectrogram → clean features
    ↓
AI Backbone (RuVector): attention, graph algorithms, compression, field model
    ↓
Signal-Line Protocol (CRV): 6-stage gestalt → sensory → topology → coherence → search → model
    ↓
Neural Network: processed signals → 17 body keypoints + vital signs + room model
    ↓
Output: real-time pose, breathing, heart rate, room fingerprint, drift alerts

No training cameras required — the Self-Learning system (ADR-024) bootstraps from raw WiFi data alone. MERIDIAN (ADR-027) ensures the model works in any room, not just the one it trained in.

🏢 Use Cases & Applications

WiFi sensing works anywhere WiFi exists. No new hardware in most cases — just software on existing access points or a $8 ESP32 add-on. Because there are no cameras, deployments avoid privacy regulations (GDPR video, HIPAA imaging) by design.

Scaling: Each AP distinguishes ~3-5 people (56 subcarriers). Multi-AP multiplies linearly — a 4-AP retail mesh covers ~15-20 occupants. No hard software limit; the practical ceiling is signal physics.

	Why WiFi sensing wins	Traditional alternative
🔒	No video, no GDPR/HIPAA imaging rules	Cameras require consent, signage, data retention policies
🧱	Works through walls, shelving, debris	Cameras need line-of-sight per room
🌙	Works in total darkness	Cameras need IR or visible light
💰	$0-$8 per zone (existing WiFi or ESP32)	Camera systems: $200-$2,000 per zone
🔌	WiFi already deployed everywhere	PIR/radar sensors require new wiring per room

🏥 Everyday — Healthcare, retail, office, hospitality (commodity WiFi)

Use Case	What It Does	Hardware	Key Metric	Edge Module
Elderly care / assisted living	Fall detection, nighttime activity monitoring, breathing rate during sleep — no wearable compliance needed	1 ESP32-S3 per room ($8)	Fall alert <2s	Sleep Apnea, Gait Analysis
Hospital patient monitoring	Continuous breathing + heart rate for non-critical beds without wired sensors; nurse alert on anomaly	1-2 APs per ward	Breathing: 6-30 BPM	Respiratory Distress, Cardiac Arrhythmia
Emergency room triage	Automated occupancy count + wait-time estimation; detect patient distress (abnormal breathing) in waiting areas	Existing hospital WiFi	Occupancy accuracy >95%	Queue Length, Panic Motion
Retail occupancy & flow	Real-time foot traffic, dwell time by zone, queue length — no cameras, no opt-in, GDPR-friendly	Existing store WiFi + 1 ESP32	Dwell resolution ~1m	Customer Flow, Dwell Heatmap
Office space utilization	Which desks/rooms are actually occupied, meeting room no-shows, HVAC optimization based on real presence	Existing enterprise WiFi	Presence latency <1s	Meeting Room, HVAC Presence
Hotel & hospitality	Room occupancy without door sensors, minibar/bathroom usage patterns, energy savings on empty rooms	Existing hotel WiFi	15-30% HVAC savings	Energy Audit, Lighting Zones
Restaurants & food service	Table turnover tracking, kitchen staff presence, restroom occupancy displays — no cameras in dining areas	Existing WiFi	Queue wait ±30s	Table Turnover, Queue Length
Parking garages	Pedestrian presence in stairwells and elevators where cameras have blind spots; security alert if someone lingers	Existing WiFi	Through-concrete walls	Loitering, Elevator Count

🏟️ Specialized — Events, fitness, education, civic (CSI-capable hardware)

Use Case	What It Does	Hardware	Key Metric	Edge Module
Smart home automation	Room-level presence triggers (lights, HVAC, music) that work through walls — no dead zones, no motion-sensor timeouts	2-3 ESP32-S3 nodes ($24)	Through-wall range ~5m	HVAC Presence, Lighting Zones
Fitness & sports	Rep counting, posture correction, breathing cadence during exercise — no wearable, no camera in locker rooms	3+ ESP32-S3 mesh	Pose: 17 keypoints	Breathing Sync, Gait Analysis
Childcare & schools	Naptime breathing monitoring, playground headcount, restricted-area alerts — privacy-safe for minors	2-4 ESP32-S3 per zone	Breathing: ±1 BPM	Sleep Apnea, Perimeter Breach
Event venues & concerts	Crowd density mapping, crush-risk detection via breathing compression, emergency evacuation flow tracking	Multi-AP mesh (4-8 APs)	Density per m²	Customer Flow, Panic Motion
Stadiums & arenas	Section-level occupancy for dynamic pricing, concession staffing, emergency egress flow modeling	Enterprise AP grid	15-20 per AP mesh	Dwell Heatmap, Queue Length
Houses of worship	Attendance counting without facial recognition — privacy-sensitive congregations, multi-room campus tracking	Existing WiFi	Zone-level accuracy	Elevator Count, Energy Audit
Warehouse & logistics	Worker safety zones, forklift proximity alerts, occupancy in hazardous areas — works through shelving and pallets	Industrial AP mesh	Alert latency <500ms	Forklift Proximity, Confined Space
Civic infrastructure	Public restroom occupancy (no cameras possible), subway platform crowding, shelter headcount during emergencies	Municipal WiFi + ESP32	Real-time headcount	Customer Flow, Loitering
Museums & galleries	Visitor flow heatmaps, exhibit dwell time, crowd bottleneck alerts — no cameras near artwork (flash/theft risk)	Existing WiFi	Zone dwell ±5s	Dwell Heatmap, Shelf Engagement

🤖 Robotics & Industrial — Autonomous systems, manufacturing, android spatial awareness

WiFi sensing gives robots and autonomous systems a spatial awareness layer that works where LIDAR and cameras fail — through dust, smoke, fog, and around corners. The CSI signal field acts as a "sixth sense" for detecting humans in the environment without requiring line-of-sight.

Use Case	What It Does	Hardware	Key Metric	Edge Module
Cobot safety zones	Detect human presence near collaborative robots — auto-slow or stop before contact, even behind obstructions	2-3 ESP32-S3 per cell	Presence latency <100ms	Forklift Proximity, Perimeter Breach
Warehouse AMR navigation	Autonomous mobile robots sense humans around blind corners, through shelving racks — no LIDAR occlusion	ESP32 mesh along aisles	Through-shelf detection	Forklift Proximity, Loitering
Android / humanoid spatial awareness	Ambient human pose sensing for social robots — detect gestures, approach direction, and personal space without cameras always on	Onboard ESP32-S3 module	17-keypoint pose	Gesture Language, Emotion Detection
Manufacturing line monitoring	Worker presence at each station, ergonomic posture alerts, headcount for shift compliance — works through equipment	Industrial AP per zone	Pose + breathing	Confined Space, Gait Analysis
Construction site safety	Exclusion zone enforcement around heavy machinery, fall detection from scaffolding, personnel headcount	Ruggedized ESP32 mesh	Alert <2s, through-dust	Panic Motion, Structural Vibration
Agricultural robotics	Detect farm workers near autonomous harvesters in dusty/foggy field conditions where cameras are unreliable	Weatherproof ESP32 nodes	Range ~10m open field	Forklift Proximity, Rain Detection
Drone landing zones	Verify landing area is clear of humans — WiFi sensing works in rain, dust, and low light where downward cameras fail	Ground ESP32 nodes	Presence: >95% accuracy	Perimeter Breach, Tailgating
Clean room monitoring	Personnel tracking without cameras (particle contamination risk from camera fans) — gown compliance via pose	Existing cleanroom WiFi	No particulate emission	Clean Room, Livestock Monitor

🔥 Extreme — Through-wall, disaster, defense, underground

These scenarios exploit WiFi's ability to penetrate solid materials — concrete, rubble, earth — where no optical or infrared sensor can reach. The WiFi-Mat disaster module (ADR-001) is specifically designed for this tier.

Use Case	What It Does	Hardware	Key Metric	Edge Module
Search & rescue (WiFi-Mat)	Detect survivors through rubble/debris via breathing signature, START triage color classification, 3D localization	Portable ESP32 mesh + laptop	Through 30cm concrete	Respiratory Distress, Seizure Detection
Firefighting	Locate occupants through smoke and walls before entry; breathing detection confirms life signs remotely	Portable mesh on truck	Works in zero visibility	Sleep Apnea, Panic Motion
Prison & secure facilities	Cell occupancy verification, distress detection (abnormal vitals), perimeter sensing — no camera blind spots	Dedicated AP infrastructure	24/7 vital signs	Cardiac Arrhythmia, Loitering
Military / tactical	Through-wall personnel detection, room clearing confirmation, hostage vital signs at standoff distance	Directional WiFi + custom FW	Range: 5m through wall	Perimeter Breach, Weapon Detection
Border & perimeter security	Detect human presence in tunnels, behind fences, in vehicles — passive sensing, no active illumination to reveal position	Concealed ESP32 mesh	Passive / covert	Perimeter Breach, Tailgating
Mining & underground	Worker presence in tunnels where GPS/cameras fail, breathing detection after collapse, headcount at safety points	Ruggedized ESP32 mesh	Through rock/earth	Confined Space, Respiratory Distress
Maritime & naval	Below-deck personnel tracking through steel bulkheads (limited range, requires tuning), man-overboard detection	Ship WiFi + ESP32	Through 1-2 bulkheads	Structural Vibration, Panic Motion
Wildlife research	Non-invasive animal activity monitoring in enclosures or dens — no light pollution, no visual disturbance	Weatherproof ESP32 nodes	Zero light emission	Livestock Monitor, Dream Stage

🧩 Edge Intelligence (ADR-041) — 60 WASM modules across 13 categories, all implemented (609 tests)

Small programs that run directly on the ESP32 sensor — no internet needed, no cloud fees, instant response. Each module is a tiny WASM file (5-30 KB) that you upload to the device over-the-air. It reads WiFi signal data and makes decisions locally in under 10 ms. ADR-041 defines 60 modules across 13 categories — all 60 are implemented with 609 tests passing.

	Category	Examples
🏥	Medical & Health	Sleep apnea detection, cardiac arrhythmia, gait analysis, seizure detection
🔐	Security & Safety	Intrusion detection, perimeter breach, loitering, panic motion
🏢	Smart Building	Zone occupancy, HVAC control, elevator counting, meeting room tracking
🛒	Retail & Hospitality	Queue length, dwell heatmaps, customer flow, table turnover
🏭	Industrial	Forklift proximity, confined space monitoring, structural vibration
🔮	Exotic & Research	Sleep staging, emotion detection, sign language, breathing sync
📡	Signal Intelligence	Cleans and sharpens raw WiFi signals — focuses on important regions, filters noise, fills in missing data, and tracks which person is which
🧠	Adaptive Learning	The sensor learns new gestures and patterns on its own over time — no cloud needed, remembers what it learned even after updates
🗺️	Spatial Reasoning	Figures out where people are in a room, which zones matter most, and tracks movement across areas using graph-based spatial logic
⏱️	Temporal Analysis	Learns daily routines, detects when patterns break (someone didn't get up), and verifies safety rules are being followed over time
🛡️	AI Security	Detects signal replay attacks, WiFi jamming, injection attempts, and flags abnormal behavior that could indicate tampering
⚛️	Quantum-Inspired	Uses quantum-inspired math to map room-wide signal coherence and search for optimal sensor configurations
🤖	Autonomous & Exotic	Self-managing sensor mesh — auto-heals dropped nodes, plans its own actions, and explores experimental signal representations

All implemented modules are no_std Rust, share a common utility library, and talk to the host through a 12-function API. Full documentation: Edge Modules Guide. See the complete implemented module list below.

🧩 Edge Intelligence — All 65 Modules Implemented (ADR-041 complete)

All 60 modules are implemented, tested (609 tests passing), and ready to deploy. They compile to wasm32-unknown-unknown, run on ESP32-S3 via WASM3, and share a common utility library. Source: crates/wifi-densepose-wasm-edge/src/

Core modules (ADR-040 flagship + early implementations):

Module	File	What It Does
Gesture Classifier	`gesture.rs`	DTW template matching for hand gestures
Coherence Filter	`coherence.rs`	Phase coherence gating for signal quality
Adversarial Detector	`adversarial.rs`	Detects physically impossible signal patterns
Intrusion Detector	`intrusion.rs`	Human vs non-human motion classification
Occupancy Counter	`occupancy.rs`	Zone-level person counting
Vital Trend	`vital_trend.rs`	Long-term breathing and heart rate trending
RVF Parser	`rvf.rs`	RVF container format parsing

Vendor-integrated modules (24 modules, ADR-041 Category 7):

📡 Signal Intelligence — Real-time CSI analysis and feature extraction

Module	File	What It Does	Budget
Flash Attention	`sig_flash_attention.rs`	Tiled attention over 8 subcarrier groups — finds spatial focus regions and entropy	S (<5ms)
Coherence Gate	`sig_coherence_gate.rs`	Z-score phasor gating with hysteresis: Accept / PredictOnly / Reject / Recalibrate	L (<2ms)
Temporal Compress	`sig_temporal_compress.rs`	3-tier adaptive quantization (8-bit hot / 5-bit warm / 3-bit cold)	L (<2ms)
Sparse Recovery	`sig_sparse_recovery.rs`	ISTA L1 reconstruction for dropped subcarriers	H (<10ms)
Person Match	`sig_mincut_person_match.rs`	Hungarian-lite bipartite assignment for multi-person tracking	S (<5ms)
Optimal Transport	`sig_optimal_transport.rs`	Sliced Wasserstein-1 distance with 4 projections	L (<2ms)

🧠 Adaptive Learning — On-device learning without cloud connectivity

Module	File	What It Does	Budget
DTW Gesture Learn	`lrn_dtw_gesture_learn.rs`	User-teachable gesture recognition — 3-rehearsal protocol, 16 templates	S (<5ms)
Anomaly Attractor	`lrn_anomaly_attractor.rs`	4D dynamical system attractor classification with Lyapunov exponents	H (<10ms)
Meta Adapt	`lrn_meta_adapt.rs`	Hill-climbing self-optimization with safety rollback	L (<2ms)
EWC Lifelong	`lrn_ewc_lifelong.rs`	Elastic Weight Consolidation — remembers past tasks while learning new ones	S (<5ms)

🗺️ Spatial Reasoning — Location, proximity, and influence mapping

Module	File	What It Does	Budget
PageRank Influence	`spt_pagerank_influence.rs`	4x4 cross-correlation graph with power iteration PageRank	L (<2ms)
Micro HNSW	`spt_micro_hnsw.rs`	64-vector navigable small-world graph for nearest-neighbor search	S (<5ms)
Spiking Tracker	`spt_spiking_tracker.rs`	32 LIF neurons + 4 output zone neurons with STDP learning	S (<5ms)

⏱️ Temporal Analysis — Activity patterns, logic verification, autonomous planning

Module	File	What It Does	Budget
Pattern Sequence	`tmp_pattern_sequence.rs`	Activity routine detection and deviation alerts	S (<5ms)
Temporal Logic Guard	`tmp_temporal_logic_guard.rs`	LTL formula verification on CSI event streams	S (<5ms)
GOAP Autonomy	`tmp_goap_autonomy.rs`	Goal-Oriented Action Planning for autonomous module management	S (<5ms)

🛡️ AI Security — Tamper detection and behavioral anomaly profiling

Module	File	What It Does	Budget
Prompt Shield	`ais_prompt_shield.rs`	FNV-1a replay detection, injection detection (10x amplitude), jamming (SNR)	L (<2ms)
Behavioral Profiler	`ais_behavioral_profiler.rs`	6D behavioral profile with Mahalanobis anomaly scoring	S (<5ms)

⚛️ Quantum-Inspired — Quantum computing metaphors applied to CSI analysis

Module	File	What It Does	Budget
Quantum Coherence	`qnt_quantum_coherence.rs`	Bloch sphere mapping, Von Neumann entropy, decoherence detection	S (<5ms)
Interference Search	`qnt_interference_search.rs`	16 room-state hypotheses with Grover-inspired oracle + diffusion	S (<5ms)

🤖 Autonomous Systems — Self-governing and self-healing behaviors

Module	File	What It Does	Budget
Psycho-Symbolic	`aut_psycho_symbolic.rs`	16-rule forward-chaining knowledge base with contradiction detection	S (<5ms)
Self-Healing Mesh	`aut_self_healing_mesh.rs`	8-node mesh with health tracking, degradation/recovery, coverage healing	S (<5ms)

🔮 Exotic (Vendor) — Novel mathematical models for CSI interpretation

Module	File	What It Does	Budget
Time Crystal	`exo_time_crystal.rs`	Autocorrelation subharmonic detection in 256-frame history	S (<5ms)
Hyperbolic Space	`exo_hyperbolic_space.rs`	Poincare ball embedding with 32 reference locations, hyperbolic distance	S (<5ms)

🏥 Medical & Health (Category 1) — Contactless health monitoring

Module	File	What It Does	Budget
Sleep Apnea	`med_sleep_apnea.rs`	Detects breathing pauses during sleep	S (<5ms)
Cardiac Arrhythmia	`med_cardiac_arrhythmia.rs`	Monitors heart rate for irregular rhythms	S (<5ms)
Respiratory Distress	`med_respiratory_distress.rs`	Alerts on abnormal breathing patterns	S (<5ms)
Gait Analysis	`med_gait_analysis.rs`	Tracks walking patterns and detects changes	S (<5ms)
Seizure Detection	`med_seizure_detect.rs`	6-state machine for tonic-clonic seizure recognition	S (<5ms)

🔐 Security & Safety (Category 2) — Perimeter and threat detection

Module	File	What It Does	Budget
Perimeter Breach	`sec_perimeter_breach.rs`	Detects boundary crossings with approach/departure	S (<5ms)
Weapon Detection	`sec_weapon_detect.rs`	Metal anomaly detection via CSI amplitude shifts	S (<5ms)
Tailgating	`sec_tailgating.rs`	Detects unauthorized follow-through at access points	S (<5ms)
Loitering	`sec_loitering.rs`	Alerts when someone lingers too long in a zone	S (<5ms)
Panic Motion	`sec_panic_motion.rs`	Detects fleeing, struggling, or panic movement	S (<5ms)

🏢 Smart Building (Category 3) — Automation and energy efficiency

Module	File	What It Does	Budget
HVAC Presence	`bld_hvac_presence.rs`	Occupancy-driven HVAC control with departure countdown	S (<5ms)
Lighting Zones	`bld_lighting_zones.rs`	Auto-dim/off lighting based on zone activity	S (<5ms)
Elevator Count	`bld_elevator_count.rs`	Counts people entering/leaving with overload warning	S (<5ms)
Meeting Room	`bld_meeting_room.rs`	Tracks meeting lifecycle: start, headcount, end, availability	S (<5ms)
Energy Audit	`bld_energy_audit.rs`	Tracks after-hours usage and room utilization rates	S (<5ms)

🛒 Retail & Hospitality (Category 4) — Customer insights without cameras

Module	File	What It Does	Budget
Queue Length	`ret_queue_length.rs`	Estimates queue size and wait times	S (<5ms)
Dwell Heatmap	`ret_dwell_heatmap.rs`	Shows where people spend time (hot/cold zones)	S (<5ms)
Customer Flow	`ret_customer_flow.rs`	Counts ins/outs and tracks net occupancy	S (<5ms)
Table Turnover	`ret_table_turnover.rs`	Restaurant table lifecycle: seated, dining, vacated	S (<5ms)
Shelf Engagement	`ret_shelf_engagement.rs`	Detects browsing, considering, and reaching for products	S (<5ms)

🏭 Industrial & Specialized (Category 5) — Safety and compliance

Module	File	What It Does	Budget
Forklift Proximity	`ind_forklift_proximity.rs`	Warns when people get too close to vehicles	S (<5ms)
Confined Space	`ind_confined_space.rs`	OSHA-compliant worker monitoring with extraction alerts	S (<5ms)
Clean Room	`ind_clean_room.rs`	Occupancy limits and turbulent motion detection	S (<5ms)
Livestock Monitor	`ind_livestock_monitor.rs`	Animal presence, stillness, and escape alerts	S (<5ms)
Structural Vibration	`ind_structural_vibration.rs`	Seismic events, mechanical resonance, structural drift	S (<5ms)

🔮 Exotic & Research (Category 6) — Experimental sensing applications

Module	File	What It Does	Budget
Dream Stage	`exo_dream_stage.rs`	Contactless sleep stage classification (wake/light/deep/REM)	S (<5ms)
Emotion Detection	`exo_emotion_detect.rs`	Arousal, stress, and calm detection from micro-movements	S (<5ms)
Gesture Language	`exo_gesture_language.rs`	Sign language letter recognition via WiFi	S (<5ms)
Music Conductor	`exo_music_conductor.rs`	Tempo and dynamic tracking from conducting gestures	S (<5ms)
Plant Growth	`exo_plant_growth.rs`	Monitors plant growth, circadian rhythms, wilt detection	S (<5ms)
Ghost Hunter	`exo_ghost_hunter.rs`	Environmental anomaly classification (draft/insect/wind/unknown)	S (<5ms)
Rain Detection	`exo_rain_detect.rs`	Detects rain onset, intensity, and cessation via signal scatter	S (<5ms)
Breathing Sync	`exo_breathing_sync.rs`	Detects synchronized breathing between multiple people	S (<5ms)

🧠 Self-Learning WiFi AI (ADR-024) — Adaptive recognition, self-optimization, and intelligent anomaly detection

Every WiFi signal that passes through a room creates a unique fingerprint of that space. WiFi-DensePose already reads these fingerprints to track people, but until now it threw away the internal "understanding" after each reading. The Self-Learning WiFi AI captures and preserves that understanding as compact, reusable vectors — and continuously optimizes itself for each new environment.

What it does in plain terms:

Turns any WiFi signal into a 128-number "fingerprint" that uniquely describes what's happening in a room
Learns entirely on its own from raw WiFi data — no cameras, no labeling, no human supervision needed
Recognizes rooms, detects intruders, identifies people, and classifies activities using only WiFi
Runs on an $8 ESP32 chip (the entire model fits in 55 KB of memory)
Produces both body pose tracking AND environment fingerprints in a single computation

Key Capabilities

What	How it works	Why it matters
Self-supervised learning	The model watches WiFi signals and teaches itself what "similar" and "different" look like, without any human-labeled data	Deploy anywhere — just plug in a WiFi sensor and wait 10 minutes
Room identification	Each room produces a distinct WiFi fingerprint pattern	Know which room someone is in without GPS or beacons
Anomaly detection	An unexpected person or event creates a fingerprint that doesn't match anything seen before	Automatic intrusion and fall detection as a free byproduct
Person re-identification	Each person disturbs WiFi in a slightly different way, creating a personal signature	Track individuals across sessions without cameras
Environment adaptation	MicroLoRA adapters (1,792 parameters per room) fine-tune the model for each new space	Adapts to a new room with minimal data — 93% less than retraining from scratch
Memory preservation	EWC++ regularization remembers what was learned during pretraining	Switching to a new task doesn't erase prior knowledge
Hard-negative mining	Training focuses on the most confusing examples to learn faster	Better accuracy with the same amount of training data

Architecture

WiFi Signal [56 channels] → Transformer + Graph Neural Network
                                  ├→ 128-dim environment fingerprint (for search + identification)
                                  └→ 17-joint body pose (for human tracking)

Quick Start

# Step 1: Learn from raw WiFi data (no labels needed)
cargo run -p wifi-densepose-sensing-server -- --pretrain --dataset data/csi/ --pretrain-epochs 50

# Step 2: Fine-tune with pose labels for full capability
cargo run -p wifi-densepose-sensing-server -- --train --dataset data/mmfi/ --epochs 100 --save-rvf model.rvf

# Step 3: Use the model — extract fingerprints from live WiFi
cargo run -p wifi-densepose-sensing-server -- --model model.rvf --embed

# Step 4: Search — find similar environments or detect anomalies
cargo run -p wifi-densepose-sensing-server -- --model model.rvf --build-index env

Training Modes

Mode	What you need	What you get
Self-Supervised	Just raw WiFi data	A model that understands WiFi signal structure
Supervised	WiFi data + body pose labels	Full pose tracking + environment fingerprints
Cross-Modal	WiFi data + camera footage	Fingerprints aligned with visual understanding

Fingerprint Index Types

Index	What it stores	Real-world use
`env_fingerprint`	Average room fingerprint	"Is this the kitchen or the bedroom?"
`activity_pattern`	Activity boundaries	"Is someone cooking, sleeping, or exercising?"
`temporal_baseline`	Normal conditions	"Something unusual just happened in this room"
`person_track`	Individual movement signatures	"Person A just entered the living room"

Model Size

Component	Parameters	Memory (on ESP32)
Transformer backbone	~28,000	28 KB
Embedding projection head	~25,000	25 KB
Per-room MicroLoRA adapter	~1,800	2 KB
Total	~55,000	55 KB (of 520 KB available)

The self-learning system builds on the AI Backbone (RuVector) signal-processing layer — attention, graph algorithms, and compression — adding contrastive learning on top.

See docs/adr/ADR-024-contrastive-csi-embedding-model.md for full architectural details.

🧩 Claude Code & Codex Plugin

RuView ships a Claude Code plugin (and Codex prompt mirror) that wraps the whole workflow — onboarding, ESP32 setup, configuration, sensing apps, model training, advanced multistatic sensing, CLI/API/WASM, mmWave radar, and witness verification — as 9 skills, 7 /ruview-* commands, and 3 agents. It lives in plugins/ruview/; the marketplace manifest is .claude-plugin/marketplace.json at the repo root.

# In Claude Code — add this repo as a plugin marketplace, then install:
/plugin marketplace add ruvnet/RuView
/plugin install ruview@ruview

# Or try it for one session without installing (from a local clone of the repo):
claude --plugin-dir ./plugins/ruview

# Then, in Claude Code:
#   /ruview-start      → onboarding (Docker demo / repo build / live ESP32)
#   /ruview-flash      → build + flash ESP32 firmware
#   /ruview-provision  → provision WiFi creds, sink IP, channel/MAC, mesh slots
#   /ruview-app        → run a sensing application (presence / vitals / pose / sleep / MAT / point cloud)
#   /ruview-train      → train / evaluate / publish a model (incl. GPU on GCloud)
#   /ruview-advanced   → multistatic / tomography / cross-viewpoint / mesh-security
#   /ruview-verify     → tests + deterministic proof + witness bundle

Codex (OpenAI CLI): cp plugins/ruview/codex/prompts/*.md ~/.codex/prompts/ — the seven /ruview-* commands are mirrored as Codex prompts; plugins/ruview/codex/AGENTS.md carries the project rules. See plugins/ruview/codex/README.md.

Verify the plugin structure: bash plugins/ruview/scripts/smoke.sh. Full details: plugins/ruview/README.md.

📖 Documentation

Document	Description
User Guide	Step-by-step guide: installation, first run, API usage, hardware setup, training
Build Guide	Building from source (Rust and Python)
Claude Code / Codex Plugin	The `ruview` plugin + marketplace — skills, `/ruview-*` commands, agents, and the Codex prompt mirror
Architecture Decisions	96 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure)
Domain Models	8 DDD models (RuvSense, Signal Processing, Training Pipeline, Hardware Platform, Sensing Server, WiFi-Mat, CHCI, rvCSI) — bounded contexts, aggregates, domain events, and ubiquitous language
rvCSI — edge RF sensing runtime	Rust-first / TypeScript-accessible / hardware-abstracted CSI runtime: multi-source ingestion (incl. real nexmon_csi `.pcap` from a Raspberry Pi 5 / Pi 4 / Pi 3B+ — CYW43455 / BCM43455c0) → validation → DSP → typed events → RuVector RF memory (ADR-095, ADR-096, domain model). Now its own repo — `ruvnet/rvcsi` — vendored here under `vendor/rvcsi`; 9 `rvcsi-*` crates on crates.io, `@ruv/rvcsi` on npm, plus a Claude Code plugin.
Desktop App	WIP — Tauri v2 desktop app for node management, OTA updates, WASM deployment, and mesh visualization
Medical Examples	Contactless blood pressure, heart rate, breathing rate via 60 GHz mmWave radar — $15 hardware, no wearable
Extended Documentation	Latest additions, key features, installation, quick start, signal processing, training, CLI, testing, deployment, and changelog

📄 License

MIT License — see LICENSE for details.

📞 Support

GitHub Issues | Discussions | PyPI

WiFi DensePose — Privacy-preserving human pose estimation through WiFi signals.

colbymchenry/codegraph

Pre-indexed code knowledge graph for Claude Code — fewer tokens, fewer tool calls, 100% local

CodeGraph

Supercharge Claude Code with Semantic Code Intelligence

94% fewer tool calls · 77% faster exploration · 100% local

Get Started

npx @colbymchenry/codegraph

_{Interactive installer configures Claude Code automatically}

Initialize Projects

cd your-project
codegraph init -i

Why CodeGraph?

When Claude Code explores a codebase, it spawns Explore agents that scan files with grep, glob, and Read — consuming tokens on every tool call.

CodeGraph gives those agents a pre-indexed knowledge graph — symbol relationships, call graphs, and code structure. Agents query the graph instantly instead of scanning files.

Benchmark Results

Tested across 6 real-world codebases comparing Claude Code's Explore agent with and without CodeGraph:

Average: 92% fewer tool calls · 71% faster

Codebase	With CG	Without CG	Improvement
VS Code · TypeScript	3 calls, 17s	52 calls, 1m 37s	94% fewer · 82% faster
Excalidraw · TypeScript	3 calls, 29s	47 calls, 1m 45s	94% fewer · 72% faster
Claude Code · Python + Rust	3 calls, 39s	40 calls, 1m 8s	93% fewer · 43% faster
Claude Code · Java	1 call, 19s	26 calls, 1m 22s	96% fewer · 77% faster
Alamofire · Swift	3 calls, 22s	32 calls, 1m 39s	91% fewer · 78% faster
Swift Compiler · Swift/C++	6 calls, 35s	37 calls, 2m 8s	84% fewer · 73% faster

Full benchmark details

All tests used Claude Opus 4.6 (1M context) with Claude Code v2.1.91. Each test spawned a single Explore agent with the same question.

Queries used:

Codebase	Query
VS Code	"How does the extension host communicate with the main process?"
Excalidraw	"How does collaborative editing and real-time sync work?"
Claude Code (Python+Rust)	"How does tool execution work end to end?"
Claude Code (Java)	"How does tool execution work end to end?"
Alamofire	"Trace how a request flows from Session.request() through to the URLSession layer"
Swift Compiler	"How does the Swift compiler handle error diagnostics?"

With CodeGraph — the agent uses codegraph_explore and stops:

Codebase	Files Indexed	Nodes	Tool Uses	Tokens	Time
VS Code (TypeScript)	4,002	59,377	3	56.6k	17s
Excalidraw (TypeScript)	626	9,859	3	57.1k	29s
Claude Code (Python+Rust)	115	3,080	3	67.1k	39s
Claude Code (Java)	—	—	1	40.8k	19s
Alamofire (Swift)	102	2,624	3	57.3k	22s
Swift Compiler (Swift/C++)	25,874	272,898	6	77.4k	35s

Without CodeGraph — the agent uses grep, find, ls, and Read extensively:

Codebase	Tool Uses	Tokens	Time	File Reads
VS Code (TypeScript)	52	89.4k	1m 37s	~15
Excalidraw (TypeScript)	47	77.9k	1m 45s	~20
Claude Code (Python+Rust)	40	69.3k	1m 8s	~15
Claude Code (Java)	26	73.3k	1m 22s	~15
Alamofire (Swift)	32	52.4k	1m 39s	~10
Swift Compiler (Swift/C++)	37	99.1k	2m 8s	~20

Key observations:

With CodeGraph, the agent never fell back to reading files — it trusted the codegraph_explore results completely
Without CodeGraph, agents spent most of their time on discovery (find, ls, grep) before they could even start reading relevant code
The Java codebase needed only 1 codegraph_explore call to answer the entire question
Cross-language queries (Python+Rust) worked seamlessly — CodeGraph's graph traversal found connections across language boundaries
The Swift benchmark (Alamofire) traced a 9-step call chain from Session.request() to URLSession.dataTask() — CodeGraph's graph traversal at depth 3 captured the full chain in one explore call
The Swift Compiler benchmark is the largest codebase tested (25,874 files, 272,898 nodes) — CodeGraph indexed it in under 4 minutes and the agent answered a complex cross-cutting question with 6 explore calls and zero file reads in 35 seconds

Key Features


Smart Context Building	One tool call returns entry points, related symbols, and code snippets — no expensive exploration agents
Full-Text Search	Find code by name instantly across your entire codebase, powered by FTS5
Impact Analysis	Trace callers, callees, and the full impact radius of any symbol before making changes
Always Fresh	File watcher uses native OS events (FSEvents/inotify/ReadDirectoryChangesW) with debounced auto-sync — the graph stays current as you code, zero config
19+ Languages	TypeScript, JavaScript, Python, Go, Rust, Java, C#, PHP, Ruby, C, C++, Swift, Kotlin, Dart, Svelte, Liquid, Pascal/Delphi
Framework-aware Routes	Recognizes web-framework routing files and links URL patterns to their handlers across 13 frameworks
100% Local	No data leaves your machine. No API keys. No external services. SQLite database only

Framework-aware Routes

CodeGraph detects web-framework routing files and emits route nodes linked by references edges to their handler classes or functions. Querying callers of a view/controller now surfaces the URL pattern that binds it.

Framework	Shapes recognized
Django	`path()`, `re_path()`, `url()`, `include()` in `urls.py` (CBV `.as_view()`, dotted paths)
Flask	`@app.route('/path', methods=[...])`, blueprint routes
FastAPI	`@app.get(...)`, `@router.post(...)`, all standard methods
Express	`app.get(...)`, `router.post(...)` with middleware chains
Laravel	`Route::get()`, `Route::resource()`, `Controller@action`, tuple syntax
Rails	`get '/x', to: 'users#index'`, hash-rocket `=>` syntax
Spring	`@GetMapping`, `@PostMapping`, `@RequestMapping` on methods
Gin / chi / gorilla / mux	`r.GET(...)`, `router.HandleFunc(...)`
Axum / actix / Rocket	`.route("/x", get(handler))`
ASP.NET	`[HttpGet("/x")]` attributes on action methods
Vapor	`app.get("x", use: handler)`
React Router / SvelteKit	Route component nodes

Quick Start

1. Run the Installer

npx @colbymchenry/codegraph

The installer will:

Prompt to install codegraph globally (needed for the MCP server)
Configure the MCP server in ~/.claude.json
Set up auto-allow permissions for CodeGraph tools
Add global instructions to ~/.claude/CLAUDE.md
Optionally initialize your current project

2. Restart Claude Code

Restart Claude Code for the MCP server to load.

3. Initialize Projects

cd your-project
codegraph init -i

That's it! Claude Code will use CodeGraph tools automatically when a .codegraph/ directory exists.

Manual Setup (Alternative)

Install globally:

npm install -g @colbymchenry/codegraph

Add to ~/.claude.json:

{
  "mcpServers": {
    "codegraph": {
      "type": "stdio",
      "command": "codegraph",
      "args": ["serve", "--mcp"]
    }
  }
}

Add to ~/.claude/settings.json (optional, for auto-allow):

{
  "permissions": {
    "allow": [
      "mcp__codegraph__codegraph_search",
      "mcp__codegraph__codegraph_context",
      "mcp__codegraph__codegraph_callers",
      "mcp__codegraph__codegraph_callees",
      "mcp__codegraph__codegraph_impact",
      "mcp__codegraph__codegraph_node",
      "mcp__codegraph__codegraph_status",
      "mcp__codegraph__codegraph_files"
    ]
  }
}

Global Instructions Reference

The installer automatically adds these instructions to ~/.claude/CLAUDE.md:

## CodeGraph

CodeGraph builds a semantic knowledge graph of codebases for faster, smarter code exploration.

### If `.codegraph/` exists in the project

**NEVER call `codegraph_explore` or `codegraph_context` directly in the main session.** These tools return large amounts of source code that fills up main session context. Instead, ALWAYS spawn an Explore agent for any exploration question (e.g., "how does X work?", "explain the Y system", "where is Z implemented?").

**When spawning Explore agents**, include this instruction in the prompt:

> This project has CodeGraph initialized (.codegraph/ exists). Use `codegraph_explore` as your PRIMARY tool — it returns full source code sections from all relevant files in one call.
>
> **Rules:**
> 1. Follow the explore call budget in the `codegraph_explore` tool description — it scales automatically based on project size.
> 2. Do NOT re-read files that codegraph_explore already returned source code for. The source sections are complete and authoritative.
> 3. Only fall back to grep/glob/read for files listed under "Additional relevant files" if you need more detail, or if codegraph returned no results.

**The main session may only use these lightweight tools directly** (for targeted lookups before making edits, not for exploration):

| Tool | Use For |
|------|---------|
| `codegraph_search` | Find symbols by name |
| `codegraph_callers` / `codegraph_callees` | Trace call flow |
| `codegraph_impact` | Check what's affected before editing |
| `codegraph_node` | Get a single symbol's details |

### If `.codegraph/` does NOT exist

At the start of a session, ask the user if they'd like to initialize CodeGraph:

"I notice this project doesn't have CodeGraph initialized. Would you like me to run `codegraph init -i` to build a code knowledge graph?"

How It Works

┌─────────────────────────────────────────────────────────────────┐
│                        Claude Code                               │
│                                                                  │
│  "Implement user authentication"                                 │
│           │                                                      │
│           ▼                                                      │
│  ┌─────────────────┐      ┌─────────────────┐                   │
│  │  Explore Agent  │ ──── │  Explore Agent  │                   │
│  └────────┬────────┘      └────────┬────────┘                   │
│           │                        │                             │
└───────────┼────────────────────────┼─────────────────────────────┘
            │                        │
            ▼                        ▼
┌───────────────────────────────────────────────────────────────────┐
│                     CodeGraph MCP Server                          │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐               │
│  │   Search    │  │   Callers   │  │   Context   │               │
│  │  "auth"     │  │  "login()"  │  │  for task   │               │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘               │
│         │                │                │                       │
│         └────────────────┼────────────────┘                       │
│                          ▼                                        │
│              ┌───────────────────────┐                            │
│              │   SQLite Graph DB     │                            │
│              │   • 387 symbols       │                            │
│              │   • 1,204 edges       │                            │
│              │   • Instant lookups   │                            │
│              └───────────────────────┘                            │
└───────────────────────────────────────────────────────────────────┘

Extraction — tree-sitter parses source code into ASTs. Language-specific queries extract nodes (functions, classes, methods) and edges (calls, imports, extends, implements).
Storage — Everything goes into a local SQLite database (.codegraph/codegraph.db) with FTS5 full-text search.
Resolution — After extraction, references are resolved: function calls → definitions, imports → source files, class inheritance, and framework-specific patterns.
Auto-Sync — The MCP server watches your project using native OS file events. Changes are debounced (2-second quiet window), filtered to source files only, and incrementally synced. The graph stays fresh as you code — no configuration needed.

CLI Reference

codegraph                         # Run interactive installer
codegraph install                 # Run installer (explicit)
codegraph init [path]             # Initialize in a project (--index to also index)
codegraph uninit [path]           # Remove CodeGraph from a project (--force to skip prompt)
codegraph index [path]            # Full index (--force to re-index, --quiet for less output)
codegraph sync [path]             # Incremental update
codegraph status [path]           # Show statistics
codegraph query <search>          # Search symbols (--kind, --limit, --json)
codegraph files [path]            # Show file structure (--format, --filter, --max-depth, --json)
codegraph context <task>          # Build context for AI (--format, --max-nodes)
codegraph affected [files...]     # Find test files affected by changes (see below)
codegraph serve --mcp             # Start MCP server

`codegraph affected`

Traces import dependencies transitively to find which test files are affected by changed source files.

codegraph affected src/utils.ts src/api.ts         # Pass files as arguments
git diff --name-only | codegraph affected --stdin   # Pipe from git diff
codegraph affected src/auth.ts --filter "e2e/*"     # Custom test file pattern

Option	Description	Default
`--stdin`	Read file list from stdin	`false`
`-d, --depth <n>`	Max dependency traversal depth	`5`
`-f, --filter <glob>`	Custom glob to identify test files	auto-detect
`-j, --json`	Output as JSON	`false`
`-q, --quiet`	Output file paths only	`false`

CI/hook example:

#!/usr/bin/env bash
AFFECTED=$(git diff --name-only HEAD | codegraph affected --stdin --quiet)
if [ -n "$AFFECTED" ]; then
  npx vitest run $AFFECTED
fi

MCP Tools

When running as an MCP server, CodeGraph exposes these tools to Claude Code:

Tool	Purpose
`codegraph_search`	Find symbols by name across the codebase
`codegraph_context`	Build relevant code context for a task
`codegraph_callers`	Find what calls a function
`codegraph_callees`	Find what a function calls
`codegraph_impact`	Analyze what code is affected by changing a symbol
`codegraph_node`	Get details about a specific symbol (optionally with source code)
`codegraph_files`	Get indexed file structure (faster than filesystem scanning)
`codegraph_status`	Check index health and statistics

Library Usage

import CodeGraph from '@colbymchenry/codegraph';

const cg = await CodeGraph.init('/path/to/project');
// Or: const cg = await CodeGraph.open('/path/to/project');

await cg.indexAll({
  onProgress: (p) => console.log(`${p.phase}: ${p.current}/${p.total}`)
});

const results = cg.searchNodes('UserService');
const callers = cg.getCallers(results[0].node.id);
const context = await cg.buildContext('fix login bug', { maxNodes: 20, includeCode: true, format: 'markdown' });
const impact = cg.getImpactRadius(results[0].node.id, 2);

cg.watch();   // auto-sync on file changes
cg.unwatch(); // stop watching
cg.close();

Configuration

The .codegraph/config.json file controls indexing:

{
  "version": 1,
  "languages": ["typescript", "javascript"],
  "exclude": ["node_modules/**", "dist/**", "build/**", "*.min.js"],
  "frameworks": [],
  "maxFileSize": 1048576,
  "extractDocstrings": true,
  "trackCallSites": true
}

Option	Description	Default
`languages`	Languages to index (auto-detected if empty)	`[]`
`exclude`	Glob patterns to ignore	`["node_modules/**", ...]`
`frameworks`	Framework hints for better resolution	`[]`
`maxFileSize`	Skip files larger than this (bytes)	`1048576` (1MB)
`extractDocstrings`	Extract docstrings from code	`true`
`trackCallSites`	Track call site locations	`true`

Supported Languages

Language	Extension	Status
TypeScript	`.ts`, `.tsx`	Full support
JavaScript	`.js`, `.jsx`, `.mjs`	Full support
Python	`.py`	Full support
Go	`.go`	Full support
Rust	`.rs`	Full support
Java	`.java`	Full support
C#	`.cs`	Full support
PHP	`.php`	Full support
Ruby	`.rb`	Full support
C	`.c`, `.h`	Full support
C++	`.cpp`, `.hpp`, `.cc`	Full support
Swift	`.swift`	Full support
Kotlin	`.kt`, `.kts`	Full support
Scala	`.scala`, `.sc`	Full support (classes, traits, methods, type aliases, Scala 3 enums)
Dart	`.dart`	Full support
Svelte	`.svelte`	Full support (script extraction, Svelte 5 runes, SvelteKit routes)
Vue	`.vue`	Full support (script + script-setup extraction, Nuxt page/API/middleware routes)
Liquid	`.liquid`	Full support
Pascal / Delphi	`.pas`, `.dpr`, `.dpk`, `.lpr`	Full support (classes, records, interfaces, enums, DFM/FMX form files)

Troubleshooting

"CodeGraph not initialized" — Run codegraph init in your project directory first.

Indexing is slow — Check that node_modules and other large directories are excluded. Use --quiet to reduce output overhead.

Indexing is slow / MCP database is locked / WASM fallback active — codegraph ships with a WASM SQLite fallback for environments where better-sqlite3 (a native module, declared as optionalDependencies) can't install. The fallback is 5-10x slower than the native backend and uses a journal mode that lets writers block readers, so MCP queries can also hit database is locked while indexing runs. Run codegraph status and look at the Backend: line:

Backend: native — you're on the fast path, nothing to do.

Backend: wasm — you're on the slow fallback. Common causes: missing C build tools, prebuilt binary unavailable for your Node version, or your Node version changed after install. Fix:

# macOS
xcode-select --install                                  # installs the C compiler

# Linux (Debian / Ubuntu)
sudo apt install build-essential python3 make

# Linux (RHEL / Fedora)
sudo yum groupinstall "Development Tools"

# Then rebuild on any platform:
npm rebuild better-sqlite3

# Or force-include as a hard dep:
npm install better-sqlite3 --save

After the fix, codegraph status should show Backend: native.

MCP server not connecting — Ensure the project is initialized/indexed, verify the path in your MCP config, and check that codegraph serve --mcp works from the command line.

Missing symbols — The MCP server auto-syncs on save (wait a couple seconds). Run codegraph sync manually if needed. Check that the file's language is supported and isn't excluded by config patterns.

License

MIT

Made for the Claude Code community

Report Bug · Request Feature

GitHub All Languages Daily Trending

oven-sh/bun

Bun

Read the docs →

What is Bun?

Install

Upgrade

Quick links

Guides

Contributing

License

K-Dense-AI/scientific-agent-skills

Scientific Agent Skills

📦 What's Included

📋 Table of Contents

🚀 Why Use This?

⚡ Accelerate Your Research

🎯 Comprehensive Coverage

🔧 Easy Integration

🌟 Maintained & Supported

🎯 Getting Started

Option 1: npx (all platforms)

Option 2: GitHub CLI (gh skill)

Version pinning

Keeping skills up to date

⚠️ Security Disclaimer

❤️ Support the Open Source Community

⚙️ Prerequisites

Installing uv

💡 Quick Examples

🧪 Drug Discovery Pipeline

🔬 Single-Cell RNA-seq Analysis

🧬 Multi-Omics Biomarker Discovery

🎯 Virtual Screening Campaign

🏥 Clinical Variant Interpretation

🌐 Systems Biology Network Analysis

🚀 Want to Skip the Setup and Just Do the Science?

🔬 Use Cases

🧪 Drug Discovery & Medicinal Chemistry

🧬 Bioinformatics & Genomics

🏥 Clinical Research & Precision Medicine

🔬 Multi-Omics & Systems Biology

📊 Data Analysis & Visualization

🧪 Laboratory Automation

📚 Available Skills

Skill Categories

🧬 Bioinformatics & Genomics (21+ skills)

🧪 Cheminformatics & Drug Discovery (10+ skills)

🔬 Proteomics & Mass Spectrometry (2 skills)

🏥 Clinical Research & Precision Medicine (8+ skills)

🖼️ Medical Imaging & Digital Pathology (3 skills)

🧠 Neuroscience & Electrophysiology (1 skill)

🤖 Machine Learning & AI (16+ skills)

🔮 Materials Science, Chemistry & Physics (7 skills)

⚙️ Engineering & Simulation (4 skills)

📊 Data Analysis & Visualization (16+ skills)

🧪 Laboratory Automation (4 skills)

🔬 Multi-omics & Systems Biology (4+ skills)

🧬 Protein Engineering & Design (3 skills)

📚 Scientific Communication (20+ skills)

🔬 Scientific Databases & Data Access (6 skills → 100+ databases total)

🔧 Infrastructure & Platforms (7+ skills)

🎓 Research Methodology & Planning (12+ skills)

⚖️ Regulatory & Standards (1 skill)

🤝 Contributing

Ways to Contribute

How to Contribute

Contribution Guidelines

Security Scanning

Recognition

Support Open Source

🔧 Troubleshooting

Common Issues

❓ FAQ

General Questions

Installation & Setup

Contributing

💬 Support

📖 Citation

BibTeX

Option 2: GitHub CLI (`gh skill`)