GitHub C++ Weekly Trending Repositories

LadybirdBrowser/ladybird

Truly independent web browser

Ladybird

Ladybird is a truly independent web browser, using a novel engine based on web standards.

Important

Ladybird is in a pre-alpha state, and only suitable for use by developers

Features

We aim to build a complete, usable browser for the modern web.

Ladybird uses a multi-process architecture with a main UI process, several WebContent renderer processes, an ImageDecoder process, and a RequestServer process.

Image decoding and network connections are done out of process to be more robust against malicious content. Each tab has its own renderer process, which is sandboxed from the rest of the system.

At the moment, many core library support components are inherited from SerenityOS:

LibWeb: Web rendering engine
LibJS: JavaScript engine
LibWasm: WebAssembly implementation
LibCrypto/LibTLS: Cryptography primitives and Transport Layer Security
LibHTTP: HTTP/1.1 client
LibGfx: 2D Graphics Library, Image Decoding and Rendering
LibUnicode: Unicode and locale support
LibMedia: Audio and video playback
LibCore: Event loop, OS abstraction layer
LibIPC: Inter-process communication

How do I build and run this?

See build instructions for information on how to build Ladybird.

Ladybird runs on Linux, macOS, Windows (with WSL2), and many other *Nixes.

How do I read the documentation?

Code-related documentation can be found in the documentation folder.

Get in touch and participate!

Join our Discord server to participate in issue and development discussions.

Please read Getting involved with Ladybird if you're new to Ladybird and want to help.

Before opening an issue, please see the issue policy and the detailed issue-reporting guidelines.

The project participation guidelines can be found in CONTRIBUTING.md.

License

Ladybird is licensed under a 2-clause BSD license.

microsoft/intelligent-terminal

A fork of Windows Terminal with native agent integration, right in your command line.

Welcome to the Intelligent Terminal repo

Table of Contents

What is Intelligent Terminal?

Intelligent Terminal is an experimental fork of Windows Terminal with native agent integration.

Intelligent Terminal works with any Agent Client Protocol (ACP)-compatible agent CLI. All you need is to install your preferred agent CLI on your PC. If you don't have a preferred agent, we'll get you setup with GitHub Copilot CLI.

Intelligent Terminal takes all the features you love in Windows Terminal such as: tabs, profiles, themes, settings, shells, and keyboard shortcuts, which all work the way you expect.

Read the announcement blog post for more details.

Installing and running Intelligent Terminal

Note

Intelligent Terminal requires Windows 10 2004 (19041) or later. You also need a supported agent CLI and subscription. GitHub Copilot is the default.

Microsoft Store (recommended)

Install the Intelligent Terminal from the Microsoft Store. This allows you to always be on the latest version when we release new builds with automatic upgrades.

WinGet

winget users can download and install the latest Intelligent Terminal release by installing the Microsoft.IntelligentTerminal package:

winget install --id Microsoft.IntelligentTerminal -e

Downloads

Distribution	Architecture	Link
App Installer	x64, arm64, x86	Download

Get Started

On first launch, choose your agent. Intelligent Terminal auto-detects several ACP-compatible agent CLIs on your machine (Copilot/Claude/Codex/Gemini/OpenCode). If none are found, it defaults to GitHub Copilot CLI and installs it for you via WinGet.
If you aren't already authenticated, the agent pane walks you through sign-in. For GitHub Copilot Enterprise, press E at the sign-in prompt and enter your enterprise host (for example, your-org.ghe.com); the last host you used is remembered.
Start asking questions and using the agent pane for assistance. The agent has context on your shell output, no copy-pasting needed. The agent pane does not run commands in your shell without your explicit approval: you choose whether to run its suggested command, copy it into your shell to run yourself, or dismiss it.

Tip

If you see "running scripts is disabled on this system" or an UnauthorizedAccess error in PowerShell, your execution policy is blocking your profile and Intelligent Terminal can't initialize shell integration. Run:

Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned

If you run into any other issues or dependency errors, see installing-dependencies.md.

Keyboard Shortcuts

All shortcuts are customizable through Intelligent Terminal settings.

Shortcut	Action
`Ctrl+Shift+.`	Toggle the agent pane
`Ctrl+Shift+I`	Switch focus to/from the agent pane
`Ctrl+Alt+.`	Open agent pane with error context
`Ctrl+Shift+/`	Open agent management
`Alt+Shift+/`	Open Command Palette in prompt mode
`Alt+Shift+B`	Open an interactive delegate-agent tab with no startup prompt
`Alt+V`	Paste a clipboard image into the agent pane

Configuration

Everything is configurable through Intelligent Terminal settings, under "Agent" settings.

Setting	Options
Agent and model	GitHub Copilot (default), or any ACP-compatible agent CLI, including custom or local agents. Configurable for both the agent pane and command palette. Each agent pane can also override its model on the fly with `/model`; changing the global setting here overrides every pane.
Local models (BYOM)	Bring your own model through GitHub Copilot or OpenCode using an OpenAI-compatible Chat Completions endpoint. Add a provider under Agent settings with a Base URL and Model ID, then select it from `/model`. API keys aren't stored, so this suits local endpoints like Ollama that don't require one.
Pane placement	Top, Bottom (default), Left, Right
Error detection	Allows Intelligent Terminal to automatically detect command failures
Error suggestions	Allows Intelligent Terminal to automatically send detected errors to the agent for fix suggestions
Agent session tracking (hooks)	Allows Intelligent Terminal to track active agent sessions and their status in the session management UI
Token usage display	Show or hide token usage and cost in the agent status bar

You can also pin a specific agent to a profile. Open a profile in Settings (for example, "PowerShell" or "Ubuntu") and set the agent you want its agent pane to use. For a WSL profile, the picker also lists agents installed inside that distro, so an Ubuntu profile can run a Linux-side agent. Profiles you don't configure keep using the global agent.

Features

Agent Status Bar

The agent status bar sits at the bottom of the window and gives you quick access to everything agent-related. On the left: the agent pane toggle (hotkey: Ctrl+Shift+.) and the error detection icon (hotkey: Ctrl+Alt+.), which lights up when a fixable error is detected. On the right: the agent management icon (hotkey: Ctrl+Shift+/) that opens your session management panel. It's a persistent, minimal control surface so you're never more than one click away from your agents. The status bar can also display your current token usage and cost as you work; toggle this on or off in Agent settings.

Agent Pane

A context-aware, docked pane with your agent CLI of choice. The pane has context on your shell output across all your shells (PowerShell, Bash/WSL). Toggle with Ctrl+Shift+., switch focus with Ctrl+Shift+I. If the agent needs to do multiple or complex tasks, it spins up background tasks in new tabs so your active shell stays focused. When the agent suggests a shell command, the pane gives you the option to run or copy it rather than running it automatically. Paste a clipboard image straight into the chat with Alt+V to show the agent a screenshot, diagram, or mockup (acting on the image depends on your agent's image support).

When you have multiple panes active, a small "Agent" indicator will appear on the pane that your agent has "focus" on.

Inside the pane you can scroll and drag to select with the mouse, double-click a word or triple-click a line, and press Ctrl+C to copy. Press Up and Down in the input to recall recent prompts.

Slash Commands

Inside the agent pane, type / to see available commands. Type /help at any time to show the list.

Command	Description
`/agent [id]`	Pick the agent source for this tab. In a WSL pane, the picker includes agents installed on Windows and in that pane's WSL distro; it never offers other distros.
`/clear`	Clear the chat scrollback (keeps the current session)
`/fix [hint]`	Diagnose the active terminal and suggest a fix; add an optional hint to steer it (e.g. `/fix the path looks wrong`)
`/help`	Show the command list
`/model [id]`	Pick the model for this pane; bare `/model` opens a picker, `/model <id>` switches directly
`/new`	Start a fresh agent session (drops history)
`/restart`	Restart the agent with a clean session
`/sessions`	Open agent management (same as `Ctrl+Shift+/`)
`/stop`	Cancel the in-flight prompt

Local Models (BYOM)

Run your agent against a model on your own machine instead of a cloud service. BYOM is supported through GitHub Copilot and OpenCode, using any OpenAI-compatible Chat Completions endpoint. Under Agent settings, add a provider with a Base URL and Model ID, then pick it from the /model picker (configured models show as modelId (BYOM)). No API key is stored, so a local Ollama model works with no key at all:

Base URL:  http://localhost:11434/v1
Model ID:  <your local model>

Agent Management

View all active agents, their status, and past sessions. Pick up a workflow where you left off or check on a long-running task. Click the agent management icon in the status bar or press Ctrl+Shift+/ to open it.

Error Detection

When a command fails, an indicator appears in the agent status bar. Click it or press Ctrl+Alt+. to open the agent pane with the error context already loaded. The agent can explain what happened and suggest or run a fix. Configure your settings to auto-detect errors only, or to also auto-suggest fixes. You can also trigger a fix at any time from the agent pane with the /fix slash command.

Command Palette

Type ? followed by your prompt in the Command Palette to kick off an agent task. Intelligent Terminal injects context from the active pane and starts the agent in a background tab. Use Alt+Shift+/ to jump directly into prompt mode.

Data & Privacy

Intelligent Terminal is a local transport layer. It passes your prompts and shell context to your selected agent CLI over stdio/ACP. Intelligent Terminal does not call any cloud APIs itself and does not persist conversation history, however, diagnostic logs may be written to disk and telemetry may be emitted as described below.

What data flows through Terminal

Your prompts (what you type in the agent pane or command palette)
Shell output context (recent command output shared with the agent for context)
Basic environment metadata (shell type, OS version)

All of this is held in memory for the active session only and discarded when the session ends.

Where your data goes depends on your agent CLI

Agent CLI	Data routing	Terms
GitHub Copilot (default)	GitHub backend	GitHub Copilot Trust Center. Enterprise protections (e.g., zero data retention) apply for eligible plans.
Third-party or custom agent CLIs	Determined by the agent vendor	Governed by that vendor's terms, not Microsoft or GitHub agreements.

Note

Terminal cannot guarantee data protections for third-party agent CLIs. When you select an agent, you're choosing where your data goes. Review your agent vendor's privacy policy before use. For more information on how to use GitHub Copilot responsibly, see Responsible use of GitHub Copilot.

Controls

Choose your agent CLI at any time in Settings > Agent
Disable auto error detection to prevent shell output from being detected automatically
Intelligent Terminal always asks before running commands on your behalf in your shell

Intelligent Terminal only collects usage data and sends it to Microsoft to help improve our products and services. Read our privacy statement to learn more. See PRIVACY.md for details and instructions on how to disable telemetry.

Data Collection

The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft's privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.

Building the Code

Building Intelligent Terminal is the same as building Windows Terminal. See the Developer Guidance section of the Windows Terminal README for prerequisites, build instructions, and debugging steps.

Feedback

Intelligent Terminal is in an experimental stage. If you have a feature request or find a bug, submit an issue on the GitHub repository.

Collecting Logs

To help us investigate bugs, please collect and attach diagnostic logs when filing an issue:

Open Command Palette with Ctrl+Shift+P
Search for Report a bug (collect logs)

A ZIP file containing your error logs will be generated and opened in File Explorer
Attach the ZIP to your issue so we can investigate with full context

Intelligent Terminal ships as a separate app and installs next to your existing Windows Terminal. If you don't want agents in your terminal, nothing changes for you. With this model, we can learn, experiment, and iterate with you, the community, on what this evolution might look like without breaking your existing Windows Terminal flows.

Contributing

We are excited to work alongside you, our amazing community, to build and enhance Intelligent Terminal!

Before you start work on a feature/fix, please read & follow the Windows Terminal Contributor's Guide. The contribution process is the same.

For local dev/build of this repo, see: quick-start-local-dev

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information, see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Security

If you believe you have found a security vulnerability in this repository, please report it following the instructions in SECURITY.md.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos is subject to those third-party's policies.

ggml-org/llama.cpp

LLM inference in C/C++

llama.cpp

LLM inference in C/C++

manifesto / ggml / ops / maintainer PRs / compile times / lib llama API / llama-server REST API

Quick start

A few options to get llama.cpp installed on your machine:

Visit https://llama.app and follow the instructions
Run with Docker - see our Docker documentation
Download pre-built binaries from the releases page
Build from source by cloning this repository - check out our build guide

Once installed:

# Download and run a model directly from Hugging Face
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

# Launch OpenAI-compatible API server
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF

VLM session with llama cli

Built-in web UI against llama serve

Description

The main goal of llama.cpp is to enable LLM (and VLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud.

Plain C/C++ implementation without any dependencies
Apple silicon is a first-class citizen - optimized via ARM NEON, Accelerate and Metal frameworks
AVX, AVX2, AVX512 and AMX support for x86 architectures
RVV, ZVFH, ZFH, ZICBOP and ZIHINTPAUSE support for RISC-V architectures
1.5-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, and 8-bit integer quantization for faster inference and reduced memory use
Custom CUDA kernels for running LLMs on NVIDIA GPUs (support for AMD GPUs via HIP and Moore Threads GPUs via MUSA)
Vulkan and SYCL backend support
CPU+GPU hybrid inference to partially accelerate models larger than the total VRAM capacity

The llama.cpp project is build on top of the ggml library.

Supported backends

Backend	Target devices
BLAS	All
BLIS	All
CANN	Ascend NPU
CUDA	Nvidia GPU
HIP	AMD GPU
Hexagon [In Progress]	Snapdragon
IBM zDNN	IBM Z & LinuxONE
MUSA	Moore Threads GPU
Metal	Apple Silicon
OpenCL	Adreno GPU
OpenVINO [In Progress]	Intel CPUs, GPUs, and NPUs
RPC	All
SYCL	Intel GPU
VirtGPU	VirtGPU APIR
Vulkan	GPU
WebGPU	All
ZenDNN	AMD CPU

Documentation

Tools

Development

Contributing

Contributors can open PRs
Collaborators will be invited based on contributions
Maintainers can push to branches in the llama.cpp repo and merge PRs into the master branch
Any help with managing issues, PRs and projects is very appreciated!
Read the CONTRIBUTING.md for more information

Acknowledgements

yhirose/cpp-httplib - Single-header HTTP server, used by llama-server - MIT license
stb-image - Single-header image format decoder, used by multimodal subsystem - Public domain
nlohmann/json - Single-header JSON library, used by various tools/examples - MIT License
miniaudio.h - Single-header audio format decoder, used by multimodal subsystem - Public domain
subprocess.h - Single-header process launching solution for C and C++ - Public domain

cactus-compute/cactus

Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.

Cactus

A hybrid edge-cloud AI engine for mobile devices & wearables.

┌─────────────────┐
│  Cactus Engine  │ ←── OpenAI-compatible APIs for text, speech, and vision.
└─────────────────┘     
         │
┌─────────────────┐
│  Cactus Graph   │ ←── Zero-copy computation graph
└─────────────────┘     
         │
┌─────────────────┐
│ Cactus Kernels  │ ←── CPU/GPU kernels for (Apple, Samsung, Pixel, etc.)
└─────────────────┘     
         │
┌─────────────────┐
│ Cactus Quants   │ ←── Custom rotation-based quantization technique
└─────────────────┘

Quick Demo (Mac)

Step 1: brew install cactus-compute/cactus/cactus
Step 2: cactus run

Cactus Engine

#include "cactus_engine.h"

cactus_model_t model = cactus_init(
    "path/to/weight/folder",
    "path to txt or dir of txts for auto-rag",
    false
);

const char* messages = R"([
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "My name is Henry Ndubuaku"}
])";

const char* options = R"({
    "max_tokens": 50,
    "stop_sequences": ["<|im_end|>"]
})";

char response[4096];
int result = cactus_complete(
    model,            // model handle
    messages,         // JSON chat messages
    response,         // response buffer
    sizeof(response), // buffer size
    options,          // generation options
    nullptr,          // tools JSON
    nullptr,          // streaming callback
    nullptr,          // user data
    nullptr,          // pcm audio buffer
    0                 // pcm buffer size
);

Example response from Gemma4-E2B

{
    "success": true,        // generation succeeded
    "error": null,          // error details if failed
    "cloud_handoff": false, // true if cloud model used
    "response": "Hi there!",
    "function_calls": [],   // parsed tool calls
    "segments": [],         // transcription segments (empty for chat)
    "confidence": 0.8193,   // model confidence
    "confidence_threshold": 0.7, // resolved handoff threshold (model-dependent)
    "time_to_first_token_ms": 45.23,
    "total_time_ms": 163.67,
    "prefill_tps": 1621.89,
    "decode_tps": 168.42,
    "ram_usage_mb": 245.67,
    "prefill_tokens": 28,
    "decode_tokens": 50,
    "total_tokens": 78
}

Cactus Graph

#include "cactus_graph.h"

CactusGraph graph;
auto a = graph.input({2, 3}, Precision::FP16);
auto b = graph.input({3, 4}, Precision::INT8);

auto x1 = graph.matmul(a, b, false);
auto x2 = graph.transpose(x1);
auto result = graph.matmul(b, x2, true);

float a_data[6] = {1.1f, 2.3f, 3.4f, 4.2f, 5.7f, 6.8f};
float b_data[12] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12};

graph.set_input(a, a_data, Precision::FP16);
graph.set_input(b, b_data, Precision::INT8);

graph.execute();
void* output_data = graph.get_output(result);

graph.hard_reset();

Inference Speed

LLM: Gemma-4-E2B-CQ4 (1k-context prefill / decode for 100 tokens)
VLM: Gemma-4-E2B-CQ4 (256px image encode time / decode)
Transcribe: Parakeet-TDT-0.6B-CQ4 (20s audio end-to-end transcribe time)
1k-Context RAM: peak MB during the LLM benchmark
No speculative decode or MTP, pure decode

Command: cactus benchmark [optional --ios or --android]

Device	LLM	VLM	Transcribe	RAM
Mac M5 Max	2964tps / 154tps	0.09s / 168tps	0.15s	1348MB
Mac M4 Pro	1963tps / 101tps	0.25s / 112tps	0.21s	1225MB
Mac M3 Pro	1294tps / 64tps	0.40s / 72tps	0.37s	735MB
iPad/Vision Pro M5	1336tps / 71tps	0.25s / 80tps	0.27s	703MB
iPhone 17 Pro	729tps / 37tps	0.5s / 39tps	0.51s	644MB
iPhone 15 Pro	517tps / 26tps	1.15s / 27tps	0.82s	633MB

N/B: With 1k-context prefill and decode for 100 runs on M5 Max

LFM2.5-VL-1.6B = 289toks/sec
Qwen3-1.7B = 155toks/sec
LFM2.5-VL-450m = 472toks/sec, image encodes in 43ms
LFM22.5-VL-230m = 555toks/sec

Output Quality

Gemma-4-E2B-it accuracy across bit widths, averaged over 3 seeds.
CQ3.26 and CQ2.54 are mixed-precision, CQ2/CQ3/CQ4 are uniformly quantized.
Full results in docs/cactus_quants.md:

Task	F16 (Original)	CQ4	CQ3.26	CQ2.54	CQ2
ARC-E	73.80	73.73	74.20	68.20	50.80
ARC-C	56.47	52.47	51.53	37.20	24.73
HellaSwag	46.93	47.07	45.20	40.73	35.87
WinoGrande	61.00	61.13	59.60	60.13	51.27
MMLU	62.33	59.45	57.63	47.19	33.18
GPQA	34.34	34.34	31.82	30.81	23.23
GSM8K	73.67	71.20	66.20	22.00	0.40
HumanEval	54.88	57.11	53.66	15.24	1.02
BFCL Simple	92.00	92.42	91.50	82.25	18.75
BFCL Multi	89.00	88.33	89.00	52.50	13.67
BFCL Parallel	84.00	83.67	82.50	30.00	3.33
BFCL Parallel-Multi	78.00	83.33	82.00	37.00	1.33

Supported Models

Any HuggingFace model can be converted using cactus convert [HF-Name], though experimental.
Liquid, Gemma. whisper. parakeet and Qwen model families are especially tested.
Some models have been pre-uploaded here, just run cactus download [HF-Name].
cactus run [HF-Name] albeit first downloads or convert the model if not found.

Needle

Needle is a 26m parameter model for on-device tool calling:

cactus run Cactus-Compute/needle [--tools my_tools.json]  # OpenAI function-calling format; demo toolset by default

Learn More

Reference	Language	Description
Cactus Engine	C	Chat completion, streaming, tool calling, transcription, embeddings, RAG, vision, vector index, cloud handoff
Cactus Graph	C++	Tensor operations, matrix multiplication, attention, normalization, activation functions
Cactus Kernels	C++	ARM NEON SIMD kernels for matmul, attention, convolution, quantization, DSP, image processing
Cactus Quants	C++	Rotation-and-codebook quantization from 4-bit to 1-bit for all weight tensors
Cactus Hybrid	C/Python	Route hard queries to the cloud automatically based on local model confidence
Python Package	Python	Python package and CLI

Bindings

Using this repo

┌────────────────────────────────────────────────────────────────────────────────┐
│                                                                                │
│ Step 0: if on Linux (Ubuntu/Debian)                                            │
│ sudo apt-get install python3.12 python3.12-venv python3-pip cmake              │
│   build-essential libcurl4-openssl-dev                                         │
│                                                                                │
│ Step 1: clone and setup                                                        │
│ git clone https://github.com/cactus-compute/cactus && cd cactus                │
│ source ./setup                                                                 │
│                                                                                │
│ Step 2: use the commands                                                       │
│────────────────────────────────────────────────────────────────────────────────│
│                                                                                │
│  cactus auth                         manage cloud API key                      │
│    --status                          show key status                           │
│    --clear                           remove saved key                          │
│                                                                                │
│  cactus run [model|path]             run a model (downloads if needed)         │
│    --bits 1|2|3|4|2.54|3.26          CQ quantization (default: 4)              │
│    --backend cpu|metal               inference backend (default: auto)         │
│    --image <path>                    image file for VLM inference              │
│    --audio <path>                    audio file for audio chat                 │
│    --system <prompt>                 system prompt                             │
│    --prompt <text>                   send prompt immediately                   │
│    --tools <json|file>               tool definitions for tool calling         │
│    --thinking                        enable thinking/reasoning mode            │
│    --token <token>                   HuggingFace token (gated models)          │
│    --reconvert                       force local rebuild from source           │
│                                                                                │
│  cactus transcribe [model]           live microphone transcription with a model│
│    --file <audio.wav>                audio file to transcribe (WAV)            │
│    --language <code>                 language code (default: en)               │
│    --bits 1|2|3|4|2.54|3.26          CQ quantization (default: 4)              │
│    --token <token>                   HuggingFace token (gated models)          │
│    --reconvert                       force local rebuild from source           │
│                                                                                │
│  cactus download [model]             get a bundle (prebuilt, else build)       │
│    --bits 1|2|3|4|2.54|3.26          CQ quantization (default: 4)              │
│    --token <token>                   HuggingFace token (gated models)          │
│    --reconvert                       refresh cached bundle                     │
│                                                                                │
│  cactus convert <model> [dir]        HuggingFace -> Cactus CQ weights          │
│    --bits 1|2|3|4                    CQ quantization (default: 4)              │
│    --token <token>                   HuggingFace token (gated models)          │
│    --reconvert                       force weight conversion from source       │
│    --lora <path>                     merge a LoRA adapter before converting    │
│    --weights-only                    stop after CQ weights (skip the graph)    │
│    --artifact-dir <path>             bundle output (default: weights/<model>)  │
│                                                                                │
│  cactus serve [model]                OpenAI-compatible local HTTP server       │
│    --host <addr>                     bind address (default: 127.0.0.1)         │
│    --port <port>                     port (default: 8080)                      │
│    --bits 1|2|3|4|2.54|3.26          CQ quantization (default: 4)              │
│    --backend cpu|metal               inference backend (default: auto)         │
│    --token <token>                   HuggingFace token (gated models)          │
│    --reconvert                       force local rebuild from source           │
│    --no-cloud-handoff                disable automatic cloud handoff           │
│    --confidence-threshold <0..1>     handoff to cloud below this confidence    │
│    --cloud-timeout-ms <n>            max wait for cloud handoff                │
│                                                                                │
│  cactus code                         run the AI coding agent (TUI / print)     │
│    --serve-model <id>                auto-start a server with this model       │
│    --bits 1|2|3|4|2.54|3.26          CQ quantization (default: 4)              │
│    --backend cpu|metal               inference backend (default: auto)         │
│    --token <token>                   HuggingFace token (gated models)          │
│    --reconvert                       force local rebuild from source           │
│    --host <addr>                     server address (default: 127.0.0.1)       │
│    --port <port>                     server port (default: 8080)               │
│    --no-serve                        require a running server (no auto-start)  │
│    --no-cloud-handoff                disable automatic cloud handoff           │
│    --confidence-threshold <0..1>     handoff to cloud below this confidence    │
│    --cloud-timeout-ms <n>            max wait for cloud handoff                │
│    -- <args...>                      pass remaining args to the agent          │
│                                                                                │
│  cactus list                         list downloaded models                    │
│                                                                                │
│  cactus build                        build cactus libraries                    │
│    --apple                           Apple (iOS/macOS)                         │
│    --android                         Android                                   │
│    --python                          shared lib for Python FFI                 │
│                                                                                │
│  cactus test                         run the test suite                        │
│    --component <name>                kernels | graph | engine | all            │
│                                      (default: all)                            │
│    --model <hf-id>                   default: google/gemma-4-E2B-it            │
│    --transcription-model <hf-id>     default: nvidia/parakeet-tdt-0.6b-v3      │
│    --bits 1|2|3|4|2.54|3.26          CQ quantization (default: 4)              │
│    --backend cpu|metal               inference backend (default: auto)         │
│    --token <token>                   HuggingFace token (gated models)          │
│    --reconvert                       force local rebuild of test models        │
│    --suite <name>                    run a single test suite by name           │
│                                      (resolved across components,              │
│                                      e.g. llm → engine)                        │
│    --list                            list components and suites                │
│    --ios                             run on connected iPhone                   │
│    --android                         run on connected Android                  │
│    --enable-telemetry                send cloud telemetry (off by default)     │
│                                                                                │
│  cactus benchmark                    run the engine benchmark suite            │
│    --model <hf-id>                   default: google/gemma-4-E2B-it            │
│    --transcription-model <hf-id>     default: nvidia/parakeet-tdt-0.6b-v3      │
│    --bits 1|2|3|4|2.54|3.26          CQ quantization (default: 4)              │
│    --backend cpu|metal               inference backend (default: auto)         │
│    --ios                             run on connected iPhone                   │
│    --android                         run on connected Android                  │
│                                                                                │
│  cactus clean                        delete build artifacts, weights, venv     │
│  cactus --help                       show this help                            │
│                                                                                │
└────────────────────────────────────────────────────────────────────────────────┘

Citation

If you use Cactus in your research, please cite it as follows:

@software{cactus,
  title        = {Cactus: AI Inference Engine for Phones & Wearables},
  author       = {Ndubuaku, Henry and Cactus Team},
  url          = {https://github.com/cactus-compute/cactus},
  year         = {2025}
}

N/B: Scroll all the way up and click the shields link for resources!

microsoft/BitNet

Official inference framework for 1-bit LLMs

bitnet.cpp

📰 News

07/23/2026: 📣 We released VibeASR.cpp — a real-time multilingual ASR inference engine on CPU using BitNet I2_S quantization, achieving RTF < 1 with very few threads on x86 (AVX2) and ARM (NEON) platforms. [Code] [Models] [Report]

07/20/2026: 📣 We released BitNet-embedding-0.6B and BitNet-embedding-270M on Hugging Face — the first 1-bit embedding models that deliver competitive embedding quality with significantly faster inference on CPUs.

1.42x to 2.28x speedup over F16 on BitNet-embedding-0.6B prefill (8 threads)
1.32x to 1.74x speedup over F16 on BitNet-embedding-270M prefill (8 threads)
Supports I2_S conversion with optimized kernels on x86 CPUs
Lossless inference with 2 bits per weight

07/16/2026: 📣 Released BitNet Embeddings 0.6B/270M: I2_S Conversion and Inference Optimization — detailed guide for converting and running BitNet embedding models with optimized I2_S kernels.

01/15/2026: 📣 Released BitNet CPU Inference Optimization — parallel kernel implementations with configurable tiling and embedding quantization support, achieving 1.15x to 2.1x additional speedup over the original implementation.

05/20/2025: 📣 Released BitNet Official GPU inference kernel — extending 1-bit inference beyond CPUs.

04/14/2025: 📣 Released BitNet Official 2B Parameter Model on Hugging Face — the first official BitNet b1.58 model trained with 4T tokens.

02/18/2025: 📑 Bitnet.cpp: Efficient Edge Inference for Ternary LLMs — system-level paper on bitnet.cpp's architecture and design.

11/08/2024: 📑 BitNet a4.8: 4-bit Activations for 1-bit LLMs — enabling 4-bit activations for further efficiency gains.

10/21/2024: 📑 1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs — the technical report behind bitnet.cpp.

10/17/2024: 📣 bitnet.cpp 1.0 released.

03/21/2024: 📑 The-Era-of-1-bit-LLMs: Training Tips, Code, FAQ

02/27/2024: 📑 The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — the foundational paper introducing BitNet b1.58.

10/17/2023: 📑 BitNet: Scaling 1-bit Transformers for Large Language Models — the original BitNet paper.

Overview

bitnet.cpp is the official inference framework for 1-bit LLMs (e.g., BitNet b1.58). It offers a suite of optimized kernels that support fast and lossless inference of 1.58-bit models on CPU and GPU (NPU support coming next).

Try it out via this online demo, or build and run it on your own CPU or GPU.

bitnet.cpp achieves speedups of 1.37x to 5.07x on ARM CPUs, with larger models experiencing greater performance gains. Additionally, it reduces energy consumption by 55.4% to 70.0%, further boosting overall efficiency. On x86 CPUs, speedups range from 2.37x to 6.17x with energy reductions between 71.9% to 82.2%. Furthermore, bitnet.cpp can run a 100B BitNet b1.58 model on a single CPU, achieving speeds comparable to human reading (5-7 tokens per second), significantly enhancing the potential for running LLMs on local devices. Please refer to the technical report for more details.

Model Releases

1. BitNet-b1.58-2B-4T - 1-bit Large Language Model

BitNet-b1.58-2B-4T is the first official BitNet b1.58 model with 2.4B parameters, trained on 4 trillion tokens. It is a ternary (1.58-bit) language model that delivers competitive performance with full-precision models of similar size while enabling significantly faster and more energy-efficient inference.

Fast CPU Inference: Achieves up to 6.17x speedup on x86 CPUs and 5.07x on ARM CPUs compared to full-precision models.
Energy Efficient: Reduces energy consumption by up to 82.2% on x86 and 70.0% on ARM.
GPU Support: Official GPU inference kernel available for accelerated deployment.
Chat-Ready: Supports conversational mode for interactive use.

🤗 Hugging Face | 🔗 Online Demo | 📄 Technical Report

2. BitNet-embedding-0.6B - 1-bit Embedding Model

BitNet-embedding-0.6B is a 0.6B-parameter 1-bit embedding model that achieves competitive embedding quality with significantly faster CPU inference. It is the first model to demonstrate that ternary weights can deliver strong performance on embedding tasks.

1.42x to 2.28x speedup over F16 on prefill (8 threads, x86)
Lossless Quality: Competitive embedding quality with 2 bits per weight
I2_S Kernel: Supports optimized I2_S conversion on x86 CPUs

🤗 Hugging Face | 📄 I2_S Guide

3. BitNet-embedding-270M - Lightweight 1-bit Embedding Model

BitNet-embedding-270M is a compact 270M-parameter 1-bit embedding model designed for resource-constrained environments, offering fast inference with minimal memory footprint.

1.32x to 1.74x speedup over F16 on prefill (8 threads, x86)
Lossless Quality: Competitive embedding quality with 2 bits per weight
Lightweight: Only 270M parameters for edge deployment scenarios

🤗 Hugging Face | 📄 I2_S Guide

Supported Models

Model	Parameters	CPU	Kernel
Model	Parameters	CPU	I2_S	TL1	TL2
Official Models
BitNet-b1.58-2B-4T	2.4B	x86	✅	❌	✅
BitNet-b1.58-2B-4T	2.4B	ARM	✅	✅	❌
BitNet-embedding-0.6B	0.6B	x86	✅	❌	❌
BitNet-embedding-0.6B	0.6B	ARM	❌	❌	❌
BitNet-embedding-270M	270M	x86	✅	❌	❌
BitNet-embedding-270M	270M	ARM	❌	❌	❌
Community Models
bitnet_b1_58-large	0.7B	x86	✅	❌	✅
bitnet_b1_58-large	0.7B	ARM	✅	✅	❌
bitnet_b1_58-3B	3.3B	x86	❌	❌	✅
bitnet_b1_58-3B	3.3B	ARM	❌	✅	❌
Llama3-8B-1.58-100B-tokens	8.0B	x86	✅	❌	✅
Llama3-8B-1.58-100B-tokens	8.0B	ARM	✅	✅	❌
Falcon3 Family	1B-10B	x86	✅	❌	✅
Falcon3 Family	1B-10B	ARM	✅	✅	❌
Falcon-E Family	1B-3B	x86	✅	❌	✅
Falcon-E Family	1B-3B	ARM	✅	✅	❌

❗️We use existing 1-bit LLMs available on Hugging Face to demonstrate the inference capabilities of bitnet.cpp. We hope the release of bitnet.cpp will inspire the development of 1-bit LLMs in large-scale settings in terms of model size and training tokens.

Installation

Requirements

python>=3.10
cmake>=3.22
clang>=18
- For Windows users, install Visual Studio 2022. In the installer, toggle on at least the following options(this also automatically installs the required additional tools like CMake):
  - Desktop-development with C++
  - C++-CMake Tools for Windows
  - Git for Windows
  - C++-Clang Compiler for Windows
  - MS-Build Support for LLVM-Toolset (clang)
- For Debian/Ubuntu users, you can download with Automatic installation script
  
  bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
conda (highly recommend)

Build from source

Important

If you are using Windows, please remember to always use a Developer Command Prompt / PowerShell for VS2022 for the following commands. Please refer to the FAQs below if you see any issues.

Clone the repo

git clone --recursive https://github.com/microsoft/BitNet.git
cd BitNet

Install the dependencies

# (Recommended) Create a new conda environment
conda create -n bitnet-cpp python=3.10
conda activate bitnet-cpp

pip install -r requirements.txt

Build the project

# Manually download the model and run with local path
huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4T
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s

usage: setup_env.py [-h] [--hf-repo {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens,tiiuae/Falcon3-1B-Instruct-1.58bit,tiiuae/Falcon3-3B-Instruct-1.58bit,tiiuae/Falcon3-7B-Instruct-1.58bit,tiiuae/Falcon3-10B-Instruct-1.58bit}] [--model-dir MODEL_DIR] [--log-dir LOG_DIR] [--quant-type {i2_s,tl1}] [--quant-embd]
                    [--use-pretuned]

Setup the environment for running inference

optional arguments:
  -h, --help            show this help message and exit
  --hf-repo {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens,tiiuae/Falcon3-1B-Instruct-1.58bit,tiiuae/Falcon3-3B-Instruct-1.58bit,tiiuae/Falcon3-7B-Instruct-1.58bit,tiiuae/Falcon3-10B-Instruct-1.58bit}, -hr {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens,tiiuae/Falcon3-1B-Instruct-1.58bit,tiiuae/Falcon3-3B-Instruct-1.58bit,tiiuae/Falcon3-7B-Instruct-1.58bit,tiiuae/Falcon3-10B-Instruct-1.58bit}
                        Model used for inference
  --model-dir MODEL_DIR, -md MODEL_DIR
                        Directory to save/load the model
  --log-dir LOG_DIR, -ld LOG_DIR
                        Directory to save the logging info
  --quant-type {i2_s,tl1}, -q {i2_s,tl1}
                        Quantization type
  --quant-embd          Quantize the embeddings to f16
  --use-pretuned, -p    Use the pretuned kernel parameters

Usage

Basic usage

# Run inference with the quantized model
python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "You are a helpful assistant" -cnv

usage: run_inference.py [-h] [-m MODEL] [-n N_PREDICT] -p PROMPT [-t THREADS] [-c CTX_SIZE] [-temp TEMPERATURE] [-cnv]

Run inference

optional arguments:
  -h, --help            show this help message and exit
  -m MODEL, --model MODEL
                        Path to model file
  -n N_PREDICT, --n-predict N_PREDICT
                        Number of tokens to predict when generating text
  -p PROMPT, --prompt PROMPT
                        Prompt to generate text from
  -t THREADS, --threads THREADS
                        Number of threads to use
  -c CTX_SIZE, --ctx-size CTX_SIZE
                        Size of the prompt context
  -temp TEMPERATURE, --temperature TEMPERATURE
                        Temperature, a hyperparameter that controls the randomness of the generated text
  -cnv, --conversation  Whether to enable chat mode or not (for instruct models.)
                        (When this option is turned on, the prompt specified by -p will be used as the system prompt.)

Demo

A demo of bitnet.cpp running a BitNet b1.58 3B model on Apple M2:

https://github.com/user-attachments/assets/7f46b736-edec-4828-b809-4be780a3e5b1

Benchmark

We provide scripts to run the inference benchmark providing a model.

usage: e2e_benchmark.py -m MODEL [-n N_TOKEN] [-p N_PROMPT] [-t THREADS]  
   
Setup the environment for running the inference  
   
required arguments:  
  -m MODEL, --model MODEL  
                        Path to the model file. 
   
optional arguments:  
  -h, --help  
                        Show this help message and exit. 
  -n N_TOKEN, --n-token N_TOKEN  
                        Number of generated tokens. 
  -p N_PROMPT, --n-prompt N_PROMPT  
                        Prompt to generate text from. 
  -t THREADS, --threads THREADS  
                        Number of threads to use.

Here's a brief explanation of each argument:

-m, --model: The path to the model file. This is a required argument that must be provided when running the script.
-n, --n-token: The number of tokens to generate during the inference. It is an optional argument with a default value of 128.
-p, --n-prompt: The number of prompt tokens to use for generating text. This is an optional argument with a default value of 512.
-t, --threads: The number of threads to use for running the inference. It is an optional argument with a default value of 2.
-h, --help: Show the help message and exit. Use this argument to display usage information.

For example:

python utils/e2e_benchmark.py -m /path/to/model -n 200 -p 256 -t 4

This command would run the inference benchmark using the model located at /path/to/model, generating 200 tokens from a 256 token prompt, utilizing 4 threads.

For the model layout that do not supported by any public model, we provide scripts to generate a dummy model with the given model layout, and run the benchmark on your machine:

python utils/generate-dummy-bitnet-model.py models/bitnet_b1_58-large --outfile models/dummy-bitnet-125m.tl1.gguf --outtype tl1 --model-size 125M

# Run benchmark with the generated model, use -m to specify the model path, -p to specify the prompt processed, -n to specify the number of token to generate
python utils/e2e_benchmark.py -m models/dummy-bitnet-125m.tl1.gguf -p 512 -n 128

Convert from `.safetensors` Checkpoints

# Prepare the .safetensors model file
huggingface-cli download microsoft/bitnet-b1.58-2B-4T-bf16 --local-dir ./models/bitnet-b1.58-2B-4T-bf16

# Convert to gguf model
python ./utils/convert-helper-bitnet.py ./models/bitnet-b1.58-2B-4T-bf16

Acknowledgements

This project is based on the llama.cpp framework. We would like to thank all the authors for their contributions to the open-source community. Also, bitnet.cpp's kernels are built on top of the Lookup Table methodologies pioneered in T-MAC. For inference of general low-bit LLMs beyond ternary models, we recommend using T-MAC.

FAQ (Frequently Asked Questions)📌

Q1: The build dies with errors building llama.cpp due to issues with std::chrono in log.cpp?

A: This is an issue introduced in recent version of llama.cpp. Please refer to this commit in the discussion to fix this issue.

Q2: How to build with clang in conda environment on windows?

A: Before building the project, verify your clang installation and access to Visual Studio tools by running:

clang -v

This command checks that you are using the correct version of clang and that the Visual Studio tools are available. If you see an error message such as:

'clang' is not recognized as an internal or external command, operable program or batch file.

It indicates that your command line window is not properly initialized for Visual Studio tools.

• If you are using Command Prompt, run:

"C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\Tools\VsDevCmd.bat" -startdir=none -arch=x64 -host_arch=x64

• If you are using Windows PowerShell, run the following commands:

Import-Module "C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\Tools\Microsoft.VisualStudio.DevShell.dll" Enter-VsDevShell 3f0e31ad -SkipAutomaticLocation -DevCmdArguments "-arch=x64 -host_arch=x64"

These steps will initialize your environment and allow you to use the correct Visual Studio tools.

k4yt3x/video2x

A machine learning-based video super resolution and frame interpolation framework. Est. Hack the Valley II, 2018.

🌟 Version 6.0.0

Video2X 6.0.0 highlights:

Complete rewrite of the Video2X project in C/C++.
Faster and more efficient architecture.
Cross-platform support for Windows and Linux.
Vastly improved output quality.
New GUI and installer for easy setup on Windows.

Click to see more details

Version 6.0.0 is a complete rewrite of this project in C/C++. It:

genuinely works this time, with much less hassle compared to the 5.0.0 beta;
is blazing fast, thanks to the new optimized pipeline and the efficiency of C/C++;
is cross-platform, available now for both Windows and Linux;
offers significantly better output quality with Anime4K v4, Real-ESRGAN, Real-CUGAN, and RIFE;
supports two modes: filtering (upscaling) and frame interpolation;
supports Anime4K v4 and all custom MPV-compatible GLSL shaders;
supports Real-ESRGAN, Real-CUGAN, and RIFE (all models) via ncnn and Vulkan;
requires zero additional disk space during processing, just space for the final output.

🖥️ Hardware Requirements

Your system must meet the minimum hardware requirements below to run Video2X.

CPU
- The precompiled binaries require CPUs with AVX2 support.
- Intel: Haswell (Q2 2013) or newer
- AMD: Excavator (Q2 2015) or newer
GPU
- The GPU must support Vulkan.
- NVIDIA: Kepler (GTX 600 series, Q2 2012) or newer
- AMD: GCN 1.0 (Radeon HD 7000 series, Q1 2012) or newer
- Intel: HD Graphics 4000 (Q2 2012) or newer

🪟 Install on Windows

Download the Latest Windows Installer Executable (6.4.0)

You can download the latest Windows release on the releases page. For basic GUI usage, refer to the documentation. If you're unable to download directly from GitHub, try the mirror site. The GUI currently supports the following languages:

English (United States)
简体中文（中国）
日本語（日本）
Português (Portugal)
Français (France)
Deutsch (Deutschland)

🐧 Install on Linux

Video2X packages are available for the Linux distros listed below. A universal AppImage is also available for other distros. If you'd like to build it from source code, refer to the PKGBUILD file for a general overview of the required dependencies and commands.

Arch Linux: AUR packages, maintained by @K4YT3X.
Arch Linux (Chinese Mainland): archlinuxcn packages, maintained by @Integral-Tech.
Other distros: Video2X-x86_64.AppImage on the releases page.

📦 Container Image

Video2X container images are available on the GitHub Container Registry for easy deployment on Linux and macOS. If you already have Docker/Podman installed, only one command is needed to start upscaling a video. For more information on how to use Video2X's Docker image, please refer to the documentation.

📔 Google Colab

You can use Video2X on Google Colab for free if you don't have a powerful GPU of your own. You can borrow a powerful GPU (NVIDIA T4, L4, or A100) on Google's server for free for a maximum of 12 hours per session. Please use the free resource fairly and do not create sessions back-to-back and run upscaling 24/7. This might result in you getting banned. You can get Colab Pro/Pro+ if you'd like to use better GPUs and get longer runtimes. Usage instructions are embedded in the Colab Notebook.

💬 Telegram Discussion Group

Join our Telegram discussion group to ask any questions you have about Video2X, chat directly with the developers, or discuss super resolution, frame interpolation technologies, or the future of Video2X in general.

📖 Documentation

Comprehensive documentation for Video2X is available at https://docs.video2x.org/. It offers detailed instructions on how to build, install, use, and develop with this program.

📽️ Video Demos (Outdated)

Upscale demo: Spirited Away's movie trailer

Spirited Away: YouTube | Bilibili
- 360P to 4K
- The original video's copyright belongs to 株式会社スタジオジブリ
Bad Apple!!: YouTube | Bilibili
- 384P 30 FPS to 4K 120 FPS with waifu2x and DAIN
- The original video's copyright belongs to あにら
The Pet Girl of Sakurasou: YouTube | Bilibili
- 240P 29.97 to 1080P 60 FPS with waifu2x and DAIN
- The original video's copyright belongs to ASCII Media Works

Standard Test Clip

The following clip can be used to test if your setup works properly. This is also the standard clip used for running performance benchmarks.

Standard Test Clip (240P) 4.54 MiB
Real-CUGAN Upscaled Sample (1704P) 3.5 MiB
Real-ESRGAN Upscaled Sample (1704P) 3.1 MiB
waifu2x Upscaled Sample (1080P) 4.54 MiB
Ground Truth (1080P) 22.2 MiB

The original clip came from the anime "さくら荘のペットな彼女."
Copyright of this clip belongs to 株式会社アニプレックス.

⚖️ License

This project is licensed under GNU AGPL version 3.
Copyright (C) 2018-2025 K4YT3X and contributors.

This project includes or depends on these following projects:

Project	License
FFmpeg/FFmpeg	LGPLv2.1, GPLv2
Tencent/ncnn	BSD 3-Clause
bloc97/Anime4K	MIT License
nihui/realcugan-ncnn-vulkan	MIT License
nihui/rife-ncnn-vulkan	MIT License
xinntao/Real-ESRGAN-ncnn-vulkan	MIT License

More licensing information can be found in the NOTICE file.

🌺 Special Thanks

Special thanks to the following individuals for their significant contributions to the project, listed in alphabetical order.

ml-explore/mlx

MLX: An array framework for Apple silicon

MLX

Quickstart | Installation | Documentation | Examples

MLX is an array framework for machine learning on Apple silicon, brought to you by Apple machine learning research.

Some key features of MLX include:

Familiar APIs: MLX has a Python API that closely follows NumPy. MLX also has fully featured C++, C, and Swift APIs, which closely mirror the Python API. MLX has higher-level packages like mlx.nn and mlx.optimizers with APIs that closely follow PyTorch to simplify building more complex models.
Composable function transformations: MLX supports composable function transformations for automatic differentiation, automatic vectorization, and computation graph optimization.
Lazy computation: Computations in MLX are lazy. Arrays are only materialized when needed.
Dynamic graph construction: Computation graphs in MLX are constructed dynamically. Changing the shapes of function arguments does not trigger slow compilations, and debugging is simple and intuitive.
Multi-device: Operations can run on any of the supported devices (currently the CPU and the GPU).
Unified memory: A notable difference from MLX and other frameworks is the unified memory model. Arrays in MLX live in shared memory. Operations on MLX arrays can be performed on any of the supported device types without transferring data.

MLX is designed by machine learning researchers for machine learning researchers. The framework is intended to be user-friendly, but still efficient to train and deploy models. The design of the framework itself is also conceptually simple. We intend to make it easy for researchers to extend and improve MLX with the goal of quickly exploring new ideas.

The design of MLX is inspired by frameworks like NumPy, PyTorch, Jax, and ArrayFire.

Examples

The MLX examples repo has a variety of examples, including:

Transformer language model training.
Large-scale text generation with LLaMA and finetuning with LoRA.
Generating images with Stable Diffusion.
Speech recognition with OpenAI's Whisper.

Quickstart

See the quick start guide in the documentation.

Installation

MLX is available on PyPI. To install MLX on macOS, run:

pip install mlx

To install the CUDA backend on Linux, run:

pip install mlx[cuda]

To install a CPU-only Linux package, run:

pip install mlx[cpu]

Checkout the documentation for more information on building the C++ and Python APIs from source.

Contributing

Check out the contribution guidelines for more information on contributing to MLX. See the docs for more information on building from source, and running tests.

We are grateful for all of our contributors. If you contribute to MLX and wish to be acknowledged, please add your name to the list in your pull request.

Citing MLX

The MLX software suite was initially developed with equal contribution by Awni Hannun, Jagrit Digani, Angelos Katharopoulos, and Ronan Collobert. If you find MLX useful in your research and wish to cite it, please use the following BibTex entry:

@software{mlx2023,
  author = {Awni Hannun and Jagrit Digani and Angelos Katharopoulos and Ronan Collobert},
  title = {{MLX}: Efficient and flexible machine learning on Apple silicon},
  url = {https://github.com/ml-explore},
  version = {0.0},
  year = {2023},
}

PetoiCamp/OpenCat-Quadruped-Robot

An open source quadruped robot pet framework for developing Boston Dynamics-style four-legged robots that are perfect for STEM, coding & robotics education, IoT robotics applications, AI-enhanced robotics application services, research, and DIY robotics kit development.

OpenCat — Open-Source Quadruped Robot Framework

On current-gen hardware (Bittle X, Nybble Q) or starting fresh? This repo covers the legacy NyBoard platform. Head to OpenCatESP32 for current hardware and active development, or buy Bittle X on Amazon.

🚀 Quaddle — Petoi's newest OpenCat-lineage quadruped — launches on Kickstarter August 2026. Reserve your spot → (details below)

OpenCat is an open-source Arduino and Raspberry Pi-based framework for building and programming quadruped robots. Developed by Petoi, the maker of futuristic programmable robotic pets, it's the framework behind two mini robot kits: the Bittle robot dog and the Nybble robot cat.

The framework handles the hard parts — gait coordination, servo control, IMU integration — so you can focus on what you're actually building on top of it.

Coming Soon: Quaddle mini robot dog

Quaddle is Petoi's newest quadruped, launching on Kickstarter August 2026 — a mini desk robot built on the same OpenCat lineage as Bittle and Nybble. What makes it worth a look: it's a full quadruped running on just 4 servos instead of the usual 8–12, which forces genuinely different gait-design and leg-coordination solutions to still get all four legs walking, running, and balancing. Its servos are also position-feedback — readable, not just drivable — which is what makes Puppet Mode possible: hand-guide the legs and record a motion directly, no code required to capture a new behavior. We'll also share 3D-printable shells and mounts for customization. Same open platform this repo already supports for gait research, RL, and sim2real work — a hands-on physical AI platform, not a toy version of the idea.

Source code is not yet public. It's an upgraded version of the OpenCat project. The ESP32-S3 code structure is similar to this repo and closer to OpenCatESP32, which already powers thousands of Bittle X and Nybble Q robots in the field. We plan to open source it before Quaddle delivery. Watch this repo and r/petoi for the announcement, or reserve a spot to get notified at launch.

About the Project

Inspired by Boston Dynamics' Spot, Dr. Rongzhong Li started OpenCat in his dorm at Wake Forest University in 2016. The goal was straightforward: make agile quadruped robots affordable and hackable enough for researchers, educators, and makers — not just well-funded labs.

Since then:

30,000+ robots shipped, cumulative across all channels
60+ countries
$1M+ raised across two prior Kickstarter and Indiegogo campaigns
20+ academic papers cite the platform — see Research Spotlight
One of the most-starred open-source quadruped robot projects on GitHub

OpenCat is now deployed across two robot platforms and used in K-12 schools, university research labs, maker spaces, and independent projects worldwide.

Current Hardware

🐶 Bittle X — mini robot dog & AI robotics kit with voice control — current generation, BiBoard/ESP32
🐱 Nybble Q — mini robot cat & AI robotics kit — current generation

The original Bittle and Nybble mini robots (NyBoard/ATmega328P) are discontinued but still fully supported by this codebase. This repo is the right place for NyBoard users; for ESP32/BiBoard see OpenCatESP32.

What the Framework Gives You

Multi-language: C/C++, Python, block-based coding
Open hardware + software: fork, modify, extend — no vendor lock-in
Arduino + Raspberry Pi native: works with the tools you already use
Sensor integration: cameras, touch, IMUs, ultrasonic, etc. — clip and go
Sim-to-real: experiment reinforcement learning models on an affordable robot
ROS compatible: SLAM, navigation, and perception pipelines documented by the community

Whether you're teaching robotics programming for the first time, building a weekend hackathon project, or running a graduate-level embodied robotics experiment, the platform scales.

What the Community Has Built

Users have shipped real robotics projects across AI, Raspberry Pi, and research — a few highlights:

Academic and research use cases: Research Spotlight

Hardware

This repo targets the NyBoard — a customized Arduino board based on ATmega328P. It coordinates up to 12 high-performance servos for walking, running, jumping, and backflipping.

Extend with:

Sensor pack
Intelligent camera module
Raspberry Pi, Nvidia Jetson Nano, or other AI co-processors via wired/wireless connections

For ESP32/BiBoard (current-gen hardware), see OpenCatESP32.

Setup

Full documentation: Petoi Doc Center The following steps give you a brief walk-through, but may miss detailed tips and illustrative pictures.

1. Clone the repo

Remove the -main (or branch name) suffix from the folder after downloading.

2. Open OpenCat.ino — select your robot and board

#define BITTLE    // Petoi 9 DOF robot dog: 1x on head + 8x on leg
//#define NYBBLE  // Petoi 11 DOF robot cat: 2x on head + 1x on tail + 8x on leg

//#define NyBoard_V0_1
//#define NyBoard_V0_2
#define NyBoard_V1_0
//#define NyBoard_V1_1

3. Enter configuration mode

Comment out #define MAIN_SKETCH, upload, follow serial prompts.

// #define MAIN_SKETCH

4. (Optional) Auto-init

#define AUTO_INIT skips prompts and auto-calibrates IMU — handy for repeated flashing.

5. Upload

Plug USB uploader into NyBoard, install driver if needed, hit upload (→) in Arduino IDE.

6. Serial Monitor

Set to no line ending and 115200 baud rate.

7. Reset joint offsets

Reset joint offsets? (Y/n)
Y

8. IMU calibration

Calibrate the IMU? (Y/n):
Y

Place the robot flat on a table. 6 long beeps → reads sensor data → saves offsets → beeps when done. Close Serial Monitor when Ready! appears.

9. Switch to main sketch

#define MAIN_SKETCH

Uncomment and re-upload.

10. Joint calibration

Boot with one side up to enter calibration mode, or calibrate directly via Serial Monitor.

11. Optional: Petoi app

Plug in Bluetooth dongle for a friendlier calibration UI:

iOS: App Store
Android: Google Play

See the calibration guide and app guide.

12. Start building

Infrared remote, Petoi app, Python, or Serial Monitor. Full play guide →

Education

OpenCat shows up in AI robotics education across K-12 programs, community colleges, university labs, and maker spaces — hands-on physical AI teaching that goes beyond screen-based coding exercises:

Community & Discussion

r/OpenCat — firmware code, framework hacking, extending and porting OpenCat
r/Petoi — hardware Q&A, builds, quadruped coding, curriculum, 3D-printed parts, general discussion

Resources

Follow the project: YouTube · Twitter · Instagram · Facebook · LinkedIn

The legacy OpenCat repository is archived and no longer maintained.

orailnoor/cross-platform-llm-client

A unified cross-platform AI client supporting seamless transitions between standard cloud APIs and on-device, offline execution of custom and uncensored language models.

PrivateLM

A production-ready, cross-platform AI chat client built with Flutter. It unifies local on-device LLM inference (Android) with cloud API access, giving users full control over how their models run.

Image generation tested on Moto G71 (Snapdragon), Oneplus 10r (Mediatek), Pixel 6A (Tensor), Poco F1 (Snapdragon), Samsung s23 (Snapdragon) 4 steps fast

Generated on pixel 6 with 20 step

What It Does

Local Inference on Android — Download and run GGUF models directly on your phone using GPU-accelerated inference (Vulkan). No internet required after download.
Cloud API Fallback — Seamlessly switch to OpenAI, Anthropic, Google Gemini, or Kimi (Moonshot AI) when you need more power or are on unsupported platforms.
Multimodal Chat — Send text and images in conversations. Vision support works with both local models (Qwen2-VL) and cloud providers.
Persistent Sessions — All chats, tasks, and settings are stored locally via Hive. Nothing leaves your device unless you explicitly choose cloud mode.
Background Services — Firebase Cloud Messaging integration for push updates and background task handling.
Smart Auto-Configuration — On first launch, the app detects your device's RAM and recommends optimal context size and token limits automatically.
Task Management — A dedicated task view for structured AI-assisted workflows alongside free-form chat.

Technical Architecture

Stack

Framework: Flutter 3.x (Dart >=3.3.0)
State Management: GetX
Local Storage: Hive
Networking: Dio + package:http
Background Execution: flutter_background_service + flutter_local_notifications
Push Notifications: Firebase Core + Firebase Messaging

Inference Pipeline

┌─────────────────────────────────────────────────────────────┐
│                        UI Layer                              │
│   ChatView / TaskView / ModelView / SettingsView            │
└──────────────────────────┬──────────────────────────────────┘
                           │
┌──────────────────────────▼──────────────────────────────────┐
│                    Controllers (GetX)                        │
│   ChatController · TaskController · ModelController         │
│   SettingsController · HomeController                       │
└──────────────────────────┬──────────────────────────────────┘
                           │
┌──────────────────────────▼──────────────────────────────────┐
│                      Services                                │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────┐ │
│  │ InferenceService│  │  CloudService   │  │DownloadSvc  │ │
│  │  (local GGUF)   │  │ (OpenAI/Claude/ │  │ (model dl)  │ │
│  │                 │  │  Gemini/Kimi)   │  │             │ │
│  └─────────────────┘  └─────────────────┘  └─────────────┘ │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────┐ │
│  │  HiveService    │  │ DeviceInfoSvc   │  │ExecutionSvc │ │
│  │  ( persistence) │  │  (RAM/GPU tier) │  │ (bg tasks)  │ │
│  └─────────────────┘  └─────────────────┘  └─────────────┘ │
└─────────────────────────────────────────────────────────────┘

Local Inference (Android)

The app uses llama_flutter_android, a custom Flutter plugin wrapping llama.cpp for ARM64 devices. At runtime it:

Detects GPU capabilities via Vulkan to determine offload layers.
Selects thread count based on device tier (ultra / high / mid / low).
Loads the GGUF model with progress streaming.
Generates tokens via generateChat() with native chat-template support (ChatML, Llama-3, Gemma, Phi).
Falls back to manual prompt construction if native templates fail.

Idle detection (5s) and hard timeouts (180s) keep the UX responsive even on underpowered hardware.

Cloud Inference

CloudService normalizes four different API shapes into a single interface:

OpenAI — standard /v1/chat/completions
Anthropic — Messages API with separate system param
Google Gemini — generateContent with inline image base64
Kimi — OpenAI-compatible endpoint from Moonshot AI

API keys are stored in Hive and never transmitted anywhere except to the provider's endpoint.

Cross-Platform Abstraction

Local inference is conditionally compiled:

Android → inference_android.dart (full llama.cpp engine)
Web → inference_stub.dart (cloud-only, local coming soon)
iOS → inference_android.dart (full llama.cpp engine via Metal GPU)

The InferenceService exposes supportsLocalInference so the UI can hide local-model UI on unsupported platforms.

Supported Platforms

Platform	Local Inference	Cloud APIs	Notes
Android	✅ Yes	✅ Yes	CPU offload via NEON; minSdk 28
iOS	✅ Yes	✅ Yes	Metal GPU acceleration
Web	❌ No	✅ Yes	Cloud-only (local coming soon)

iOS / iPad

The iPad release is distributed as a standalone ZIP package for sideloading. Download the latest PrivateLM-iOS.zip from the Releases page, extract it, and install the .ipa via AltStore, Sideloadly, or Xcode. iPhone support is experimental — iPad is the recommended iOS target due to RAM requirements for local models.

Build Configuration

Prerequisites

Flutter SDK >=3.3.0
Android SDK (API 26+)
JDK 17
NDK (bundled with Android SDK)

Android

flutter pub get
flutter build apk --debug

Release APKs require a stable signing key. Copy android/key.properties.example to android/key.properties, fill in the keystore values, and keep both the key and its backup. Android accepts an APK upgrade only when it is signed with the same key as the installed APK.

cp android/key.properties.example android/key.properties
flutter build apk --release --split-per-abi

Never rotate the signing key between GitHub releases. The per-ABI APKs must also keep increasing the build number in pubspec.yaml.

iOS

flutter pub get
cd ios
pod install
flutter build ios

Web

flutter pub get
flutter build web --release

License

MIT — see LICENSE for details.

cloudflare/workerd

The JavaScript / Wasm runtime that powers Cloudflare Workers

👷 `workerd`, Cloudflare's JavaScript/Wasm Runtime

workerd (pronounced: "worker-dee") is a JavaScript / Wasm server runtime based on the same code that powers Cloudflare Workers.

You might use it:

As an application server, to self-host applications designed for Cloudflare Workers.
As a development tool, to develop and test such code locally.
As a programmable HTTP proxy (forward or reverse), to efficiently intercept, modify, and route network requests.

Introduction

Design Principles

Server-first: Designed for servers, not CLIs nor GUIs.
Standard-based: Built-in APIs are based on web platform standards, such as fetch().
Nanoservices: Split your application into components that are decoupled and independently-deployable like microservices, but with performance of a local function call. When one nanoservice calls another, the callee runs in the same thread and process.
Homogeneous deployment: Instead of deploying different microservices to different machines in your cluster, deploy all your nanoservices to every machine in the cluster, making load balancing much easier.
Capability bindings: workerd configuration uses capabilities instead of global namespaces to connect nanoservices to each other and external resources. The result is code that is more composable -- and immune to SSRF attacks.
Always backwards compatible: Updating workerd to a newer version will never break your JavaScript code. workerd's version number is simply a date, corresponding to the maximum "compatibility date" supported by that version. You can always configure your worker to a past date, and workerd will emulate the API as it existed on that date.

Read the blog post to learn more about these principles.

WARNING: `workerd` is not a hardened sandbox

workerd tries to isolate each Worker so that it can only access the resources it is configured to access. However, workerd on its own does not contain suitable defense-in-depth against the possibility of implementation bugs. When using workerd to run possibly-malicious code, you must run it inside an appropriate secure sandbox, such as a virtual machine. The Cloudflare Workers hosting service in particular uses many additional layers of defense-in-depth.

With that said, if you discover a bug that allows malicious code to break out of workerd, please submit it to Cloudflare's bug bounty program for a reward.

Getting Started

Supported Platforms

In theory, workerd should work on any POSIX system that is supported by V8 and Windows.

In practice, workerd is tested on:

Linux and macOS (x86-64 and arm64 architectures)
Windows (x86-64 architecture)

On other platforms, you may have to do tinkering to make things work.

Building `workerd`

To build workerd, you need:

Bazel
- If you use Bazelisk (recommended), it will automatically download and use the right version of Bazel for building workerd.
On Linux:
- We use the clang/LLVM toolchain to build workerd and support version 19 and higher. Earlier versions of clang may still work, but are not officially supported.
- Clang 19+ (e.g. package clang-19 on Debian Trixie). If clang is installed as clang-<version> please create a symlink to it in your PATH named clang, or use --repo_env=CC=clang-<version> on bazel command lines to specify the compiler name.
- libc++ 19+ (e.g. packages libc++-19-dev and libc++abi-19-dev)
- LLD 19+ (e.g. package lld-19).
- python3, python3-distutils, and tcl8.6
On macOS:
- Xcode 16.3 installation (available on macOS 15 and higher). Building with just the Xcode Command Line Tools is not being tested, but should work too.
- Homebrew installed tcl-tk package (provides Tcl 8.6)
On Windows:
- Install App Installer from the Microsoft Store for the winget package manager and then run install-deps.bat from an administrator prompt to install bazelisk, LLVM, and other dependencies required to build workerd on Windows.
- Add startup --output_user_root=C:/tmp to the .bazelrc file in your user directory.
- When developing at the command-line, run bazel-env.bat in your shell first to select tools and Windows SDK versions before running bazel.

You may then build workerd at the command-line with:

bazel build //src/workerd/server:workerd

You can pass --config=release to compile in release mode:

bazel build //src/workerd/server:workerd --config=release

You can also build from within Visual Studio Code using the instructions in docs/vscode.md.

The compiled binary will be located at bazel-bin/src/workerd/server/workerd.

If you run a Bazel build before you've installed some dependencies (like clang or libc++), and then you install the dependencies, you must resync locally cached toolchains, or clean Bazel's cache, otherwise you might get strange errors:

bazel fetch --configure --force

If that fails, you can try:

bazel clean --expunge

The cache will now be cleaned and you can try building again.

If you have a fairly recent clang packages installed you can build a more performant release version of workerd:

bazel build --config=thin-lto //src/workerd/server:workerd

Configuring `workerd`

workerd is configured using a config file written in Cap'n Proto text format.

A simple "Hello World!" config file might look like:

using Workerd = import "/workerd/workerd.capnp";

const config :Workerd.Config = (
  services = [
    (name = "main", worker = .mainWorker),
  ],

  sockets = [
    # Serve HTTP on port 8080.
    ( name = "http",
      address = "*:8080",
      http = (),
      service = "main"
    ),
  ]
);

const mainWorker :Workerd.Worker = (
  serviceWorkerScript = embed "hello.js",
  compatibilityDate = "2023-02-28",
  # Learn more about compatibility dates at:
  # https://developers.cloudflare.com/workers/platform/compatibility-dates/
);

Where hello.js contains:

addEventListener("fetch", event => {
  event.respondWith(new Response("Hello World"));
});

Complete reference documentation is provided by the comments in workerd.capnp.

There is also a library of sample config files.

Running `workerd`

To serve your config, do:

workerd serve my-config.capnp

For more details about command-line usage, use workerd --help.

Prebuilt binaries are distributed via npm. Run npx workerd ... to use these. If you're running a prebuilt binary, you'll need to make sure your system has the right dependencies installed:

On Linux:
- glibc 2.35 or higher (already included on e.g. Ubuntu 22.04, Debian Bookworm)
On macOS:
- macOS 13.5 or higher
- The Xcode command line tools, which can be installed with xcode-select --install
x86_64 CPU with at least SSE4.2 and CLMUL ISA extensions, or arm64 CPU with CRC extension (enabled by default under armv8.1-a). These extensions are supported by all recent x86 and arm64 CPUs.

Local Worker development with `wrangler`

You can use Wrangler (v3.0 or greater) to develop Cloudflare Workers locally, using workerd. First, run the following command to configure Miniflare to use this build of workerd.

export MINIFLARE_WORKERD_PATH="<WORKERD_REPO_DIR>/bazel-bin/src/workerd/server/workerd"

Then, run:

wrangler dev

Serving in production

workerd is designed to be unopinionated about how it runs.

One good way to manage workerd in production is using systemd. Particularly useful is systemd's ability to open privileged sockets on workerd's behalf while running the service itself under an unprivileged user account. To help with this, workerd supports inheriting sockets from the parent process using the --socket-fd flag.

Here's an example system service file, assuming your config defines two sockets named http and https:

# /etc/systemd/system/workerd.service
[Unit]
Description=workerd runtime
After=local-fs.target remote-fs.target network-online.target
Requires=local-fs.target remote-fs.target workerd.socket
Wants=network-online.target

[Service]
Type=exec
ExecStart=/usr/bin/workerd serve /etc/workerd/config.capnp --socket-fd http=3 --socket-fd https=4
Sockets=workerd.socket

# If workerd crashes, restart it.
Restart=always

# Run under an unprivileged user account.
User=nobody
Group=nogroup

# Hardening measure: Do not allow workerd to run suid-root programs.
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

And corresponding sockets file:

# /etc/systemd/system/workerd.socket
[Unit]
Description=sockets for workerd
PartOf=workerd.service

[Socket]
ListenStream=0.0.0.0:80
ListenStream=0.0.0.0:443

[Install]
WantedBy=sockets.target

Once these files are in place you can enable the service -- see the systemd documentation or ask your favorite LLM for details.

ROCm/FastFlowLM

Run LLMs on AMD Ryzen™ AI NPUs in minutes. Just like Ollama - but purpose-built and deeply optimized for the AMD NPUs.

⚡ FastFlowLM (FLM) — Unlock Ryzen™ AI NPUs

Run large language models — now with Vision, Audio, Embedding and MoE support — on AMD Ryzen™ AI NPUs in minutes.
No GPU required. Faster and over 10× more power-efficient. Supports context lengths up to 256k tokens. Ultra-Lightweight (17 MB). Installs within 20 seconds.

📦 The only out-of-box, NPU-first runtime built exclusively for Ryzen™ AI.
🤝 Think Ollama — but deeply optimized for NPUs.
✨ From Idle Silicon to Instant Power — FastFlowLM Makes Ryzen™ AI Shine.

FastFlowLM (FLM) supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo, Kraken, and Gorgon Point).

🔗 Quick Links

🔽 Download | 📊 Benchmarks | 📦 Model List

🐧 Linux Getting Started Guide

📖 Docs | 📺 Demos | 💬 Discord

🚀 Quick Start

A packaged FLM Windows installer is available here: flm-setup.msi. For more details, see the release notes.

📺 Watch the quick start video (Windows)

Important

⚠️ Ensure NPU driver version is >= 32.0.203.304 (.304 is the minimum requirement but .311 is recommended; check via Task Manager→Performance→NPU or Device Manager).
⚙️ Tip:

RECOMMENDED: Try running Windows Update or Driver Download.
Official AMD Install Doc (AMD account required).
Unofficial forum downloads (CAUTION: third-party content not verified by AMD; download and use at your own risk).

After installation, open PowerShell (Win + X → I). To run a model in terminal (CLI Mode):

flm run llama3.2:1b

Notes:

Internet access to HuggingFace is required to download the optimized model kernels.

Sometimes downloads from HuggingFace may get corrupted. If this happens, run flm pull <model_tag> --force (e.g. flm pull llama3.2:1b --force) to re-download and fix them.

By default, models are stored in:

Windows: C:\Users\<USER>\.flm\models\

Linux: ~/.config/flm/

During installation on Windows, you can select a different base folder (e.g., if you choose C:\Users\<USER>\flm, models will be saved under C:\Users\<USER>\flm\models\).

On Linux, you can override the default location by setting the FLM_MODEL_PATH environment variable.

To disable the startup version check, set FLM_DISABLE_UPDATE_CHECK=1.

⚠️ If HuggingFace is not accessible in your region, manually download the model (check this issue) and place it in the chosen directory.

🎉🚀 FastFlowLM (FLM) is ready — your NPU is unlocked and you can start chatting with models right away!

Open Task Manager (Ctrl + Shift + Esc). Go to the Performance tab → click NPU to monitor usage.

⚡ Quick Tips:

Use /verbose during a session to turn on performance reporting (toggle off with /verbose again).

Type /bye to exit a conversation.

Run flm list in PowerShell to show all available models.

To start the local server (Server Mode):

flm serve llama3.2:1b

The model tag (e.g., llama3.2:1b) sets the initial model, which is optional. If another model is requested, FastFlowLM will automatically switch to it. The local server runs on port 52625 (default).

📰 In the News

08/11/2026 🎉 FLM is now part of ROCm (v1.0.0) — the repo has moved to AMD's open-source ROCm organization.
08/11/2026 🎉 FLM releases its first SmolVLA model (v1.0.0) — a Vision-Language-Action robotics policy running on the NPU. See the model card and benchmarks.
07/17/2026 🎉 FLM is now part of AMD news. Read our story — from a 2025 university project to AMD.
03/11/2026 🎉 FLM now supports Linux 🐧 ! To get started, check out the quick start guide or the Lemonade Server docs, and watch the short video for a quick walkthrough of FLM on Linux via Lemonade 🍋.
10/01/2025 🎉 FLM was integrated into AMD's Lemonade Server 🍋. Watch this short demo about using FLM in Lemonade.

🧠 Local AI on NPU

FLM makes it easy to run cutting-edge LLMs (and now VLMs) locally with:

⚡ Fast and low power
🧰 Simple CLI and API (REST and OpenAI API)
🔐 Fully private and offline

No model rewrites, no tuning — it just works.

✅ Highlights

Runs fully on AMD Ryzen™ AI NPU — no GPU or CPU load
Lightweight runtime (17 MB) — installs within 20 seconds, easy to integrate
Developer-first flow — like Ollama, but optimized for NPU
Support for long context windows — up to 256k tokens (e.g., Qwen3-4B-Thinking-2507)
No low-level tuning required — You focus on your app, we handle the rest

📄 License

All orchestration code and CLI tools are open-source under the MIT License.
These NPU-accelerated binary kernels are completely free for any use, including commercial use.
Please acknowledge FastFlowLM in your README/project page (or product) as follows:
```
Powered by [FastFlowLM](https://github.com/ROCm/FastFlowLM)
```

💬 Have feedback/issues or want early access to our new releases? Open an issue or Join our Discord community

🙏 Acknowledgements

Powered by the advanced AMD Ryzen™ AI NPU architecture
Inspired by the widely adopted llama.cpp and Ollama
Tokenization accelerated with MLC-ai/tokenizers-cpp
Chat formatting via Google/minja
Low-level kernels optimized using the powerful IRON+AIE-MLIR

🛠️ Building from Source

For developers who want to build FastFlowLM from source, we provide CMake presets for a convenient and consistent build experience.

Prerequisites

Git
CMake (version 3.22 or higher)
A C++20 compatible compiler (e.g., GCC, Clang, MSVC)
Ninja (recommended)

Build Instructions

More details on the exact procedure, with dependencies to be installed, for Linux can be found in linux-getting-started.md.

Clone the repository:

git clone --recursive https://github.com/ROCm/FastFlowLM.git
cd FastFlowLM/src

Configure CMake using presets:
- For Linux:
```
cmake --preset linux-default
```
  This will configure the build to install to /opt/fastflowlm.
- For Windows (in a developer command prompt):
```
cmake --preset windows-default
```
Build the project:
```
cmake --build build
```
Install the project (optional):
- For Linux:
```
sudo cmake --install build
```
- For Windows (with administrator privileges):
```
cmake --install build
```

ArduPilot/ardupilot

ArduPlane, ArduCopter, ArduRover, ArduSub source

ArduPilot Project

ArduPilot is the most advanced, full-featured, and reliable open source autopilot software available. It has been under development since 2010 by a diverse team of professional engineers, computer scientists, and community contributors. Our autopilot software is capable of controlling almost any vehicle system imaginable, from conventional airplanes, quad planes, multi-rotors, and helicopters to rovers, boats, balance bots, and even submarines. It is continually being expanded to provide support for new emerging vehicle types.

The ArduPilot project is made up of

ArduCopter: code, wiki
ArduPlane: code, wiki
Rover: code, wiki
ArduSub : code, wiki
Antenna Tracker : code, wiki

User Support & Discussion Forums

Support Forum: https://discuss.ardupilot.org/
Community Site: https://ardupilot.org

Developer Information

Github repository: https://github.com/ArduPilot/ardupilot
Main developer wiki: https://ardupilot.org/dev/
Developer discussion: https://discuss.ardupilot.org
Developer chat: https://discord.com/channels/ardupilot

Top Contributors

How To Get Involved

The ArduPilot project is open source and we encourage participation and code contributions: guidelines for contributors to the ardupilot codebase
We have an active group of Beta Testers to help us improve our code: release procedures
Desired Enhancements and Bugs can be posted to the issues list.
Help other users with log analysis in the support forums
Improve the wiki and chat with other wiki editors on Discord #documentation
Contact the developers on one of the communication channels

License

The ArduPilot project is licensed under the GNU General Public License, version 3.

Maintainers

ArduPilot is comprised of several parts, vehicles and boards. The list below contains the people that regularly contribute to the project and are responsible for reviewing patches on their specific area.

Andrew Tridgell:
- Vehicle: Plane, AntennaTracker
- Board: Pixhawk, Pixhawk2, PixRacer
Francisco Ferreira:
- Bug Master
Grant Morphett:
- Vehicle: Rover
Willian Galvani:
- Vehicle: Sub
- Board: Navigator
Michael du Breuil:
- Subsystem: Batteries
- Subsystem: GPS
- Subsystem: Scripting
Peter Barker:
- Subsystem: DataFlash, Tools
Randy Mackay:
- Vehicle: Copter, Rover, AntennaTracker
Siddharth Purohit:
- Subsystem: CAN, Compass
- Board: Cube*
Tom Pittenger:
- Vehicle: Plane
Bill Geyer:
- Vehicle: TradHeli
Emile Castelnuovo:
- Board: VRBrain
Georgii Staroselskii:
- Board: NavIO
Gustavo José de Sousa:
- Subsystem: Build system
Julien Beraud:
- Board: Bebop & Bebop 2
Leonard Hall:
- Subsystem: Copter attitude control and navigation
Matt Lawrence:
- Vehicle: 3DR Solo & Solo based vehicles
Matthias Badaire:
- Subsystem: FRSky
Mirko Denecke:
- Board: BBBmini, BeagleBone Blue, PocketPilot
Paul Riseborough:
- Subsystem: AP_NavEKF2
- Subsystem: AP_NavEKF3
Víctor Mayoral Vilches:
- Board: PXF, Erle-Brain 2, PXFmini
Amilcar Lucas:
- Subsystem: Marvelmind
Samuel Tabor:
- Subsystem: Soaring/Gliding
Henry Wurzburg:
- Subsystem: OSD
- Site: Wiki
Peter Hall:
- Vehicle: Tailsitters
- Vehicle: Sailboat
- Subsystem: Scripting
Andy Piper:
- Subsystem: Crossfire
- Subsystem: ESC
- Subsystem: OSD
- Subsystem: SmartAudio
Alessandro Apostoli:
- Subsystem: Telemetry
- Subsystem: OSD
Rishabh Singh:
- Subsystem: Avoidance/Proximity
David Bussenschutt:
- Subsystem: ESP32,AP_HAL_ESP32
Charles Villard:
- Subsystem: ESP32,AP_HAL_ESP32

ossrs/srs

SRS is a simple, high-performance, AI-driven real-time media server supporting RTMP, WebRTC, HLS, HTTP-FLV, HTTP-TS, SRT, MPEG-DASH, and GB28181, with codec support for H.264, H.265, AV1, VP9, AAC, Opus, and G.711.

SRS(Simple Realtime Server)

SRS/8.0 (Free) is a simple, high-performance, AI-driven real-time media server. It supports RTMP, WebRTC, HLS, HTTP-FLV, SRT, MPEG-DASH, and GB28181 on Linux and macOS across x86_64, ARMv7, AArch64, Apple M1, RISC-V, LoongArch, and MIPS architectures. Supported codecs include H.264, H.265, AV1, VP9, AAC, Opus, and G.711. See the complete list of features.

Note: For more details on the single-node architecture for SRS, please visit the following link.

SRS is licenced under MIT, and some third-party libraries are distributed under their licenses.

Usage

Please check the Getting Started guide in English or Chinese. We highly recommend using SRS with docker:

docker run --rm -it -p 1935:1935 -p 1985:1985 -p 8080:8080 \
    -p 8000:8000/udp -p 10080:10080/udp ossrs/srs:6

Tips: If you're in China, use this image registry.cn-hangzhou.aliyuncs.com/ossrs/srs:6 for faster speed.

Open http://localhost:8080/ to verify, and then stream using the following FFmpeg command:

ffmpeg -re -i ./doc/source.flv -c copy -f flv -y rtmp://localhost/live/livestream

Alternatively, stream by OBS using the following configuration:

Service: Custom
Server: rtmp://localhost/live
Stream Key: livestream

Play the following streams using media players:

To play an RTMP stream with URL rtmp://localhost/live/livestream on VLC player, open the player, go to Media > Open Network Stream, enter the URL and click Play.
You can play HTTP-FLV stream URL http://localhost:8080/live/livestream.flv on a webpage using the srs-player, an HTML5-based player.
Use srs-player for playing HLS stream with URL http://localhost:8080/live/livestream.m3u8.

If you'd like to use WebRTC, convert RTMP to WebRTC, or convert WebRTC to RTMP, please check out the wiki documentation in either English or Chinese.

To learn more about RTMP, HLS, HTTP-FLV, SRT, MPEG-DASH, WebRTC protocols, clustering, HTTP API, DVR, and transcoding, please check the documents in English or Chinese.

AI Agent

I recommend to use AI to understand and maintain your SRS, please follow the wiki AI Agent for details.

Contributing

The maintainers, and contributors are listed here. The maintainers who made significant contributions and maintained parts of SRS are listed below, ranked by the number of commits:

Winlin: Founder of the project, focusing on ST and Issues/PR. Responsible for architecture and maintenance.
XiaoZhihong: Concentrates on WebRTC/QUIC and SRT, with expertise in network QoS. Contributed to ARM on ST and was the original contributor for WebRTC.
ChenHaibo: Specializes in GB28181 and HTTP API, contributing to patches for FFmpeg with WHIP.
ZhangJunqin: Focused on H.265, Prometheus Exporter, and API module.
XiaLixin: Specializes in GB28181, with expertise in live streaming and WebRTC.
Jacob Su: Jacob Su has contributed to various modules of SRS.
ShiWei: Specializes in SRT and H.265, maintaining SRT and FLV patches for FFmpeg. An expert in codecs and FFmpeg.
ChenGuanghua: Focused on WebRTC/QoS and introduced the Asan toolchain to SRS.
LiPeng: Concentrates on WebRTC and contributes to memory management and smart pointers.
ZhaoWenjie: One of the earliest contributors, focusing on HDS. Has expertise in client technology.
WuPengqiang: Focused on H.265, initially contributed to the FFmpeg module in SRS for transcoding AAC with OPUS for WebRTC.

A huge THANK YOU goes out to:

All the contributors of SRS.
All the friends of SRS who gave big support.
Genes, Mabbott, and Michael Talyanksy for making and sharing State Threads.

We're really thankful to everyone in the community for helping us find bugs and improve the project. To stay in touch and keep helping our community, please check out this guide.

Of course, financial support is very important to us, so we can build the website and use AI tools to maintain the community. Please visit OpenCollective to become a backer or sponsor.

LICENSE

SRS is licenced under MIT, and some third-party libraries are distributed under their licenses.

Releases

2026-08-12, Release v7.0-d0, v7.0-d0, 7.0 dev0, v7.0.157, 313784 lines.
2026-08-12, Release v6.0-r1, v6.0-r1, 6.0 release1, v6.0.191, 171396 lines.
2025-12-03, Release v6.0-r0, v6.0-r0, 6.0 release0, v6.0.184, 170962 lines.
2025-11-03, Release v6.0-b3, v6.0-b3, 6.0 beta3, v6.0.183, 170957 lines.
2025-10-16, Release v6.0-b2, v6.0-b2, 6.0 beta2, v6.0.181, 170948 lines.
2025-09-15, Release v6.0-b1, v6.0-b1, 6.0 beta1, v6.0.177, 170611 lines.
2025-08-12, Release v6.0-b0, v6.0-b0, 6.0 beta0, v6.0.172, 170417 lines.
2025-05-03, Release v6.0-a2, v6.0-a2, 6.0 alpha2, v6.0.165, 169712 lines.
2024-09-01, Release v6.0-a1, v6.0-a1, 6.0 alpha1, v6.0.155, 169636 lines.
2024-07-27, Release v6.0-a0, v6.0-a0, 6.0 alpha0, v6.0.145, 169259 lines.
2024-07-04, Release v6.0-d6, v6.0-d6, 6.0 dev6, v6.0.134, 168904 lines.
2024-06-15, Release v6.0-d5, v6.0-d5, 6.0 dev5, v6.0.129, 168454 lines.
2024-02-15, Release v6.0-d4, v6.0-d4, 6.0 dev4, v6.0.113, 167695 lines.
2023-11-19, Release v6.0-d3, v6.0-d3, 6.0 dev3, v6.0.101, 167560 lines.
2023-09-28, Release v6.0-d2, v6.0-d2, 6.0 dev2, v6.0.85, 167509 lines.
2023-08-31, Release v6.0-d1, v6.0-d1, 6.0 dev1, v6.0.72, 167135 lines.
2023-07-09, Release v6.0-d0, v6.0-d0, 6.0 dev0, v6.0.59, 166739 lines.
2024-06-15, Release v5.0-r3, v5.0-r3, 5.0 release3, v5.0.213, 163585 lines.
2024-04-03, Release v5.0-r2, v5.0-r2, 5.0 release2, v5.0.210, 163515 lines.
2024-02-15, Release v5.0-r1, v5.0-r1, 5.0 release1, v5.0.208, 163441 lines.
2023-12-30, Release v5.0-r0, v5.0-r0, 5.0 release0, v5.0.205, 163363 lines.
2023-11-19, Release v5.0-b7, v5.0-b7, 5.0 beta7, v5.0.200, 163305 lines.
2023-10-25, Release v5.0-b6, v5.0-b6, 5.0 beta6, v5.0.195, 163303 lines.
2023-09-28, Release v5.0-b5, v5.0-b5, 5.0 beta5, v5.0.185, 163254 lines.
2023-08-31, Release v5.0-b4, v5.0-b4, 5.0 beta4, v5.0.176, 162919 lines.
2023-08-02, Release v5.0-b3, v5.0-b3, 5.0 beta3, v5.0.170, 162704 lines.
2023-07-09, Release v5.0-b2, v5.0-b2, 5.0 beta2, v5.0.166, 162520 lines.
2023-06-11, Release v5.0-b1, v5.0-b1, 5.0 beta1, v5.0.157, 162494 lines.
2023-05-14, Release v5.0-b0, v5.0-b0, 5.0 beta0, v5.0.155, 162600 lines.
2023-03-23, Release v5.0-a5, v5.0-a5, 5.0 alpha5, v5.0.148, 162066 lines.
2023-02-12, Release v5.0-a4, v5.0-a4, 5.0 alpha4, v5.0.141, 161897 lines.
2023-01-02, Release v5.0-a3, v5.0-a3, 5.0 alpha3, v5.0.128, 161327 lines.
2022-12-18, Release v5.0-a2, v5.0-a2, 5.0 alpha2, v5.0.112, 161233 lines.
2022-12-01, Release v5.0-a1, v5.0-a1, 5.0 alpha1, v5.0.100, 160817 lines.
2022-11-25, Release v5.0-a0, v5.0-a0, 5.0 alpha0, v5.0.98, 159813 lines.
2022-11-22, Release v4.0-r4, v4.0-r4, 4.0 release4, v4.0.268, 145482 lines.
2022-09-16, Release v4.0-r3, v4.0-r3, 4.0 release3, v4.0.265, 145328 lines.
2022-08-24, Release v4.0-r2, v4.0-r2, 4.0 release2, v4.0.257, 144890 lines.
2022-06-29, Release v4.0-r1, v4.0-r1, 4.0 release1, v4.0.253, 144680 lines.
2022-06-11, Release v4.0-r0, v4.0-r0, 4.0 release0, v4.0.252, 144680 lines.
2020-06-27, Release v3.0-r0, 3.0 release0, 3.0.141, 122674 lines.
2020-02-02, Release v3.0-b0, 3.0 beta0, 3.0.112, 121709 lines.
2019-10-04, Release v3.0-a0, 3.0 alpha0, 3.0.56, 107946 lines.
2017-03-03, Release v2.0-r0, 2.0 release0, 2.0.234, 86373 lines.
2016-08-06, Release v2.0-b0, 2.0 beta0, 2.0.210, 89704 lines.
2015-08-23, Release v2.0-a0, 2.0 alpha0, 2.0.185, 89022 lines.
2014-12-05, Release v1.0-r0, all bug fixed, 1.0.10, 59391 lines.
2014-10-09, Release v0.9.8, all bug fixed, 1.0.0, 59316 lines.
2014-04-07, Release v0.9.1, live streaming. 30000 lines.
2013-10-23, Release v0.1.0, rtmp. 8287 lines.
2013-10-17, Created.

Features

Please read FEATURES.

Changelog

Please read CHANGELOG.

Performance

Please read PERFORMANCE.

Architecture

Please read ARCHITECTURE.

Ports

Please read PORTS.

APIs

Please read APIS.

Mirrors

Please read MIRRORS.

Dockers

Please read DOCKERS.

electron/electron

Build cross-platform desktop apps with JavaScript, HTML, and CSS

📝 Available Translations: 🇨🇳 🇧🇷 🇪🇸 🇯🇵 🇷🇺 🇫🇷 🇺🇸 🇩🇪. View these docs in other languages on our Crowdin project.

The Electron framework lets you write cross-platform desktop applications using JavaScript, HTML and CSS. It is based on Node.js and Chromium and is used by the Visual Studio Code and many other apps.

Follow @electronjs on Twitter for important announcements.

This project adheres to the Contributor Covenant code of conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to coc@electronjs.org.

Installation

To install prebuilt Electron binaries, use npm. The preferred method is to install Electron as a development dependency in your app:

npm install electron --save-dev

For more installation options and troubleshooting tips, see installation. For info on how to manage Electron versions in your apps, see Electron versioning.

Platform support

Each Electron release provides binaries for macOS, Windows, and Linux.

macOS (Ventura and up): Electron provides 64-bit Intel and Apple Silicon / ARM binaries for macOS.
Windows (Windows 10 and up): Electron provides x64 (amd64) and arm64 binaries for Windows.
Linux: Electron provides x64 (amd64) and arm64 binaries for Linux. Electron supports major Linux distributions (e.g., Ubuntu, Fedora, Debian) in versions that are still supported by both Chromium and the distro maker (without requiring a paid subscription). The prebuilt binaries are built on Ubuntu.

In general, Electron tries to align with Chromium on platform support.

Electron Fiddle

Use Electron Fiddle to build, run, and package small Electron experiments, to see code examples for all of Electron's APIs, and to try out different versions of Electron. It's designed to make the start of your journey with Electron easier.

Resources for learning Electron

electronjs.org/docs - All of Electron's documentation
electron/fiddle - A tool to build, run, and package small Electron experiments
electronjs.org/community#boilerplates - Sample starter apps created by the community

Programmatic usage

Most people use Electron from the command line, but if you require electron inside your Node app (not your Electron app) it will return the file path to the binary. Use this to spawn Electron from Node scripts:

const electron = require('electron')
const proc = require('node:child_process')

// will print something similar to /Users/maf/.../Electron
console.log(electron)

// spawn Electron
const child = proc.spawn(electron)

Mirrors

China

See the Advanced Installation Instructions to learn how to use a custom mirror.

Documentation translations

We crowdsource translations for our documentation via Crowdin. We currently accept translations for Chinese (Simplified), French, German, Japanese, Portuguese, Russian, and Spanish.

Contributing

If you are interested in reporting/fixing issues and contributing directly to the code base, please see CONTRIBUTING.md for more information on what we're looking for and how to get started.

Community

Info on reporting bugs, getting help, finding third-party tools and sample apps, and more can be found on the Community page.

License

MIT

When using Electron logos, make sure to follow OpenJS Foundation Trademark Policy.

mnh-jansson/open-battery-information

Open Battery Information

This project aims to provide tools and information about various batteries in order to aid repair. It is very common for manufacturers to lock the BMS when a fault is detected to protect the device and the user. Very important feature! So when is it a problem? Well, there is always a chance for false triggering of this protection, or the fault could have been temporary or even repaired. In this case it would be wasteful to throw out a perfectly good BMS just because its software says it is faulty.

This is the problem we would like to solve!

Important

The desktop application is deprecated.

The old desktop application (Python/Tkinter) has been removed from this repository and is no longer maintained. The latest version of the Python application can be found in the v0.2.3 tag. It has been replaced by a new web-based UI that runs in your browser:

Web UI / OBI-1 tool: https://github.com/OpenBatteryInformation/openbatteryinformation.github.io
Live site: https://openbatteryinformation.github.io/

The web UI connects to the Arduino over Web Serial and can even flash the firmware directly from your browser, so no Python installation is needed anymore.

This repository now only hosts the Arduino firmware (see ArduinoOBI/).

Contact information

For any questions, please e-mail: openbatteryinformation@gmail.com

Support

I have spent alot of time on this project and now releasing all this information to the public in hope that it will help other people save batteries and money. If you would like to show some appreciation for my work, please consider supporting me by buying me a coffee or sponsor me on Github!

Instructions

Step 1: Set Up ArduinoOBI

Navigate to the ArduinoOBI folder in the project directory.
Follow the instructions in its README.md. This section will guide you through building and flashing the firmware to your Arduino Uno (or ESP32-C3), ensuring everything is set up correctly.

Step 2: Flash the Firmware

You have two options for flashing the firmware onto your board:

Option 1: Flash from the Browser (Recommended)

No toolchain required. Open the firmware uploader in Chrome or Edge, connect your board via USB and flash the firmware directly from your browser.

Option 2: Build and Flash with PlatformIO

Install VS Code and the PlatformIO extension.
Open the ArduinoOBI folder as a project in VS Code.
Build and upload the firmware for your board following the steps in ArduinoOBI/README.md.

Step 3: Use the OBI-1 Tool

Once the firmware is running on your board:

Open the OBI-1 web tool in Chrome or Edge.
Connect to the serial port of your Arduino (or ESP32-C3).
Insert a battery and press Read battery to read model, cell voltages, temperatures, charge count and status.

Prebuilt firmware binaries (uno.hex / esp32.bin) are also attached to the Releases page of this repository.

microsoft/onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

ONNX Runtime is a cross-platform inference and training machine-learning accelerator.

ONNX Runtime inference can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries such as scikit-learn, LightGBM, XGBoost, etc. ONNX Runtime is compatible with different hardware, drivers, and operating systems, and provides optimal performance by leveraging hardware accelerators where applicable alongside graph optimizations and transforms. Learn more →

ONNX Runtime training can accelerate the model training time on multi-node NVIDIA GPUs for transformer models with a one-line addition for existing PyTorch training scripts. Learn more →

Get Started & Resources

General Information: onnxruntime.ai
Usage documentation and tutorials: onnxruntime.ai/docs
YouTube video tutorials: youtube.com/@ONNXRuntime
Upcoming Release Roadmap
Companion sample repositories:
- ONNX Runtime Inferencing: microsoft/onnxruntime-inference-examples
- ONNX Runtime Training: microsoft/onnxruntime-training-examples
Plugin EP repositories:
- ONNX Runtime QNN Plugin EP: onnxruntime/onnxruntime-qnn

Releases

The current release and past releases can be found here: https://github.com/microsoft/onnxruntime/releases.

For details on the upcoming release, including release dates, announcements, features, and guidance on submitting feature requests, please visit the release roadmap: https://onnxruntime.ai/roadmap.

Data/Telemetry

This project may collect usage data and send it to Microsoft to help improve our products and services. See the privacy statement for more details.

Contributions and Feedback

We welcome contributions! Please see the contribution guidelines.

For feature requests or bug reports, please file a GitHub Issue.

For general discussion or questions, please use GitHub Discussions.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

License

This project is licensed under the MIT License.

GitHub C++ Weekly Trending Repositories

LadybirdBrowser/ladybird

Ladybird

Features

How do I build and run this?

How do I read the documentation?

Get in touch and participate!

License

microsoft/intelligent-terminal

Welcome to the Intelligent Terminal repo

What is Intelligent Terminal?

Installing and running Intelligent Terminal

Microsoft Store (recommended)

WinGet

Downloads

Get Started

Keyboard Shortcuts

Configuration

Features

Agent Status Bar

Agent Pane

Slash Commands

Local Models (BYOM)

Agent Management

Error Detection

Command Palette

Data & Privacy

What data flows through Terminal

Where your data goes depends on your agent CLI

Controls

Data Collection

Building the Code

Feedback

Collecting Logs

Contributing

Code of Conduct

Security

Trademarks

ggml-org/llama.cpp

llama.cpp

Quick start

Description

Supported backends

Documentation

Tools

Development

Contributing

Acknowledgements

cactus-compute/cactus

Cactus

Quick Demo (Mac)

Cactus Engine

Cactus Graph

Inference Speed

Output Quality

Supported Models

Needle

Learn More

Bindings

Using this repo

Citation

microsoft/BitNet

bitnet.cpp

📰 News

Overview

Model Releases

1. BitNet-b1.58-2B-4T - 1-bit Large Language Model

2. BitNet-embedding-0.6B - 1-bit Embedding Model

3. BitNet-embedding-270M - Lightweight 1-bit Embedding Model

Supported Models

Installation

Requirements

Build from source

Usage

Basic usage

Demo

Benchmark

Convert from .safetensors Checkpoints

Acknowledgements

FAQ (Frequently Asked Questions)📌

Convert from `.safetensors` Checkpoints

👷 `workerd`, Cloudflare's JavaScript/Wasm Runtime

WARNING: `workerd` is not a hardened sandbox

Building `workerd`

Configuring `workerd`

Running `workerd`

Local Worker development with `wrangler`