# Vibe Mouse: AI Coding Shortcut Controller with Multi-Device Adapter Support

> Vibe Mouse is an AGPL-3.0 Python project that maps mouse buttons, voice commands, gamepads, and other input devices to AI IDE shortcuts, including a voice-to-LLM bridge that sends speech directly to DeepSeek, GLM, or Kimi. It is designed for developers who want to reduce keyboard friction when working with AI coding tools like Cursor, Trae, or Windsurf.

**CSCB/vibe-mouse** — Core open-source Vibecoding interactive mouse project, belonging to a self-developed open-source system including the nation's first open-source robot actuator and stepless mobile intelligent monitoring solution.

- Repository: https://github.com/CSCB/vibe-mouse
- Stars: 1,179 · Forks: 0
- Language: Python
- License: AGPL-3.0
- Published: 2026-09-17 · Updated: 2026-09-17 · Language: en
- Canonical page: https://hysenlabs.com/projects/cscb-vibe-mouse

## What Vibe Mouse Is and Who Uses It

AI coding tools like Cursor, Trae, and Windsurf are controlled primarily through keyboard shortcuts: Ctrl+K to invoke inline code generation, Ctrl+L to toggle the chat panel, Ctrl+Y or Ctrl+Enter to accept a diff. Developers who work with these tools repeatedly switch between typing code and pressing these shortcuts, a pattern that interrupts flow.

Vibe Mouse replaces that keyboard switching with configurable physical inputs. The default configuration maps the middle mouse button to accept AI code (Accept Diff), side button 1 to invoke inline code generation (Inline Edit), and side button 2 to toggle the AI chat panel. Any of these can be reassigned through the visual config tool or config.json.

The project is labeled Core open-source Vibecoding interactive mouse project in the README, and it describes a broader system with three independently operable modules: the Vibe Mouse interaction layer, an open-source robot actuator, and an intelligent monitoring system. Each module is open-sourced separately and can be combined or used alone.

The README describes the target audience as individual development, laboratory projects, smart device retrofitting, and robotics scenario development.

## Core Architecture: Adapters, Events, and the Executor

The internal data flow follows a consistent pipeline regardless of input type:

```
Device Event → Adapter → Event (unified) → DeviceManager → Executor (keyboard shortcut)
```

Each input type has a dedicated adapter class. The mouse adapter uses pynput. The keyboard adapter handles multimedia keys. A gamepad adapter supports Xbox, PlayStation, and Switch controllers via pygame. A Bluetooth adapter uses bleak for BLE and pybluez for Classic Bluetooth. An IR adapter handles TV and AC remotes via pyserial or broadlink. An HID adapter covers Stream Deck and barcode scanners via hidapi. A network adapter handles MQTT, WebSocket, and HTTP inputs from IoT devices. A voice adapter handles wake words and voice commands.

All adapters emit a unified Event object, defined in core/event.py, containing DeviceType, InputType, and the event payload. The DeviceManager routes these unified events to the Executor, which maps them to keyboard shortcut sequences and optionally triggers feedback.

This pluggable design means adding a new input device requires writing a new adapter class extending the base.py abstract class, registering it in device_manager.py, and adding the mapping to config.json. No other files need changes.

## Installation and Getting Started

The core dependencies install through pip. The requirements.txt lists the mandatory packages:

```bash
pip install pynput==1.7.6 pystray==0.19.5 Pillow==10.2.0 pyinstaller==6.4.0
```

Additional adapter dependencies are listed as comments with # prefix in requirements.txt, meaning they are optional and installed only when the corresponding adapter is needed. For example, pygame is needed for the gamepad adapter, bleak for BLE, and SpeechRecognition plus pyaudio for voice input.

The program runs in the system tray. To start it:

```bash
python core/main.py
```

For the desktop GUI panel showing current tool, active Skill, switch history, and auto-detection toggle:

```bash
python core/main.py --gui
```

A build.py script generates a standalone .exe or .app for distribution:

```bash
python build.py
```

The visual config tool does not require a backend: open config-tool.html in any browser to configure tools, devices, mappings, feedback, Token Plan, and Skills visually.

Cross-platform key adaptation is automatic: on macOS, Cmd is substituted for Ctrl in shortcuts without any manual configuration change.

## Voice-to-LLM Bridge and Prompt Refine

The voice-to-LLM bridge, implemented in core/voice_llm.py, captures speech recognition output and sends it to an LLM API. The README describes support for Huawei Cloud Token Plan and MaaS providers including GLM, DeepSeek, and Kimi. Configuration requires entering the relevant API Key in config.json or config-tool.html.

The optional Prompt Refine layer, also in voice_llm.py, runs before the LLM call: it converts colloquial speech to a precise, structured prompt, corrects homophones and accents, and expands vague requests into actionable instructions. The README gives the example of saying "generate a quicksort function" and receiving AI-generated code, with Prompt Refine improving the quality of the prompt sent to the LLM.

The voice adapter in core/adapters/voice_adapter.py handles wake word detection and voice command processing. The README lists pocketsphinx as an optional offline recognition engine (requiring an access key for pvporcupine wake word mode), with SpeechRecognition and pyaudio as the online path.

## The Skill System and Auto Window Detection

The Skill system, implemented in core/skill_engine.py, defines a Skill as a complete work mode. A Skill can override shortcuts, feedback configuration, device activation, and the system prompt that goes to the LLM, all in a single YAML or JSON file. Binding a mouse button or key to skill:name activates that Skill; voice keywords can also auto-match a Skill by trigger words.

The README lists four built-in templates: quick-code, code-review, presentation, and explain-code. Custom Skills can be stored in the skills/ directory.

Auto Window Detection, implemented in core/window_detector.py, is a Beta feature. It detects which AI IDE is currently in the foreground by process name and automatically switches shortcuts. The supported IDEs include Trae, Cursor, Windsurf, VS Code, DevEco, and CodeArts. Without this feature, switching from Cursor to Windsurf requires a manual shortcut profile change; with it, the swap is automatic when the foreground window changes.

Tool Switch History in core/tool_switcher.py saves a full state snapshot (active Skill, shortcut overrides, feedback config) with every tool switch. Switching back to a previous tool restores its exact context.

## Supported IDE Shortcuts and AGPL Licensing

The README documents the supported shortcut mappings for each AI IDE. Trae, Cursor, Windsurf, Copilot, Claude, and Augment all have specific mappings for Inline Edit, Toggle Chat, Accept, Reject, and Voice Input, where available. The configuration is in core/config.py and can be overridden through config.json or the visual config tool.

The AGPL-3.0 license has a specific implication the README calls out: all derivative works must remain open-source in compliance with the specified license. This covers not only forks and modifications distributed as software but also any version used to provide a network service, since AGPL extends the copyleft requirement to network use. Teams building commercial products on top of Vibe Mouse need to comply with AGPL or seek a separate license.

The last push to the repository was on 2026-07-24. The project has no published GitHub releases. AutoHotKey is an alternative for Windows-only shortcut automation without the LLM integration; it handles keyboard and mouse remapping but has no voice recognition layer and no cross-platform support.

## Conclusion

Vibe Mouse is useful for developers who spend significant time switching between triggering AI inline edits, accepting diffs, and toggling chat panels, and who want to offload those actions to physical buttons or voice. The auto window detection and Skill system are both labeled Beta, so production reliance on them carries some risk. Before deploying, verify that your AI IDE is in the supported tools list (trae, cursor, windsurf, copilot, among others) and that the AGPL-3.0 license is compatible with how you intend to distribute any derivative work, since AGPL requires all modifications to remain open-source even when used to provide a network service.

## FAQ

### Which AI IDEs does Vibe Mouse support?

The README lists Trae, Cursor, Windsurf, Copilot, Claude, DevEco, CodeArts, and Augment as supported tools with specific shortcut mappings. The auto window detection Beta feature detects Trae, Cursor, Windsurf, VS Code, DevEco, and CodeArts by foreground process name and switches shortcuts automatically.

### What does the AGPL-3.0 license mean for Vibe Mouse users?

AGPL-3.0 requires all derivative works to remain open-source, including versions used to provide a network service. The README states this requirement explicitly. Teams integrating Vibe Mouse into commercial products or services need to either comply with AGPL or obtain a different license.

### Can Vibe Mouse work without voice input or a special mouse?

Yes. Voice input and all adapter types except the core mouse adapter are optional dependencies listed as comments in requirements.txt. The default configuration works with any three-button mouse using the standard buttons; the Middle button, Side button 1, and Side button 2 mappings cover the most common AI IDE actions. Gamepad, Bluetooth, IR, HID, and network adapters are independent optional additions.

## Sources

- [CSCB/vibe-mouse on GitHub](https://github.com/CSCB/vibe-mouse)
- [Issues](https://github.com/CSCB/vibe-mouse/issues)
- [License: AGPL-3.0](https://github.com/CSCB/vibe-mouse/blob/main/LICENSE)
- [README](https://github.com/CSCB/vibe-mouse/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/cscb-vibe-mouse