Hand Mouse Control

Hand-based pointer controller.

Yes, there's a Python library for this too. There's a Python library for everything.

Hand Mouse Control turns your webcam into a mouse. It tracks your hand with MediaPipe, maps your index finger to the cursor, and reads pinch gestures as clicks, drags, scrolling, and zoom — all wrapped in a drop-in QWidget you can embed in any Qt app.

Features

• Cursor control — move your index finger, move the pointer.
• Pinch gestures — thumb + finger combos trigger different actions.
• Click & drag — quick pinch to click, hold to drag.
• Scroll & zoom — pinch and move vertically.
• Live preview — optional on-screen landmark overlay and gesture label.
• Qt widget — just another widget, embed it wherever you like.

Installation

pip install git+https://github.com/davidleonstr/handmouse.git

Or clone and install locally:

git clone https://github.com/davidleonstr/handmouse.git
cd handmouse
pip install .

The MediaPipe hand-landmarker model is downloaded automatically on first run.

Usage

from qtpy.QtWidgets import QApplication
from handmouse import HandMouseWidget

app = QApplication([])

widget = HandMouseWidget()
widget.resize(700, 520)
widget.showLabel = True   # show gesture status overlay
widget.setWorking(True)   # start controlling the mouse
widget.show()

app.exec_()

See examples/playground.py for a runnable example.

Gestures

Gesture	Action
Thumb + Index	Click (hold to drag)
Thumb + Middle	Right click
Thumb + Ring + move	Scroll
Thumb + Pinky + move	Zoom (Ctrl + scroll)
Index finger	Moves the pointer

Configuration

HandMouseWidget accepts tuning parameters on init:

HandMouseWidget(
    marginX=0.25,              # horizontal capture margin
    marginY=0.25,              # vertical capture margin
    pinchThreshold=0.40,       # how close fingers must be to register a pinch
    minMovementThreshold=0.004,# ignore jitter below this delta
    holdToDragTimer=0.25,      # seconds of pinch before drag starts
    scrollSensibility=2200,    # scroll speed multiplier
    zoomSensibility=2200,      # zoom speed multiplier
    alphaSoftener=0.25,        # cursor movement smoothing factor
)

Requirements

• Python 3.11+
• A webcam
• QtPy, opencv-python, mediapipe, PyAutoGUI

Notes

• Only one hand is tracked for cursor/gesture control at a time.
• PyAutoGUI's fail-safe is enabled — move the cursor to a screen corner to abort.
• Works on any platform supported by PyAutoGUI and OpenCV's camera capture.

License

MIT — see LICENSE for details.