Yu-tao-Li/dsh-computer-use-win

Install

dsh plugin --profile web add dsh-computer-use-win

What it does

Lets the DSH agent "see" and operate the Windows desktop: locate UI controls, take screenshots, recognize text, click, type, and manage windows. In effect it gives the agent a pair of eyes and a pair of hands.

Features

• 23 desktop capabilities plus 1 tool-management entry point
• A PowerShell-based UIA (Windows accessibility interface) backend that locates controls by name instead of hard-coded coordinates
• Screenshots, OCR text recognition, keyboard and mouse input, window management
• Explicit control prompts; pressing Esc stops the current turn at any time
• 0.5.0 adds lazy startup: only 1 management tool loads at launch, with the rest loaded on demand to save context
• Bilingual Chinese and English settings interface

Good fit for

• The agent needs to operate legacy software that has no API
• Repetitive desktop clicking workflows you want to automate
• Reading information from interface screenshots (OCR)

Advice

• Try it in a virtual machine or non-critical environment first and confirm that click targeting is reliable
• Add human confirmation at key steps rather than running fully automatic
• If you use many tools, enable lazy startup so they load only when needed, saving considerable context

Cautions

• Being able to control keyboard and mouse amounts to handing over desktop control; the risk level is high
• Interface changes can break targeting, so operation flows need fault tolerance
• It uses a PowerShell backend, and some corporate environments block script execution