Windows Automation Skill

Automate Windows desktop interactions using PyAutoGUI

Overview

This skill provides comprehensive Windows desktop automation capabilities. Use it when you need to: - Simulate mouse clicks and keyboard input - Automate GUI interactions in non-web applications - Capture screenshots - Manage clipboard content - Launch applications - Run system commands

Setup

The required packages are already installed: - pyautogui - Core automation engine - pywin32 - Windows API access - pyperclip - Clipboard operations

If needed, install manually:

pip install pyautogui pywin32 pyperclip

Tools

🖱️ Mouse Control

`windows_click`

Click at specified screen coordinates.

windows_click(x=100, y=200, button="left", clicks=1)

Parameters: | Param | Type | Default | Description | |-------|------|---------|-------------| | x | float | required | X coordinate | | y | float | required | Y coordinate | | button | str | "left" | "left", "right", "middle" | | clicks | int | 1 | Number of clicks |

`windows_double_click`

Double click at position.

windows_double_click(x=500, y=300)

`windows_right_click`

Right click at position.

windows_right_click(x=100, y=200)

`windows_move`

Move mouse to position.

windows_move(x=800, y=600, duration=0.5)

Parameters: | Param | Type | Default | Description | |-------|------|---------|-------------| | x | float | required | X coordinate | | y | float | required | Y coordinate | | duration | float | 0.2 | Movement duration in seconds |

⌨️ Keyboard Control

`windows_type`

Type text using keyboard.

windows_type(text="Hello World!", interval=0.05)

Parameters: | Param | Type | Default | Description | |-------|------|---------|-------------| | text | str | required | Text to type | | interval | float | 0.05 | Delay between keystrokes |

`windows_press`

Press a single key.

windows_press(key="enter")

Supported keys: - Function keys: f1 - f24 - Special keys: enter, esc, tab, space, backspace, delete - Arrow keys: up, down, left, right - Modifiers: ctrl, alt, shift, win

`windows_hotkey`

Press keyboard shortcut combination.

windows_hotkey("ctrl", "c")   # Copy
windows_hotkey("ctrl", "v")   # Paste
windows_hotkey("alt", "f4")   # Close window
windows_hotkey("win", "r")    # Open Run dialog

📸 Screen Operations

`windows_screenshot`

Take a screenshot.

windows_screenshot(filename="screenshot.png")

Parameters: | Param | Type | Default | Description | |-------|------|---------|-------------| | filename | str | "screenshot.png" | Filename to save |

ℹ️ Information

`windows_cursor_position`

Get current cursor position.

windows_cursor_position()
# Returns: {"x": 500, "y": 300}

`windows_screen_size`

Get screen resolution.

windows_screen_size()
# Returns: {"width": 1920, "height": 1080}

`windows_system_info`

Get basic system information.

windows_system_info()
# Returns: {
#   "platform": "Windows",
#   "version": "10.0.22631",
#   "screen_size": {"width": 1920, "height": 1080},
#   "cursor_position": {"x": 500, "y": 300}
# }

🪟 Applications

`windows_launch_app`

Launch a Windows application.

windows_launch_app(name="notepad")
windows_launch_app(name="calc")
windows_launch_app(name="excel")
windows_launch_app(name="C:\Program Files\App\app.exe")

💻 System

`windows_command`

Run a PowerShell or cmd command.

windows_command(command="ipconfig")
windows_command(command="Get-Process | Select-Object -First 5")
windows_command(command="dir C:\")

📋 Clipboard

`windows_clipboard_read`

Read clipboard content.

windows_clipboard_read()
# Returns: "clipboard content"

`windows_clipboard_write`

Write text to clipboard.

windows_clipboard_write(text="Hello from OpenClaw!")

Usage Examples

Example 1: Open Notepad and Type Text

User: Open notepad and type hello
→ windows_launch_app with name="notepad"
→ windows_type with text="Hello World!"

Example 2: Fill a Form

User: Click on the name field and enter John
→ windows_click x=300 y=200
→ windows_type text="John"
→ windows_press key="tab"
→ windows_type text="[email protected]"

Example 3: Screenshot and Save

User: Take a screenshot
→ windows_screenshot filename="screen_20260101.png"

Example 4: Copy File Path

User: Copy C:Documentsfile.txt to clipboard
→ windows_clipboard_write text="C:\Documents\file.txt"

Example 5: Get System Info

User: What's my screen resolution?
→ windows_screen_size

Safety Notes

⚠️ The mouse failsafe is disabled for smoother operation
📍 Screen coordinates start at (0, 0) in top-left corner
🖥️ Multi-monitor setups may have different coordinate systems
🔒 Some applications may block simulated input (games, secure apps)

Dependencies

Package	Purpose
pyautogui	Core automation
pywin32	Windows API
pyperclip	Clipboard

Built for OpenClaw 🦞

windows-automation

Installation