You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4f9ca1f
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: apps/desktop/native/computer-use/README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ Compile each architecture with `swiftc -parse-as-library -O -target arm64-apple-
8
8
9
9
Persistent newline-delimited JSON over stdin/stdout: `{id,method,params}` returns `{id,result}` or `{id,error:{code,message,dispatchState?}}`. The producer contract is `worker/packages/contracts/src/computer-use.ts` in mothership, copied into the desktop bridge by contract sync. `request_permission` is a local onboarding method, never a model tool; params are `{permission:"accessibility"|"screenCapture"|"reminders"|"calendars"|"contacts"}`. Status/preflight never triggers prompts; it reports `reminders`, `calendars` and `contacts` as `granted`, `denied` or `not_determined`.
10
10
11
-
`app_command` takes `{command:{name,...}}` and runs one allowlisted command through EventKit or Contacts, never Apple Events or scripts. Reads: `list_reminder_lists`, `list_calendars` and `list_contact_groups` return `{kind:"command",items:[{id,title,allowsModifications}]}`. Creates return `{kind:"command",created:{id,title,list}}`: `create_reminder` (`title`, `notes`, `due` as `YYYY-MM-DD` or ISO 8601 with an offset, `listId`, `priority` 0/1/5/9), `create_event` (`title`, `start`/`end` as ISO 8601 with offsets or `YYYY-MM-DD` with `allDay`, at most 31 days, `location`, `notes`, `calendarId`, `alertMinutesBefore`), `create_contact_group` (`title`; an existing group with that name is returned instead of a duplicate) and `create_contact` (`givenName`, `familyName`, `organization`, `jobTitle`, up to 5 `emails` and `phones`, `groupId`). Without the matching access a command fails with `reminders_permission_required`, `calendars_permission_required` or `contacts_permission_required` before touching anything. There is no update, delete or search. The desktop service confirms each create with the user before sending it.
11
+
`app_command` takes `{command:{name,...}}` and runs one allowlisted command through EventKit or Contacts, never Apple Events or scripts. Reads: `list_reminder_lists`, `list_calendars` and `list_contact_groups` return `{kind:"command",items:[{id,title,allowsModifications}]}`. Creates return `{kind:"command",created:{id,title,list}}`: `create_reminder` (`title`, `notes`, `due` as `YYYY-MM-DD` or an ISO 8601 date-time, `listId`, `priority` 0/1/5/9), `create_event` (`title`, `start`/`end` as ISO 8601 date-times or `YYYY-MM-DD` with `allDay`, at most 31 days, `location`, `notes`, `calendarId`, `alertMinutesBefore`), `create_contact_group` (`title`; an existing group with that name is returned instead of a duplicate) and `create_contact` (`givenName`, `familyName`, `organization`, `jobTitle`, up to 5 `emails` and `phones`, `groupId`). A date-time without an offset is this Mac's local time, as typing it into the app would be. Without the matching access a command fails with `reminders_permission_required`, `calendars_permission_required` or `contacts_permission_required` before touching anything. There is no update, delete or search. The desktop service confirms each create with the user before sending it.
12
12
13
13
The helper observes up to 2000 accessibility nodes (depth 64, queue 4000, traversal deadline 8 seconds, each AX call timeout 250ms). Selected-window and focused-element ancestry are prioritized. The menu bar and menus are read in place, so an open menu's items and their submenus come first; items of closed menus are read last, capped at 80 menu nodes. Electron accessibility is enabled using its documented `AXManualAccessibility` attribute. The helper observes 100 windows and 1000 apps. Requests are bounded to 128 KiB during ingestion. Text/value input is capped at 32000 UTF-16 units. State references are per-process, per-app, expire after 60 seconds, and are consumed by every action that reaches the app; a request that fails before sending anything reports `dispatchState: "not_started"` and keeps them valid. An Accessibility failure counts as sent unless Accessibility rejected the request before the app saw it (unsupported attribute or action, illegal argument). Restarts invalidate all references. Screen captures exclude window shadows, preserve the selected window’s coordinate geometry, and are capped to 1600 pixels on the longest side and 8 MiB encoded. Complete JSON replies are capped at 16 MiB: oversized observations omit screenshots first, then truncate accessibility nodes, while preserving action outcomes. Screenshots are optional and failure is reported in `screenshotError` without discarding accessibility state. macOS 26 uses the screenshot-specific capture API; the macOS 14/15 API path includes only the selected window in a display filter and crops its exact frame. That older path requires the window to fit within one display and reports `capture_geometry_unavailable` otherwise.
0 commit comments