OpenWorker Architecture — Andrew Ng's Open-Source AI Coworker

On July 22, 2026, Andrew Ng dropped OpenWorker — an open-source desktop AI coworker that runs on your machine, works across 25+ app integrations, and doesn't lock you into any single model provider. Within 24 hours it crossed 5,000 GitHub stars. The pitch is straightforward: "AI that gets your everyday tasks done" — polished documents, Slack replies with the numbers, updated calendars, triaged inboxes. Finished work, not just chat.

But the architecture underneath is what makes it interesting. OpenWorker is not a thin wrapper around an LLM — it's a carefully layered system with an approval-first security model, model independence baked into the foundation, scheduled automation support, and MCP extensibility. Let's walk through each layer.

Three-Layer Architecture

OpenWorker's architecture is partitioned into three clean layers, each with a distinct responsibility and tech stack:

Layer 1: Desktop Shell (Electron GUI)

The top layer is the native desktop experience — a React UI wrapped in a Tauri shell (not Electron, despite initial expectations — Tauri gives it a smaller binary footprint and tighter OS integration). The desktop shell handles:

• Session management — multi-turn conversations with full transcripts
• Approval inbox — queued actions that need user sign-off before execution
• Automation dashboard — scheduled runs, their outputs, and history
• Model & connector configuration — bring your own keys, swap providers, enable/disable integrations
• Speech-to-text input via a Rust sidecar (stt/ directory)

The desktop app can also run as a standalone server with a browser UI via Vite in development mode, making it accessible for headless deployments and CI integration.

Layer 2: Agent Engine (Python / aisuite)

The middle layer is the brain — a Python-based agent server built on aisuite, Andrew Ng's open-source multi-provider model interface. This layer handles:

• Task decomposition — breaking high-level requests ("prepare a customer brief") into executable steps
• Tool orchestration — selecting and sequencing API calls across connected services
• Approval gating — intercepting consequential actions and routing them to the user for sign-off
• Memory & context — maintaining conversation state and project context across sessions
• Automation scheduling — cron-style triggers for recurring tasks
• MCP client — connecting to arbitrary tools that speak the Model Context Protocol

The engine communicates with the desktop shell over a local HTTP server (default port 8765), secured with a per-launch token. In the desktop app, this token stays in memory — it's never written to disk.

Layer 3: Tools & Models (25+ Connectors, Any Model)

The bottom layer is the integration surface — everything the agent can read from and write to:

• 25+ application connectors — GitHub, Slack, Jira, Notion, Linear, HubSpot, Outlook, monday.com, Gmail, Google Calendar, and more
• Model providers — OpenAI, Anthropic, Google Gemini, DeepSeek, Qwen, Mistral, Grok (xAI), Kimi (Moonshot), GLM (Z.ai), MiniMax, Inkling (Thinking Machines), plus Together and Fireworks for open-weight models
• Local models via Ollama — fully offline operation with no data leaving your machine
• MCP connectors — any tool that speaks the Model Context Protocol plugs in directly
• Filesystem & terminal — read/write local files, execute shell commands (both approval-gated)

Approval-First Security Model

The defining architectural decision in OpenWorker is the approval-first pattern. Rather than asking the LLM to decide what's safe — which is unreliable at best — OpenWorker hard-gates consequential actions at the infrastructure level:

• Every write, send, and shell command must be approved. The agent can propose sending a Slack message, updating a calendar, or running a terminal command, but it cannot execute without explicit user sign-off.
• Unattended runs park their asks in an inbox. When an automation fires (e.g., a scheduled morning brief), any consequential action it wants to take is queued for review rather than executed blindly. You approve or redirect when you're back at the keyboard.
• Read-only operations are unrestricted. The agent can read from your calendar, check Jira status, or scan GitHub issues without approval — it only gates destructive or externally visible actions.
• Approval awareness is not model-dependent. The gating happens in the agent engine layer, not in the LLM prompt. Even if the model confidently asserts it should send an email, the engine blocks it until you say yes.

This is the right design. LLM safety guardrails are prompt-level and trivially circumvented. Architectural gating — intercepting the action at the tool execution layer — is the only reliable approach for an agent that has access to your email, calendar, and code repositories.

Model Independence: No Vendor Lock-In

OpenWorker is built on aisuite, Andrew Ng's open-source library that provides a unified interface across model providers. The result is genuine model independence:

• Bring your own key. Paste an API key for OpenAI, Anthropic, Google, or any supported provider — switch anytime without changing your workflows.
• Run fully local. Point OpenWorker at a local Ollama instance and nothing leaves your machine. Your data goes through the model you choose, on hardware you control.
• Verified model list. The team maintains a curated list of models confirmed to work for tool-calling. You can use any model string, but verified models have been battle-tested.
• Eleven cloud providers, one interface. OpenAI, Anthropic, Google Gemini, DeepSeek, Qwen, Mistral, Grok (xAI), Kimi (Moonshot), GLM (Z.ai), MiniMax, Inkling (Thinking Machines), plus Together and Fireworks for open-weight models.

This is a sharp contrast to vertically integrated agents that bundle the model with the product. OpenWorker decouples the agent runtime from the model runtime — you own the former, and you choose who provides the latter.

Automations: Scheduled Intelligence

Beyond interactive sessions, OpenWorker supports scheduled automations — recurring agent runs that execute on a cron-like schedule:

• Morning briefs. A daily digest of calendar events, priority emails, Jira status, and Slack mentions — compiled and waiting when you start your day.
• Weekly reports. Cross-tool summaries pulling data from GitHub, Jira, Linear, and Slack into a single document.
• Channel watchers. Standing monitors over Slack channels or GitHub repos that surface relevant activity on a schedule.
• Full transcripts. Every automation run lands in the app with a complete transcript — you can see exactly what the agent considered, proposed, and (if approved) executed.

The key design constraint: automations follow the same approval rules as interactive sessions. If a morning brief wants to send a Slack message on your behalf, that action goes into the approval inbox — it doesn't fire automatically. This respects the approval-first pattern even when you're not actively driving the session.

MCP Support: Extensible by Design

OpenWorker supports the Model Context Protocol (MCP), the emerging standard for connecting AI agents to external tools. Any tool that speaks MCP — databases, internal APIs, custom services — plugs into OpenWorker with per-tool enable/disable control.

This is critical for enterprise and power-user scenarios. The 25 built-in connectors cover common SaaS tools, but every organization has internal systems that no off-the-shelf agent will know about. MCP provides a standardized interface so those systems become first-class citizens in the agent's tool inventory — without writing custom connector code for every integration.

Combined with the approval-first model, MCP support means you can safely expose powerful internal tools (database queries, deployment triggers, financial system access) to the agent while maintaining architectural gating on every consequential action.

Privacy & Data Residency

OpenWorker is local-first by default. Everything lives on your machine:

• The agent loop runs locally
• Conversations are stored in the app's local secret store
• Connector tokens and model keys never leave your device
• The only cloud component is a small OAuth broker for connector authentication — and it's optional; you can use API keys instead and stay fully offline

The data flow is straightforward: your data leaves your machine only through the model provider you choose and the integrations you enable. If you run Ollama locally and use only filesystem tools, no data leaves your machine at all.

What This Means for the Agent Landscape

OpenWorker lands at an inflection point. Desktop AI agents are proliferating — GitHub Copilot, Cursor, Claude Code, and a dozen others — but most are either developer-only, vertically integrated, or cloud-dependent. OpenWorker's bet is that the winning architecture is:

• General-purpose — not just coding, but documents, calendars, email, project management
• Open-source — inspectable, forkable, auditable (MIT license)
• Model-independent — no lock-in, bring your own keys or run local
• Approval-first — architectural gating, not prompt-level guardrails
• Extensible — MCP for arbitrary tool integration beyond the 25 built-in connectors

Whether this bet pays off depends on execution — the open beta is rough around the edges, as the team acknowledges. But the architecture is sound, the principles are right, and the open-source model means the community can harden it faster than any single company could alone.

Relevance to Delta V

For sovereign AI deployments and agent architectures, OpenWorker validates several of our core design principles:

• Local-first by default. Keys and conversations stay on-device. The agent runtime is under your control. This aligns with our position that production AI systems must be auditable and self-hostable.
• Architectural approval gating. We've argued that prompt-level safety is insufficient for agents with real-world access. OpenWorker's approval-first pattern at the engine layer is exactly the right approach.
• Model independence via aisuite. Decoupling the agent runtime from the model runtime is the correct abstraction. We've been building on similar principles for our multi-agent systems.
• MCP for extensibility. Standardized tool interfaces are essential for agent interoperability. OpenWorker's MCP support pushes the ecosystem toward a common protocol.

We'll be watching OpenWorker's development closely — and likely contributing. The agent landscape needs more open-source, local-first, approval-gated options, and fewer walled gardens.