Simon.

What I'm building

Simon started as a single assistant with custom MCP tools. It has grown into a suite for managing AI workflows and projects, with multiple agents and persistent memory.

A project brings its conversations, tasks, files, and findings together. Project scope and state evolve as the work progresses, and local file storage keeps inputs and outputs attached to that project with automatic compaction and embedding. That gives both me and the agents something to return to: earlier results to build on, a record of what's happened, and tasks that still need attention.

Within each project, I can choose a team of agents and a lead. The lead breaks an objective into tasks for specialist agents, then reviews what they produce. This is the part I'm especially interested in exploring: how agents with different responsibilities can work together on something larger, and how this delegation of responsibility allows for more reliable outcomes versus relying on a single agent.

I can review a plan before it runs or set up scheduled work that returns to an ongoing objective, with limits on cadence, cycles, and model use. Classic chat and voice conversations are also supported, and save context to the persistent project storage automatically.

From a prompt to a printed vase

The workflow illustrated above is based on a run I actually completed with Simon and RobbinsHome. From an initial prompt, the system 3D modeled the vases, prepared the models for printing, sliced the print files, and sent them to the printer. The printing then completed automatically.

This is the kind of project that makes me excited to keep building: seeing an idea move through the agents' work and become a physical object, with Simon and the home system working together.

Under the hood

I'm building Simon around a local server and a browser workspace, with persistent storage for the work between sessions.

Backend

Python and FastAPI, with separate workers for background conversations and project agent teams.

Frontend

HTML, CSS, and JavaScript, with streaming text updates and WebRTC voice.

Storage

PostgreSQL and managed local file storage, with optional cloud connections for additional sources and destinations.

AI and tools

The OpenAI Python SDK and tool integrations, organized around agents, teams, and project workflows.

Future work

I want Simon to get better at projects that unfold over days or weeks. A big next step is letting work wait for a person, another agent, or an outside event, then pick up from the same point when a response arrives.

I'm also working toward more flexible schedules for individual agents and teams, with a clearer view of what's running, what's waiting, and what needs my attention. The part I'm most excited about is seeing how far a team can carry a project while keeping me involved in the decisions that matter.