Running an Open-Source Project in Autonomous Mode

oats is our free, open-source AI notetaker for Mac and Windows. It's public, so anyone can file an issue. That's the point of open source, and it's also where a maintainer's week goes.

Most of that week is toil. You read a report, work out what kind of issue it is, ask for the missing repro steps, open a branch, write the fix, wait for CI, address review comments, and repeat. Very little of it needs the maintainer's judgment. Issue and PR templates help, but someone still has to drive each item to closure.

oats gets few human hours to maintain, so I run it in autonomous mode. GitHub Actions workflows run Claude Code on a self-hosted runner. Every run ends in a pull request or a comment, and nothing merges without me. I steer with one label and one comment, and only mine count. The repo is public, and an agent that can commit shouldn't take instructions from strangers.

Three of the workflows save the most time.

1. Auto-classification: removing toil

Within a minute of an issue being filed, triage sets its type to Bug, Feature, or Task and posts a comment. Nobody has to sort the queue.

The type decides which workflow picks the issue up, so classification isn't just bookkeeping. Triage reads the report, not the template it was filed on: a report on the feature template that describes broken behavior gets typed Bug.

The comment is where the time goes back:

• Bugs get a first-pass analysis: the likely subsystem with real file paths, the repro detail that's missing, and any sign it isn't a bug at all (misconfiguration, expected behavior, already fixed).
• Features get the underlying problem restated, the parts of oats they'd touch, and two or three open questions.
• Tasks usually get no comment. They wait until I decide.

Asking for missing repro steps used to be half my replies. Now the triage comment names the exact fields to request.

2. Autofix: bugs, review findings, and tasks

This one saves the most time. It covers three kinds of work.

Reported bugs. When a report holds up, I add autofix:approved. About fifteen minutes later there's a PR stating the root cause and, just as important, what it could not verify: it didn't run the app, use a real cloud account, or touch a device permission. That section is my test plan. When it can't find the cause, it comments "no confident fix" with what it investigated and what would unblock it. That's far cheaper than a wrong fix.

Critical and major review findings. CodeRabbit reviews every PR, whether an agent or a person wrote it. When a review lands, apply-fixes.yml picks out the 🔴 Critical and 🟠 Major findings, applies them, re-runs the Tauri and Rust builds, and pushes. It fixes 🟡 Minor findings only after triage confirms they're real, and it ignores nitpicks. It replies on every thread and resolves the ones it fixed. Anything that needs a human stays open. The review-and-fix round that used to cost me an afternoon now finishes before I open the PR.

Tasks, with a gate. On a Task, the same label starts autopilot: the superpowers pipeline of clarify, spec, plan, and subagent-driven implementation, run headless and end to end. First it looks for a definition of done: a - [ ] … checklist or a Done when: section. Without one, it declines, comments on what's missing, and adds needs-acceptance-criteria. That pushback matters more than the implementation. An agent with no definition of done still builds something, and someone has to read all of it to learn whether it was the right thing.

3. Shaping: from a vague request to an agent-ready spec

Community feature requests are usually vague. "Add export" could mean a dozen things, and an agent given a vague request builds whichever one it guesses.

Shaping resolves the vagueness first. Triage's open questions are the ones whose answers change the design, and the thread works through them. Then I comment /shape with my decisions:

/shape

Save-to-disk via the native save dialog, not the share sheet. Include the transcript, but behind a checkbox that defaults to off. Local recordings only for a first cut.

Those three sentences settle the open questions, cut the scope, and push the deferred half into Non-goals. What comes back is one file in docs/superpowers/specs/: Problem, Goal, Non-goals, Design, Error handling, Testing, and Open questions.

That spec is written for an agent to build from. A Task that points at it and carries a Done when: list goes straight into autopilot. Shaping turns a request into a precise spec. Autopilot turns the spec into a feature.

Not a loop

Loop engineering, running an agent in a retry loop until it decides it's done, still keeps a developer in the loop: watching the agent, re-prompting it, reading its diffs. Autonomous mode takes the developer role out.

• No code review. Nobody reads a diff line by line. CodeRabbit reviews every PR, apply-fixes.yml fixes what it finds, and the build gate checks the result.
• The work is describing the feature. My time goes into the issue, the /shape comment, and the Done when: list.
• Almost no prompt iteration. I don't coach the agent through a run. When the output is off, I fix the description, not the prompt.
• The only hands-on work is testing. I try the delivered feature in the app and judge the UX. No agent can do that part for me yet.

The outcome: huge time saving

Maintaining oats now takes about two decisions per issue: whether it's worth an attempt, and whether the result is good. I read a triage comment and add a label. I write my decisions into a /shape comment. I test a PR against its "could not verify" list and merge it.

Everything else runs without me: sorting the queue, asking for missing details, first drafts, review fixes, builds, and reports of when it gave up and why. In the last week, github-actions[bot] made more commits to oats than I did.

Bug fixes and features don't just ship faster. They ship without me opening an IDE. GitHub is the control panel: I file and read issues, add labels, comment /shape, read the PR summary, and merge, all in the browser. The code is written, reviewed, and fixed on the runner. I check out a branch only to test a feature by hand.

One outcome I didn't expect: the workflow that pushes back on me most is Task. Most of my tasks come back with needs-acceptance-criteria. I thought I knew what I wanted, but I hadn't written down what "done" looks like. Each pushback costs me a few minutes and saves me hours reviewing something built on a guess. After enough of them, I write a Done when: list without being asked. The agent didn't just take work off my plate. It made me better at describing the work.

If you file an issue on oats

Triage reads your issue and investigates the real codebase, so what you write drives everything downstream. The reports that turn into fixes fastest have:

• Repro steps that actually trigger the bug. This matters more than anything else.
• Which backend: Ariso cloud or on-device. Many code paths split here.
• Version and OS, so a live bug isn't confused with one already fixed.
• The literal error text, quoted. That's what gets searched.
• For features, the problem before the solution: who is blocked, on what, and what they do today.

Comments from anyone are read as context, and good ones shape the design. They just can't start a run.

Try it on your repo

Start with triage alone. It only reads and comments, so it's low-risk, and you'll see value within a day. Then add one gated fix workflow. The gate is the part to copy:

Two rules keep it safe on a public repo. Gate every trigger on one account. And let a separate job, one the agent never touched, do the push and open the PR.

The autonomous-mode guide and the full workflows in .github/workflows/ are public. Borrow them.