Human-in-the-Loop Engineering

AI can write code quickly. But shipping reliable software will, at least for the time being, remain a human endeavor.

At Ariso, I have been building a workflow around that idea. The goal is to let AI handle more of the implementation work while keeping engineering judgment, verification, and accountability with a person. Just as a manager is responsible for their team's output, you are responsible for your agents' output.

It starts with sprint planning

The process begins in sprint planning. Nothing to write home about here — it's very similar to what it's always been, maybe with some sprinkling of Ari decision making.

Once the planning is over, Ari automatically creates a project for it, and then to-dos appear on my dashboard. That dashboard becomes the starting point for the work I need to build. Instead of keeping a list in my head or jumping between disconnected systems, I can see the implementation work alongside the rest of the project context.

Turning requirements into code with Ivan

From the dashboard, I take each item and rewrite it in my own words inside Ivan, our engineering utility for working with AI-generated code.

That rewrite matters. It forces me to clarify what I actually want before asking for an implementation. The task in the system may be useful for tracking, but my own description contains the technical context, constraints, and intended behavior that Ivan needs to produce a useful first pass.

Ivan takes that description and builds a draft pull request. From there, CodeRabbit reviews it and Ivan automatically addresses the first round of comments.

The dashboard keeps the review queue visible

Once Ivan opens the draft pull requests, custom dashboard widgets show me which ones are waiting for review. The widget currently focuses on draft pull requests opened by me, giving me a clear queue of AI-generated work that still needs human verification.

That visibility is important because the workflow only works if review is part of the system. A draft PR should not disappear into a repository and become someone's vague intention to look at it later. It needs to be visible, ordered, and easy to process.

Ivan review prepares the human reviewer

Before I review the draft PRs, I run them all through the ivan review command in bulk. This utility prepares me to review the generated code as a human reviewer.

The purpose is not to ask another automated process to approve the change. It is to give me useful context before I open the diff: what changed, what the implementation is trying to accomplish, where the risky areas might be, and what I should pay attention to while reviewing.

This is not a code review in and of itself, but rather an outline for me to do the actual review.

turbo-hot-cache makes testing practical

After the Ivan review step, I run turbo-hot-cache. turbo-hot-cache warms every relevant branch so the repo can build in seconds, not minutes, when I'm hopping between reviews.

The human review loop

With the context prepared and the branches warmed, I work through the draft PRs systematically.

For each PR, I:

• Read Ivan's review prep on its local web server
• Open the pull request and read the generated code
• Compare the implementation with the original requirement
• Look for incorrect assumptions, missing edge cases, and changes outside the intended scope
• Run the code locally
• Test the behavior manually
• Make corrections or send the work back through Ivan when needed
• Mark the PR as ready for review

The final status change is meaningful. Moving a PR from draft to ready for review says that a human has inspected the implementation and taken responsibility for presenting it to the rest of the team.

Engineering with a human in the loop

This workflow treats AI-generated code as an input to engineering, not a replacement for engineering.

The system handles a lot of mechanical work: tracking projects, surfacing follow-ups, generating initial implementations, preparing review context, and reducing build wait time. I still decide whether the implementation is correct. I still run the software. I still verify that it matches the requirement.

That division of work lets AI increase the amount of code one engineer can produce without removing the parts of the job that require understanding and judgment.

Human-in-the-loop engineering is a practical operating model: let machines produce and organize more of the work, then build deliberate checkpoints where a human verifies the result before it moves forward.