Ivan's Inferences #2
Adopting LangChain's Middleware API
AI agents that simply call tools are everywhere. You give a model access to a set of functions, it decides when to call them, and the results flow back into the conversation. It's elegant, powerful, and—for many use cases—entirely sufficient.
But then you hit the wall.
You're building something more sophisticated: a calendar assistant that doesn't just create events, but understands your preferences, schedules video conferences, and updates event details automatically. Suddenly, "just call tools" isn't enough. You need tools that talk to each other, workflows that trigger after specific actions, and context that flows between operations.
This is where LangChain's Middleware API becomes essential.
The Problem: Complex Tool Collaboration
Let's make this concrete. You're building a calendar agent that integrates with Google Calendar over MCP (Model Context Protocol). The user says:
"Schedule a meeting with Sarah next Tuesday at 2pm to discuss the Q1 roadmap."
Sounds simple. But your requirements are:
- Before creating the event: Inject the user's preferences—default meeting duration, preferred video conferencing tool, standard description templates
- After the event is created: Automatically schedule a Zoom meeting, then update the calendar event with the conference link
This isn't one tool call. It's an orchestrated workflow where tools depend on and augment each other.
The Old Way: Callbacks and Tool Chains
Before LangChain's middleware API (pre-v1), solving this required two painful patterns:
Pattern 1: Tools That Call Other Tools
You'd create a "super tool" that internally orchestrates other tools:
const createMeetingWithZoom = tool({
name: "create_meeting_with_zoom",
description: "Creates a calendar event and sets up video conferencing",
async execute({ title, time, attendees }) {
// First, manually inject user preferences
const userPrefs = await getUserPreferences();
// Call the calendar tool
const event = await calendarTool.execute(...);
// Then call Zoom tool
const zoomLink = await zoomTool.execute(...);
// Update the event with the link
await calendarUpdateTool.execute({
eventId: event.id,
description: `${event.description}\n\nJoin: ${zoomLink}`,
});
return event;
},
});The problems compound quickly:
- Tight coupling: Your "super tool" hardcodes the orchestration logic
- No reusability: The calendar tool alone can't be used without Zoom
- Invisible to the model: The LLM doesn't understand what's happening internally
- Testing nightmare: You can't test individual steps in isolation
Pattern 2: Callback Managers
LangChain's callback system let you hook into events:
const callbackManager = new CallbackManager();
callbackManager.addHandler({
handleToolEnd: async (output, runId, parentRunId, tags) => {
if (tags?.includes("calendar_create")) {
// Trigger Zoom creation... somehow
// But how do you access the event data?
// How do you update the calendar afterward?
}
},
});Callbacks gave you hooks, but not control. You could observe that something happened, but actually orchestrating follow-up actions required threading state through a labryinth of handler functions.
The Middleware API: Composable Control
LangChain's middleware API introduces two powerful primitives that solve these problems elegantly:
wrapModelCall: Intercept and modify requests before they reach the modelwrapToolCall: Intercept tool executions and run logic after specific tools complete
These aren't callbacks—they're composable wrappers that give you full control over the execution pipeline.
wrapModelCall: Injecting Context Before Inference
The first challenge was injecting user preferences before the model decides how to create an event. With wrapModelCall, you intercept the request and augment the tool descriptions:
import { createMiddleware } from "langchain";
const withUserPreferences = createMiddleware({
name: "UserPreferencesMiddleware",
wrapModelCall: async (request, handler) => {
augmentToolDescriptions(request.tools, await getUserPreferences());
return handler(request);
},
});Now when the model sees the calendar tool, it understands the user's preferences as part of the tool's contract. The model can make informed decisions—scheduling within working hours, using the preferred video tool—without you hardcoding business logic.
wrapToolCall: Post-Execution Workflows
The second challenge was triggering the Zoom workflow after event creation. wrapToolCall lets you intercept specific tool executions:
import { createMiddleware } from "langchain";
const withZoomIntegration = createMiddleware({
name: "ZoomIntegrationMiddleware",
wrapToolCall: async (request, handler) => {
const result = await handler(request);
if (request.toolCall.name === "google_calendar_create_event") {
const zoom = await createZoomMeeting(result);
await updateCalendarWithZoomLink(result.id, zoom.joinUrl);
}
return result;
},
});The elegance here is separation of concerns:
- The calendar tool stays focused on calendar operations
- The Zoom integration is a composable layer
- You can add, remove, or modify integrations without touching core tools
- Each piece is independently testable
Composing Middlewares
The real power emerges when you compose multiple middlewares:
import { createMiddleware } from "langchain";
const middlewares = [
withUserPreferences, // Inject preferences before model call
withZoomIntegration, // Auto-add video conferencing
withNotifications, // Send Slack notifications on creation
];
const agent = createReactAgent({
llm: new ChatOpenAI({ model: "gpt-4" }),
tools: await mcpClient.getTools(),
middlewares,
});Each middleware is a single-purpose unit. You compose them declaratively. The execution order is explicit and predictable.
Why This Matters
The middleware pattern isn't just about cleaner code—it's about building AI systems that scale:
Separation of concerns: Core tools stay simple. Integration logic lives in composable layers.
Testability: Test your calendar tool without Zoom. Test your Zoom middleware with a mock calendar response.
Flexibility: Swap integrations per-user, per-workspace, or per-request. A/B test different workflows without touching core logic.
Observability: Each middleware can add its own logging, metrics, and tracing.
Type safety: The middleware signatures enforce that you can't accidentally break the execution chain.
The Shift in Mental Model
The old way of thinking: "I need a tool that does X, Y, and Z."
The middleware way: "I need a tool that does X. I'll compose Y and Z as layers around it."
This is the same shift that made Express middleware, Redux middleware, and fetch interceptors so powerful. It's separation of concerns applied to AI agent architecture.
If you're building agents that need to coordinate multiple services, inject user context, or trigger workflows based on tool results, the middleware API is worth adopting. It transforms brittle, monolithic tool implementations into composable, maintainable systems.