Middleware APIs

Reference for middleware in Agenite

Overview

Middleware in Agenite allows you to intercept and modify agent behavior at various stages of execution. Middleware can be used for logging, error handling, state management, and more.

Interface

type AsyncGeneratorMiddleware<
  Yield extends MiddlewareBaseYieldValue = MiddlewareBaseYieldValue,
  Return = unknown,
  Next extends MiddlewareBaseNextValue = MiddlewareBaseNextValue,
  Generator extends AsyncGenerator<
    BaseYieldValue & {
      executionContext: StepContext<any>;
    },
    unknown,
    BaseNextValue
  > = BaseAgeniteIterateGenerator,
> = (
  generator: Generator,
  context: StepContext<any>
) => AsyncGenerator<
  Yield | GeneratorYieldType<Generator>,
  Return,
  Next | GeneratorNextType<Generator>
>;

The middleware wraps the agent’s execution generator, allowing you to:

• Intercept and modify values yielded by the generator
• Process the final return value
• Handle next values passed into the generator
• Access the execution context

Middleware Types

export type MiddlewareBaseYieldValue = {
  type: `middleware.${string}`;
  [key: string]: unknown;
};

export type MiddlewareBaseNextValue = {
  type: `middleware.${string}`;
  [key: string]: unknown;
};

Using middleware with an agent

const agent = new Agent({
  name: 'my-agent',
  provider: new BedrockProvider({ model: 'anthropic.claude-3-5-sonnet-20240620-v1:0' }),
  middlewares: [
    // Your middleware functions
    loggingMiddleware(),
    errorHandlingMiddleware()
  ],
  // ... other options
});

Custom middleware examples

Logging middleware

function loggingMiddleware(): AsyncGeneratorMiddleware {
  return async function* (generator, context) {
    console.log('Agent execution started');
    
    let nextValue = undefined;
    try {
      while (true) {
        const { value, done } = await generator.next(nextValue);
        
        if (done) {
          console.log('Agent execution completed');
          return value;
        }
        
        // Log each yielded value
        console.log('Event:', value.type);
        
        // Pass through to the caller
        nextValue = yield value;
      }
    } catch (error) {
      console.error('Agent execution error:', error);
      throw error;
    }
  };
}

Metrics middleware

function metricsMiddleware(): AsyncGeneratorMiddleware {
  return async function* (generator, context) {
    const metrics = {
      startTime: Date.now(),
      yields: 0,
      toolCalls: 0,
    };
    
    let nextValue = undefined;
    try {
      while (true) {
        const { value, done } = await generator.next(nextValue);
        
        if (done) {
          const duration = Date.now() - metrics.startTime;
          console.log('Execution metrics:', {
            ...metrics,
            duration,
          });
          return value;
        }
        
        metrics.yields++;
        if (value.type === 'agenite.tool-call.params') {
          metrics.toolCalls++;
        }
        
        nextValue = yield value;
      }
    } catch (error) {
      throw error;
    }
  };
}

State persistence middleware

function persistenceMiddleware(storage: Storage): AsyncGeneratorMiddleware {
  return async function* (generator, context) {
    // Load persisted state if exists
    const persistedState = await storage.get(`agent:${context.agent.agentConfig.name}`);
    if (persistedState) {
      Object.assign(context.state, JSON.parse(persistedState));
    }
    
    let nextValue = undefined;
    try {
      while (true) {
        const { value, done } = await generator.next(nextValue);
        
        if (done) {
          // Save final state
          await storage.set(
            `agent:${context.agent.agentConfig.name}`,
            JSON.stringify(value)
          );
          return value;
        }
        
        nextValue = yield value;
      }
    } catch (error) {
      throw error;
    }
  };
}

Best practices

1. Order matters: Middlewares execute in the order specified in the array, with each wrapping the ones that come after it
2. Keep it focused: Each middleware should have a single responsibility
3. Error handling: Implement proper error handling within the middleware
4. Performance: Be mindful of performance impacts, especially in middleware that runs for every yield
5. Typing: Use proper TypeScript types for your custom middleware

Next steps

• Learn about providers
• See examples
• Read about core concepts