> ## Documentation Index
> Fetch the complete documentation index at: https://docs.agenite.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Provider APIs

> Reference for LLM providers in Agenite

## Overview

Agenite supports multiple LLM providers through a consistent interface. Each provider implements the `LLMProvider` interface from the `@agenite/llm` package.

## Common interface

```typescript theme={null}
interface LLMProvider {
  name: string;
  version?: string;

  /**
   * Simple text generation with full response
   */
  generate(
    input: string | BaseMessage[],
    options?: Partial<GenerateOptions>
  ): Promise<GenerateResponse>;

  /**
   * Simple streaming with partial returns
   */
  stream(
    input: string | BaseMessage[],
    options?: Partial<GenerateOptions>
  ): AsyncGenerator<PartialReturn, GenerateResponse, unknown>;

  /**
   * Low-level generation API with full control
   */
  iterate(
    input: string | BaseMessage[],
    options: IterateGenerateOptions
  ): AsyncGenerator<PartialReturn, GenerateResponse, unknown>;
}
```

Most providers extend the `BaseLLMProvider` class, which implements the `iterate` method for you, so you only need to implement `generate` and `stream`.

## OpenAI provider

### Installation

```bash theme={null}
npm install @agenite/openai
```

### Usage

```typescript theme={null}
import { OpenAI } from '@agenite/openai';

const provider = new OpenAI({
  apiKey: 'your-api-key',
  model: 'gpt-4-turbo-preview',
});

const agent = new Agent({
  name: 'openai-agent',
  provider,
  // ... other options
});
```

### Configuration

```typescript theme={null}
interface OpenAIConfig extends BaseLLMConfig {
  apiKey: string;
  model?: OpenAIModel;
  responseFormat?: "text" | "json_object";
  tools?: ToolDefinition[];
  forceTool?: string;
  systemPrompt?: string;
}

type OpenAIModel =
  | "gpt-4-turbo-preview"
  | "gpt-4-0125-preview"
  | "gpt-4-1106-preview"
  | "gpt-4"
  | "gpt-4-32k"
  | "gpt-3.5-turbo"
  | "gpt-3.5-turbo-16k";

interface BaseLLMConfig {
  organization?: string;
  baseURL?: string;
  maxRetries?: number;
  timeout?: number;
}
```

## Anthropic provider

### Installation

```bash theme={null}
npm install @agenite/anthropic
```

### Usage

```typescript theme={null}
import { Anthropic } from '@agenite/anthropic';

const provider = new Anthropic({
  apiKey: 'your-api-key',
  model: 'claude-3-sonnet-20240229',
});

const agent = new Agent({
  name: 'anthropic-agent',
  provider,
  // ... other options
});
```

### Configuration

```typescript theme={null}
interface AnthropicConfig extends BaseLLMConfig {
  apiKey: string;
  model?: string;
  temperature?: number;
  maxTokens?: number;
  stopSequences?: string[];
}
```

## AWS Bedrock provider

### Installation

```bash theme={null}
npm install @agenite/bedrock
```

### Usage

```typescript theme={null}
import { Bedrock } from '@agenite/bedrock';

const provider = new Bedrock({
  region: 'us-west-2',
  model: 'anthropic.claude-3-5-sonnet-20240620-v1:0',
});

const agent = new Agent({
  name: 'bedrock-agent',
  provider,
  // ... other options
});
```

### Configuration

```typescript theme={null}
interface BedrockConfig extends BaseLLMConfig {
  region?: string;
  model: string;
  temperature?: number;
  maxTokens?: number;
  topP?: number;
  stopSequences?: string[];
}
```

## Ollama provider

### Installation

```bash theme={null}
npm install @agenite/ollama
```

### Usage

```typescript theme={null}
import { Ollama } from '@agenite/ollama';

const provider = new Ollama({
  model: 'llama3.2',
  baseUrl: 'http://localhost:11434',
});

const agent = new Agent({
  name: 'ollama-agent',
  provider,
  // ... other options
});
```

### Configuration

```typescript theme={null}
interface OllamaConfig extends BaseLLMConfig {
  model: string;
  baseUrl?: string;
  temperature?: number;
  topP?: number;
  topK?: number;
  repeatPenalty?: number;
  seed?: number;
  stopSequences?: string[];
}
```

## Creating custom providers

You can create custom providers by extending the `BaseLLMProvider` class:

```typescript theme={null}
import { BaseLLMProvider, GenerateOptions, GenerateResponse, PartialReturn } from '@agenite/llm';

class CustomProvider extends BaseLLMProvider {
  readonly name = 'CustomProvider';
  readonly version = '1.0.0';

  constructor(private config: MyCustomConfig) {
    super();
  }

  async generate(
    input: string | BaseMessage[],
    options?: Partial<GenerateOptions>
  ): Promise<GenerateResponse> {
    // 1. Convert input to standard format using convertStringToMessages
    const messages = convertStringToMessages(input);
    
    // 2. Transform to your API's format
    // 3. Call your API
    // 4. Transform the response to Agenite's format
    
    return {
      content: [{ type: 'text', text: 'Response from custom LLM' }],
      tokens: [{ model: 'my-model', inputTokens: 10, outputTokens: 20 }],
      duration: 500,
      stopReason: 'endTurn'
    };
  }

  async *stream(
    input: string | BaseMessage[],
    options?: Partial<GenerateOptions>
  ): AsyncGenerator<PartialReturn, GenerateResponse, unknown> {
    // 1. Convert input to standard format
    // 2. Set up streaming from your API
    
    // 3. Yield chunks as they arrive
    yield { type: 'text', text: 'Partial response...' };
    yield { type: 'text', text: 'More text...' };
    
    // 4. Return final response similar to generate
    return {
      content: [{ type: 'text', text: 'Complete response' }],
      tokens: [{ model: 'my-model', inputTokens: 10, outputTokens: 20 }],
      duration: 500,
      stopReason: 'endTurn'
    };
  }
  
  // The iterate method is already implemented by BaseLLMProvider
}
```

## Response formats

All providers return responses in a standardized format:

```typescript theme={null}
interface TokenUsage {
  inputTokens: number;
  outputTokens: number;
  inputCost: number;
  outputCost: number;
  model: string;
}

interface GenerateResponse {
  content: Array<ContentBlock>;
  tokenUsage: TokenUsage;
  duration: number;
  stopReason?: StopReason;
}

type ContentBlock =
  | TextBlock          // { type: 'text', text: string }
  | ImageBlock         // { type: 'image', source: ... }
  | ToolUseBlock       // { type: 'toolUse', id: string, name: string, input: unknown }
  | ToolResultBlock    // { type: 'toolResult', toolUseId: string, ... }
  | DocumentBlock      // { type: 'document', source: ... }
  | ThinkingBlock      // { type: 'thinking', thinking: string }
  | RedactedThinkingBlock;

type StopReason = 'toolUse' | 'maxTokens' | 'stopSequence' | 'endTurn';
```

## Best practices

1. **Error handling**: Implement robust error handling for API failures
2. **Rate limiting**: Respect provider rate limits
3. **Token tracking**: Always include accurate token usage metrics
4. **Streaming**: Implement efficient streaming to minimize latency
5. **Content mapping**: Properly map specific responses to Agenite's standard format

## Next steps

* Learn about the [Agent API](/api-reference/agent)
* Explore the [Tool API](/api-reference/tool)
* Read about [middleware](/api-reference/middleware)