---
title: "AI Connection - DreamFactory Documentation"
source: "https://wiki.dreamfactory.com/AI_Services/AI_Connection"
canonical_url: "https://wiki.dreamfactory.com/AI_Services/AI_Connection"
converted_at: "2026-06-04T17:05:50.013Z"
format: "markdown"
converted_by: "html-to-md-ai"
---
[]()
	
	
	
	# AI Services/AI Connection

	
		From DreamFactory Wiki
		
		
		
		
		[Jump to navigation](#mw-head)
		[Jump to search](#searchInput)
		**Connect DreamFactory to AI/LLM providers (Anthropic, OpenAI, xAI, Ollama, OpenAI-compatible) with built-in rate limiting, cost tracking, prompt logging, and fallback chains.**

## Contents

- [1 AI Connection](#AI_Connection)

- [2 Supported Providers](#Supported_Providers)

- [3 Creating an AI Connection](#Creating_an_AI_Connection)

- [3.1 Configuration Fields](#Configuration_Fields)

- [3.2 Cost Tracking Fields](#Cost_Tracking_Fields)

- [4 Endpoints](#Endpoints)

- [4.1 Completion](#Completion)

- [4.2 Chat](#Chat)

- [4.3 Streaming](#Streaming)

- [4.4 Embeddings](#Embeddings)

- [4.5 Models](#Models)

- [4.6 Health](#Health)

- [4.7 Usage](#Usage)

- [4.8 Data Chat](#Data_Chat)

- [5 OpenAI-Compatible Drop-in Endpoint](#OpenAI-Compatible_Drop-in_Endpoint)

- [6 Fallback Chains](#Fallback_Chains)

- [7 Rate Limiting](#Rate_Limiting)

- [8 Budget Enforcement](#Budget_Enforcement)

- [9 Environment Variables](#Environment_Variables)

- [10 Artisan Commands](#Artisan_Commands)

- [11 See also](#See_also)

## AI Connection

The AI Connection service type lets DreamFactory act as a unified gateway to AI/LLM providers. Instead of embedding provider API keys in every application, you create one or more AI Connection services in DreamFactory, and your applications call DreamFactory's REST endpoints. DreamFactory handles authentication, rate limiting, usage logging, cost tracking, prompt auditing, and provider failover behind the scenes.

**ℹ️ Note:** AI Connection requires a DreamFactory commercial license (Gold or Platinum).

## Supported Providers

**Provider**

**Config Value**

**Default Base URL**

**API Key Required**

**Notes**

**Anthropic (Claude)**

`anthropic`

`[https://api.anthropic.com](https://api.anthropic.com)`

Yes

Supports chat, completion, streaming, tool use

**OpenAI (GPT)**

`openai`

`[https://api.openai.com](https://api.openai.com)`

Yes

Supports chat, completion, embeddings, streaming, tool use

**xAI (Grok)**

`xai`

`[https://api.x.ai](https://api.x.ai)`

Yes

Supports chat, completion, streaming, tool use

**Ollama (Local)**

`ollama`

`[http://localhost:11434](http://localhost:11434)`

No

Self-hosted models. Supports chat, completion, embeddings, tool use (model-dependent)

**OpenAI-Compatible**

`openai_compatible`

*(must set manually)*

Varies

Any endpoint implementing the OpenAI API shape (vLLM, LiteLLM, Azure OpenAI, etc.)

When you select a provider, the Base URL is auto-filled. Override it for self-hosted endpoints, regional gateways, or proxy configurations.

## Creating an AI Connection

1. In the DreamFactory admin panel, go to **Services > Create**.

2. Select **AI Connection** from the service type list.

3. Fill in the required fields:

### Configuration Fields

**Field**

**Required**

**Description**

**AI Provider**

Yes

Which LLM provider this connection talks to. Picking one auto-fills the Base URL.

**API Key**

Varies

Provider API key (encrypted at rest). Required for Anthropic, OpenAI, and xAI. Not required for Ollama.

**Base URL**

Auto

Where requests are sent. Auto-filled from the provider. Override for self-hosted or proxied endpoints.

**Default Model**

No

Model used when a caller does not specify one (e.g. `claude-sonnet-4-5`, `gpt-4o`, `grok-2`, `llama3.2`).

**Max Output Tokens**

No

Cap on response length in tokens. Each request can override. Default: 1024.

**Temperature**

No

Controls randomness. 0 = deterministic, 0.7 = balanced, 1.0 = creative. Default: 0.7.

**Timeout (seconds)**

No

Abort if the provider has not responded. Increase for local Ollama models or long completions. Default: 30.

**Organization ID**

No

OpenAI team-plan billing. Most providers ignore this.

**System Prompt**

No

Instructions injected at the top of every conversation. Cannot be removed by callers. Use for tone, topic restrictions, or persona.

**Rate Limit (RPM)**

No

Per-user cap on requests per minute. 0 or blank = unlimited.

**Allowed Models**

No

JSON array restricting which models callers can request. Empty = all. Example: `["claude-sonnet-4-5","claude-haiku-4-5"]`

**Allowed Roles**

No

JSON array of DreamFactory role IDs permitted to use this connection. Empty = any authenticated user.

**Data Access API Key**

No

When the AI runs tools (read database, list files), it acts under this API key's role. Create a least-privilege key.

**Extra Headers**

No

JSON object of extra HTTP headers for every request. Example: `{"anthropic-version":"2023-06-01"}`

**Extra Parameters**

No

JSON object merged into every request body for provider-specific options. Example: `{"top_p":0.9}`

### Cost Tracking Fields

**Field**

**Description**

**Default cost per 1k input tokens**

Fallback USD rate for the Gateway dashboard. Per-model rates take priority.

**Default cost per 1k output tokens**

Fallback USD rate. Output tokens typically cost 4-5x input tokens.

**Per-model rate sheet**

JSON array of per-model prices for accurate multi-model cost tracking.

**Monthly budget (USD)**

Spend cap per calendar month. The Gateway dashboard projects whether you will exceed it. Does not block requests.

**Per-model rate sheet example:**

```
[
  {"model": "gpt-4o", "input_per_1k": 0.0025, "output_per_1k": 0.01},
  {"model": "gpt-4o-mini", "input_per_1k": 0.00015, "output_per_1k": 0.0006}
]

```

## Endpoints

Once an AI Connection service is created (for example, named `my_ai`), the following endpoints are available.

### Completion

**`POST /api/v2/{service-name}/completion`**

Single-turn text completion. Send a prompt, get a response.

**Request:**

```
{
  "prompt": "Explain what DreamFactory is in one sentence.",
  "model": "claude-sonnet-4-5",
  "max_tokens": 256,
  "temperature": 0.7,
  "stream": false
}

```

**Field**

**Required**

**Description**

`prompt`

Yes

The text prompt. Max 100,000 characters.

`model`

No

Overrides the service default model.

`max_tokens`

No

Max response tokens. Default: 1024.

`temperature`

No

0.0-2.0. Default: 0.7.

`stream`

No

Set `true` for Server-Sent Events streaming.

**Response:**

```
{
  "content": "DreamFactory is an API gateway that generates REST APIs for databases and services.",
  "provider": "anthropic",
  "model": "claude-sonnet-4-5-20250929",
  "input_tokens": 14,
  "output_tokens": 28,
  "finish_reason": "end_turn",
  "latency_ms": 1240
}

```

### Chat

**`POST /api/v2/{service-name}/chat`**

Multi-turn conversation. Send an array of messages with roles.

**Request:**

```
{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What databases does DreamFactory support?"}
  ],
  "model": "gpt-4o",
  "max_tokens": 512,
  "temperature": 0.7,
  "stream": false
}

```

**Field**

**Required**

**Description**

`messages`

Yes

Array of message objects, each with `role` ("system", "user", or "assistant") and `content`.

`model`

No

Overrides the service default.

`max_tokens`

No

Default: 1024.

`temperature`

No

0.0-2.0. Default: 0.7.

`stream`

No

Set `true` for SSE streaming.

**Response:**

```
{
  "content": "DreamFactory supports MySQL, PostgreSQL, SQL Server, Oracle, MongoDB, Snowflake, and many more.",
  "provider": "openai",
  "model": "gpt-4o",
  "input_tokens": 34,
  "output_tokens": 42,
  "finish_reason": "stop",
  "latency_ms": 1850
}

```

### Streaming

Both the `/completion` and `/chat` endpoints support streaming by setting `"stream": true` in the request body. The response uses Server-Sent Events (SSE) in the OpenAI `chat.completion.chunk` format, regardless of the upstream provider. This means any OpenAI-compatible SDK can consume the stream unchanged.

**Response headers:**

```
Content-Type: text/event-stream
Cache-Control: no-cache, no-transform

```

**SSE frame format:**

```
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hello"},"index":0}],"model":"claude-sonnet-4-5"}

data: [DONE]

```

Streaming requests are logged to `ai_usage_log` with accurate token counts. Partial deliveries (client disconnect before completion) are logged with `status=partial`.

### Embeddings

**`POST /api/v2/{service-name}/embeddings`**

Generate vector embeddings from text. Supported by OpenAI, Ollama, and OpenAI-compatible providers. Not all providers support this endpoint.

**Request:**

```
{
  "input": ["DreamFactory is an API gateway.", "It supports many databases."],
  "model": "text-embedding-3-small"
}

```

**Field**

**Required**

**Description**

`input`

Yes

A string or array of strings to embed.

`model`

No

Embedding model. Overrides service default.

**Response:**

```
{
  "data": [
    {"embedding": [0.0023, -0.0091, "..."], "index": 0},
    {"embedding": [0.0041, -0.0012, "..."], "index": 1}
  ],
  "model": "text-embedding-3-small",
  "usage": {
    "prompt_tokens": 12,
    "total_tokens": 12
  }
}

```

### Models

**`GET /api/v2/{service-name}/models`**

List the models available from this provider. Filtered by the connection's `allowed_models` setting if configured.

**Response:**

```
{
  "resource": [
    {"id": "claude-sonnet-4-5-20250929", "name": "Claude Sonnet 4.5", "context_window": 200000},
    {"id": "claude-haiku-4-5-20251001", "name": "Claude Haiku 4.5", "context_window": 200000}
  ]
}

```

### Health

**`GET /api/v2/{service-name}/health`**

Check whether the AI provider is reachable and responsive.

**Response:**

```
{
  "available": true,
  "provider": "anthropic",
  "model": "claude-sonnet-4-5",
  "model_count": 42,
  "latency_ms": 245
}

```

### Usage

**`GET /api/v2/{service-name}/usage`**

Retrieve aggregated usage statistics for this AI Connection. Supports period filtering.

**Query parameters:**

**Parameter**

**Default**

**Description**

`period`

`24h`

Time window. Examples: `24h`, `7d`, `30d`, `90d`.

**Response:**

```
{
  "total_requests": 1547,
  "total_input_tokens": 234567,
  "total_output_tokens": 89012,
  "avg_latency_ms": 1340,
  "errors": 3,
  "period": "7d",
  "since": "2026-05-25T00:00:00+00:00",
  "by_model": {
    "gpt-4o": {"requests": 1200, "input_tokens": 180000, "output_tokens": 70000},
    "gpt-4o-mini": {"requests": 347, "input_tokens": 54567, "output_tokens": 19012}
  },
  "by_user": [
    {"user_id": 1, "requests": 800, "input_tokens": 120000}
  ]
}

```

### Data Chat

**`POST /api/v2/{service-name}/data-chat`** and **`GET /api/v2/{service-name}/data-chat`**

A stateless "chat with your data" endpoint built into every AI Connection. It runs an agentic tool-calling loop where the AI can query your DreamFactory-governed databases (get tables, get schema, query data, call stored procedures/functions) and return answers in natural language.

**GET** returns the current configuration (which databases are accessible, whether tool use is supported).

**POST** runs the agentic loop:

```
{
  "messages": [
    {"role": "user", "content": "How many orders were placed last month?"}
  ],
  "model": "gpt-4o",
  "maxTokens": 4096,
  "temperature": 0.2
}

```

The AI uses the **Data Access API Key** configured on the service to determine which databases and tables it can access.

## OpenAI-Compatible Drop-in Endpoint

DreamFactory provides an OpenAI-compatible gateway that lets existing applications work without code changes. Point your OpenAI SDK at DreamFactory:

```
from openai import OpenAI

client = OpenAI(
    api_key="<your-dreamfactory-api-key>",
    base_url="https://your-df-host/api/v2/_ai/v1",
)

response = client.chat.completions.create(
    model="claude-sonnet",
    messages=[{"role": "user", "content": "Hello!"}],
)

```

**Endpoints:**

**Endpoint**

**Description**

`POST /api/v2/_ai/v1/chat/completions`

Chat completions (sync + streaming)

`GET /api/v2/_ai/v1/models`

List available model aliases

Requests are routed via **model aliases**. An admin creates aliases that map logical names (like `claude-sonnet`) to specific AI Connection services and physical model names. This provides a layer of indirection: you can switch the underlying provider without changing client code.

Model aliases, rate limits, prompt logging, SIEM audit, and fallback chains all apply to OpenAI-compatible requests.

## Fallback Chains

An AI Connection can be configured with fallback service IDs. When the primary provider returns a retryable error (429 rate limit, 5xx server error, or connection timeout), DreamFactory automatically tries the next provider in the chain. Non-retryable errors (401, 403, 400, 422) are not retried.

Each attempt in the chain writes its own `ai_usage_log` row, so cost and latency stay attributable per-provider. A customer sees "Anthropic 429, then OpenAI succeeded" as two separate entries, not one fudged row.

## Rate Limiting

The **Rate limit (RPM)** field on an AI Connection enforces a per-user cap on requests per minute. When a user exceeds the limit, the request returns HTTP 429. This is applied before the provider call, so no tokens are consumed.

## Budget Enforcement

Budgets can be configured at multiple dimensions (global, per-service, per-user, per-role, per-app) with configurable periods (hourly, daily, monthly). Budgets with `hard_stop` enabled block requests when the cumulative spend for the period reaches the cap (HTTP 429). Non-hard-stop budgets are observability-only and trigger webhook alerts at 50%, 80%, and 100% thresholds.

Budget alerting webhooks fire once per threshold per period and include structured payloads with spend details and utilization ratios.

## Environment Variables

**Variable**

**Default**

**Description**

`DF_MCP_HOST`

`[http://127.0.0.1:8006](http://127.0.0.1:8006)`

Internal MCP server host for data-chat tool calls

## Artisan Commands

**Command**

**Description**

`ai:prune-usage-logs`

Remove usage log entries older than the configured retention period (default 90 days)

`ai:prune-prompt-logs`

Remove prompt log entries older than the configured retention period

## See also

- [AI Chat](/index.php?title=AI_Services/Ai_Chat&action=edit&redlink=1)

- [AI Gateway Analytics](/index.php?title=AI_Services/Ai_Gateway&action=edit&redlink=1)

- [MCP Server](/AI_Services/Mcp_Service)

Retrieved from "[https://wiki.dreamfactory.com/index.php?title=AI_Services/AI_Connection&oldid=912](https://wiki.dreamfactory.com/index.php?title=AI_Services/AI_Connection&oldid=912)"
		[Categories](/Special:Categories): - [Ai Connection](/index.php?title=Category:Ai_Connection&action=edit&redlink=1)
- [Ai Integration](/Category:Ai_Integration)
- [Llm](/index.php?title=Category:Llm&action=edit&redlink=1)
- [Difficulty Intermediate](/Category:Difficulty_Intermediate)