> ## Documentation Index
> Fetch the complete documentation index at: https://vlyai-1c28d863.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI

> Add OpenAI's models — GPT-4o, GPT-4o mini, embeddings, image generation, audio — to your vly app. The default LLM provider for AI features.

export const Prerequisites = ({items}) => <Note>
    <strong>Before you start.</strong>
    <ul>
      {(items || ["A vly.ai account (free tier is enough)", "A modern browser (Chrome, Firefox, Safari, Edge — all current versions)", "About 10 minutes"]).map((item, i) => <li key={i}>{item}</li>)}
    </ul>
  </Note>;

[OpenAI](https://openai.com) is the most-used AI integration in vly. The integration supports chat completions, embeddings (for vector search), image generation, and audio (Whisper STT and TTS).

## Prerequisites

<Prerequisites
  items={[
"A vly project",
"An OpenAI account with API access",
"An OpenAI API key (starts with sk-...)",
"A small amount of credit on your OpenAI account ($5 covers extensive testing)"
]}
/>

## Setup

<Steps>
  <Step title="Get your OpenAI API key">
    1. Sign in at [platform.openai.com](https://platform.openai.com).
    2. Click **API keys → Create new secret key**.
    3. Name it `vly-yourapp` (or similar).
    4. Copy the key (starts with `sk-...`). You won't see it again.
  </Step>

  <Step title="Connect OpenAI in vly">
    In a prompt:

    ```text theme={null}
    Add OpenAI for AI features. I'll provide my API key.
    ```

    vly opens the integration setup dialog. Paste your key and confirm. The key is stored encrypted as `OPENAI_API_KEY`.
  </Step>

  <Step title="Use it in a feature">
    ```text theme={null}
    When a task is created, generate a one-line summary of the task
    description using OpenAI's gpt-4o-mini model. Save it in a new
    `summary` field on the task.
    ```

    vly creates the action, calls the API, and writes the result back via a mutation. See the code patterns below for what gets generated.
  </Step>
</Steps>

## Configuration

<ParamField path="OPENAI_API_KEY" type="string" required>
  Your OpenAI API key. Server-only. Starts with `sk-...`.
</ParamField>

<ParamField path="OPENAI_ORG_ID" type="string">
  Optional. Your OpenAI organization ID, if you want to scope usage to a specific org.
</ParamField>

<ParamField path="OPENAI_MODEL" type="string" default="gpt-4o-mini">
  Default model for generic AI features. Override per-call. Common values: `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`.
</ParamField>

## Code patterns

### Chat completion

```typescript convex/ai.ts theme={null}
import { action } from "./_generated/server";
import { v } from "convex/values";
import OpenAI from "openai";

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY! });

export const summarize = action({
  args: { text: v.string() },
  handler: async (_ctx, { text }) => {
    const response = await openai.chat.completions.create({
      model: "gpt-4o-mini",
      messages: [
        { role: "system", content: "Summarize the following in one sentence." },
        { role: "user",   content: text },
      ],
      max_tokens: 100,
    });
    return response.choices[0].message.content ?? "";
  },
});
```

### Streaming chat

For chatbot UIs, stream tokens to the client as they arrive:

```typescript convex/ai.ts theme={null}
import { action } from "./_generated/server";

export const streamChat = action({
  args: { messages: v.array(v.object({
    role: v.union(v.literal("user"), v.literal("assistant"), v.literal("system")),
    content: v.string(),
  })) },
  handler: async (ctx, { messages }) => {
    const stream = await openai.chat.completions.create({
      model: "gpt-4o",
      messages,
      stream: true,
    });

    let buffer = "";
    for await (const chunk of stream) {
      const delta = chunk.choices[0]?.delta?.content ?? "";
      buffer += delta;
      // Update a Convex document with the running buffer so the
      // client's reactive query auto-updates.
      await ctx.runMutation(api.chats.appendStream, {
        chatId: messages[0].chatId,
        content: buffer,
      });
    }
    return buffer;
  },
});
```

The client reads via `useQuery(api.chats.get, { id })` and sees tokens stream in real time.

### Embeddings (for vector search)

```typescript convex/ai.ts theme={null}
export const embed = action({
  args: { text: v.string() },
  handler: async (_ctx, { text }) => {
    const response = await openai.embeddings.create({
      model: "text-embedding-3-small",
      input: text,
    });
    return response.data[0].embedding;
  },
});
```

Pair with [Convex vector search](/convex/vector-search) for semantic queries.

### Image generation (DALL-E)

```typescript convex/ai.ts theme={null}
export const generateImage = action({
  args: { prompt: v.string() },
  handler: async (_ctx, { prompt }) => {
    const response = await openai.images.generate({
      model: "dall-e-3",
      prompt,
      n: 1,
      size: "1024x1024",
    });
    return response.data[0].url;
  },
});
```

For persistent storage, download the image URL and upload to [Convex file storage](/convex/file-storage).

### Audio (Whisper STT)

```typescript convex/ai.ts theme={null}
export const transcribe = action({
  args: { audioUrl: v.string() },
  handler: async (_ctx, { audioUrl }) => {
    const audio = await fetch(audioUrl).then((r) => r.arrayBuffer());
    const file  = new File([audio], "audio.mp3", { type: "audio/mpeg" });

    const response = await openai.audio.transcriptions.create({
      file,
      model: "whisper-1",
    });
    return response.text;
  },
});
```

### Tool / function calling

For agents that can call your own functions:

```typescript theme={null}
const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages,
  tools: [{
    type: "function",
    function: {
      name: "search_tasks",
      description: "Search tasks by title or tag",
      parameters: {
        type: "object",
        properties: {
          query: { type: "string", description: "Search text" },
        },
        required: ["query"],
      },
    },
  }],
});

if (response.choices[0].message.tool_calls) {
  // Run the requested function and feed the result back
  // ...
}
```

Use this for [AI chatbots that take actions](/recipes/ai-chatbot).

## Cost management

OpenAI bills per token. Control costs by:

<CardGroup cols={2}>
  <Card title="Use the smallest model that works" icon="circle-dollar-sign">
    `gpt-4o-mini` is **15× cheaper** than `gpt-4o` and good enough for most summarization, extraction, and simple chat tasks.
  </Card>

  <Card title="Cap tokens" icon="ruler">
    Set `max_tokens` on every call. Prevents runaway responses and runaway bills.
  </Card>

  <Card title="Cache responses" icon="archive">
    For deterministic prompts (summarize the same text), cache the result in a Convex table keyed by content hash.
  </Card>

  <Card title="Set a usage limit in OpenAI" icon="alert-triangle">
    OpenAI dashboard → Settings → Limits. Set a hard monthly cap; calls fail when exceeded (better than a surprise bill).
  </Card>
</CardGroup>

## Choosing a model

| Task | Recommended | Notes |
| - | - | - |
| Quick chat / chatbot | `gpt-4o-mini` | Fast, cheap, good enough for 80% of conversational use |
| High-quality reasoning | `gpt-4o` | When you need careful logic — code review, complex extraction |
| Embeddings | `text-embedding-3-small` | 1536 dims, cheap, accurate |
| Image generation | `dall-e-3` | Best quality, slower than DALL-E 2 |
| Speech-to-text | `whisper-1` | Multi-lingual, robust to noise |
| Text-to-speech | `tts-1` | 6 voices, natural-sounding |

## Comparison with other providers

| Capability | OpenAI | [Anthropic](/integrations/anthropic) | [Google AI](/integrations/google-ai) |
| - | - | - | - |
| Chat / completion | ✓ | ✓ | ✓ |
| Embeddings | ✓ | ✗ | ✓ |
| Image generation | ✓ (DALL-E) | ✗ | ✓ (Imagen) |
| Audio (STT) | ✓ (Whisper) | ✗ | ✓ |
| Audio (TTS) | ✓ | ✗ | ✓ |
| Function calling | ✓ | ✓ | ✓ |
| Vision (image input) | ✓ | ✓ | ✓ |
| Pricing | \$\$ | \$\$ | \$ |

For chatbots, Anthropic's Claude is often a better choice for nuanced conversations. For everything else, OpenAI is the default.

## Troubleshooting

<AccordionGroup>
  <Accordion title="'You exceeded your current quota'" icon="alert-triangle">
    Your OpenAI account has no credit, or you hit your usage limit. Add credit at [platform.openai.com/billing](https://platform.openai.com/billing).
  </Accordion>

  <Accordion title="'The model gpt-4o does not exist or you do not have access'" icon="alert-triangle">
    Some models require tier-1+ access. Check [Tier requirements](https://platform.openai.com/docs/guides/rate-limits/usage-tiers). Most accounts auto-graduate after \$5 of usage.
  </Accordion>

  <Accordion title="Streaming response 'cuts off'" icon="alert-triangle">
    `max_tokens` is set too low, or the model hit its context window. Increase `max_tokens`, or for very long inputs, use a model with a larger context window.
  </Accordion>

  <Accordion title="Slow response time" icon="clock">
    Some prompts are slow regardless of model. Add a loading state in the UI; for chatbots, switch to streaming so users see partial output.
  </Accordion>
</AccordionGroup>

## Related

<CardGroup cols={3}>
  <Card title="AI chatbot recipe" icon="message-circle" href="/recipes/ai-chatbot">
    Complete walkthrough — streaming, history, tool calls.
  </Card>

  <Card title="Image generator recipe" icon="image" href="/recipes/image-generator">
    User-facing image generation with DALL-E.
  </Card>

  <Card title="Vector search" icon="brain" href="/convex/vector-search">
    Semantic search with OpenAI embeddings + Convex.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.