RAG Setup & Integration Guide

This guide covers everything you need to set up RAG (Retrieval-Augmented Generation) in your Stack application, from database configuration to building production-ready knowledge bases.

Prerequisites

Before adding RAG to your application, ensure you have:

Stack installed and configured
Node.js 18+ and pnpm
PostgreSQL with pgvector available — embeddings are pgvector columns, so postgresql is the only datasource RAG runs on
OpenAI API key (for OpenAI embeddings) OR Ollama installed (for local embeddings)

Installation

Install the RAG package:

bash

pnpm add @opensaas/stack-rag

Install your chosen embedding provider:

bash

# For OpenAI embeddings
pnpm add openai

# OR for Ollama (local)
# No package needed - just install and run Ollama

Database Setup

Embeddings live in a native pgvector vector(n) column beside the row, with their metadata in a jsonb column next to it. There is no storage backend to choose, and nothing in your config names pgvector — ragPlugin declares the extension pack itself.

Because every column the plugin emits is a pgvector column, postgresql is the only datasource RAG runs on. pnpm generate refuses any other one, naming the datasource it found.

Making pgvector available

The extension has to be present on the Postgres server before a migration can enable it. The Dev database opensaas dev starts carries it, and the managed services generally offer it — Neon, Supabase and RDS among them. For a server you run yourself:

Using Docker (easiest):

bash

docker run -d \
  --name postgres-rag \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=postgres \
  -e POSTGRES_DB=your_database \
  -p 5432:5432 \
  pgvector/pgvector:pg16

Using Homebrew (macOS):

bash

brew install postgresql@16
brew services start postgresql@16

brew install pgvector
brew services restart postgresql@16

Using apt (Ubuntu/Debian):

bash

sudo apt install postgresql-16 postgresql-16-pgvector
sudo systemctl restart postgresql

Enabling it

You do not run any SQL for this, and there is no install script. ragPlugin's declaration is a generator emission: pnpm generate seeds the pack's contract space under migrations/pgvector/ — it writes no app migration of its own — and that space enables the extension ahead of your tables.

Start the dev loop in one terminal:

bash

pnpm dev

Then, in another:

bash

pnpm generate
pnpm db:update

pnpm db:update needs that loop running. See Migrations and the dev loop for why, and for the --plan <id> a destructive change asks for.

In a deployment there is no loop, so the same committed space is applied as a migration instead: prisma migration plan once against a database you are willing to open a planning connection to, commit the result, then prisma db migrate. There is no opensaas db migrate — opensaas db carries only update.

The privilege it needs

pgvector is not a trusted extension, so enabling it is not something an unprivileged role can do — superuser, a provider grant, or a one-off pre-create by someone who holds the privilege. Plan for that before your first release: the full statement, including what a server with no pgvector at all does, is at Cost: pgvector needs a privilege you may not have.

Indexes

A vector index is declared on the field that owns the column, not written as SQL:

typescript

contentEmbedding: embedding({
  sourceField: 'content',
  dimensions: 1536,
  distanceFunction: 'cosine',
  index: { method: 'hnsw', m: 16, efConstruction: 64 },
})

Index guidelines:

ivfflat: faster to build, good for 10k–1M vectors. Set lists to about the square root of the row count.
hnsw: better search quality, good for 100k+ vectors. A higher m buys quality with memory.
The operator class is derived from the field's distanceFunction and column type, so the two cannot disagree.

Declaring an index today derives the column type and the operator class but does not yet build an index — searches are correct, and exact, without one. See Search exactness for the pack limit behind that and for what changes once it lifts.

Provider Configuration

OpenAI Embeddings

OpenAI provides high-quality embeddings via their API. Best for production applications.

Get API Key

Sign up at OpenAI Platform
Navigate to API Keys
Create a new secret key
Add to .env:

bash

OPENAI_API_KEY=sk-...

Configure Provider

typescript

import { ragPlugin, openaiEmbeddings } from '@opensaas/stack-rag'

ragPlugin({
  provider: openaiEmbeddings({
    apiKey: process.env.OPENAI_API_KEY!,
    model: 'text-embedding-3-small', // or 'text-embedding-3-large'
  }),
})

OpenAI Models

Model	Dimensions	Use Case	Cost (per 1M tokens)
`text-embedding-3-small`	1536	General purpose, cost-effective	$0.02
`text-embedding-3-large`	3072	Higher quality, more expensive	$0.13

Recommendations:

Use text-embedding-3-small for most applications
Use text-embedding-3-large for critical search quality
Consider Ollama for development to avoid API costs

Ollama Embeddings (Local)

Ollama runs embedding models locally. Great for development and applications that need privacy or want to avoid API costs.

Installation

bash

# Install Ollama from https://ollama.ai
# Or use Homebrew on macOS:
brew install ollama

# Start Ollama service
ollama serve

# Pull an embedding model
ollama pull nomic-embed-text

Configure Provider

dimensions is required here. Ollama reports its model's output size only from a live embed call, and pnpm generate must not depend on a running Ollama, so the dimension has to be declared:

typescript

import { ragPlugin, ollamaEmbeddings } from '@opensaas/stack-rag'

ragPlugin({
  provider: ollamaEmbeddings({
    baseURL: 'http://localhost:11434',
    model: 'nomic-embed-text',
    dimensions: 768,
  }),
})

Ollama Models

Model	Dimensions	Use Case
`nomic-embed-text`	768	General purpose, fast
`mxbai-embed-large`	1024	Higher quality
`all-minilm`	384	Very fast, smaller

Recommendations:

Use nomic-embed-text for most applications
Use mxbai-embed-large for better quality
Use all-minilm for maximum speed

Multiple Providers

You can configure multiple providers and choose which to use per field:

typescript

ragPlugin({
  providers: {
    openai: openaiEmbeddings({
      apiKey: process.env.OPENAI_API_KEY!,
      model: 'text-embedding-3-small',
    }),
    ollama: ollamaEmbeddings({
      model: 'nomic-embed-text',
      dimensions: 768,
    }),
  },
})

Each field then names the provider it wants by key — local Ollama for short titles, OpenAI for the body:

typescript

lists: {
  Article: list({
    fields: {
      title: text(),
      titleEmbedding: embedding({
        sourceField: 'title',
        provider: 'ollama',
        dimensions: 768,
      }),
      content: text(),
      contentEmbedding: embedding({
        sourceField: 'content',
        provider: 'openai',
        dimensions: 1536,
      }),
    },
  }),
}

Complete Setup Example

Here's a complete example configuration for a knowledge base application:

Environment Variables

bash

# .env
DATABASE_URL=postgresql://user:password@localhost:5432/knowledge_base
OPENAI_API_KEY=sk-...

OpenSaas Configuration

typescript

// opensaas.config.ts
import { config, list } from '@opensaas/stack-core'
import { text, select, checkbox, timestamp } from '@opensaas/stack-core/fields'
import { ragPlugin, openaiEmbeddings } from '@opensaas/stack-rag'
import { searchable } from '@opensaas/stack-rag/fields'

export default config({
  plugins: [
    ragPlugin({
      provider: openaiEmbeddings({
        apiKey: process.env.OPENAI_API_KEY!,
        model: 'text-embedding-3-small',
      }),
    }),
  ],
  db: { provider: 'postgresql' },
  lists: {
    Article: list({
      fields: {
        title: text({
          validation: { isRequired: true },
        }),
        content: searchable(
          text({
            validation: { isRequired: true },
          }),
          {
            provider: 'openai',
            dimensions: 1536,
          },
        ),
        category: select({
          options: [
            { label: 'Technology', value: 'tech' },
            { label: 'Science', value: 'science' },
            { label: 'Business', value: 'business' },
          ],
        }),
        published: checkbox({
          defaultValue: false,
        }),
        publishedAt: timestamp(),
      },
      access: {
        operation: {
          query: ({ session }) => (session ? true : { published: { equals: true } }),
          create: ({ session }) => !!session,
          update: ({ session }) => !!session,
          delete: ({ session }) => !!session,
        },
      },
    }),
  },
})

Generate Schema

bash

pnpm generate

pnpm dev applies the change; if it is already running, it applies as soon as you save the config.

Create a Semantic Search Function

The whole search is one scoped query — the Access Filter, the minScore bound and the ranking all live inside nearest():

typescript

// lib/search.ts
'use server'

import { createEmbeddingProvider } from '@opensaas/stack-rag/providers'
import { getContext } from '@/.opensaas/context'

export async function searchArticles(query: string, limit = 10) {
  const context = await getContext()

  const provider = createEmbeddingProvider({
    type: 'openai',
    apiKey: process.env.OPENAI_API_KEY!,
  })
  const queryVector = await provider.embed(query)

  const matches = await context.db.Article.where({
    published: { equals: true },
  }).nearest('contentEmbedding', queryVector, { limit, minScore: 0.25 })

  return matches.map((match) => ({
    id: match.item.id,
    title: match.item.title,
    content: match.item.content,
    category: match.item.category,
    similarity: match.score,
  }))
}

minScore is read on the column's own distance function, not on a normalised 0–1 scale:

`distanceFunction`	`score`	Range
`cosine` (default)	the raw cosine	`[-1, 1]`
`l2`	`1 / (1 + d)`	`(0, 1]`
`inner_product`	the dot product	unbounded

So on a cosine column minScore: 0 admits everything more alike than opposite, and anything above about 0.5 is a tight bound most real corpora will not reach. Start loose and tighten against your own data.

Use in a Component

typescript

// app/search/page.tsx
'use client'

import { useState } from 'react'
import { searchArticles } from '@/lib/search'

type SearchHit = Awaited<ReturnType<typeof searchArticles>>[number]

export default function SearchPage() {
  const [query, setQuery] = useState('')
  const [results, setResults] = useState<SearchHit[]>([])
  const [loading, setLoading] = useState(false)

  async function handleSearch(e: React.FormEvent) {
    e.preventDefault()
    setLoading(true)
    setResults(await searchArticles(query))
    setLoading(false)
  }

  return (
    <div className="container mx-auto py-8">
      <form onSubmit={handleSearch} className="mb-8">
        <input
          type="text"
          value={query}
          onChange={(e) => setQuery(e.target.value)}
          placeholder="Search articles..."
          className="w-full px-4 py-2 border rounded"
        />
        <button
          type="submit"
          disabled={loading}
          className="mt-2 px-4 py-2 bg-blue-500 text-white rounded"
        >
          {loading ? 'Searching...' : 'Search'}
        </button>
      </form>

      <div className="space-y-4">
        {results.map((result) => (
          <div key={result.id} className="border rounded p-4">
            <h3 className="text-xl font-bold">{result.title}</h3>
            <p className="text-gray-600 mt-2">{result.content.substring(0, 200)}...</p>
            <div className="mt-2 text-sm text-gray-500">
              Similarity: {(result.similarity * 100).toFixed(1)}%
            </div>
          </div>
        ))}
      </div>
    </div>
  )
}

Example Walkthrough: Building a Chatbot with RAG

This walkthrough demonstrates building an AI chatbot that uses semantic search to provide informed responses based on a knowledge base.

Step 1: Configure RAG

typescript

// opensaas.config.ts
export default config({
  plugins: [
    ragPlugin({
      provider: openaiEmbeddings({
        apiKey: process.env.OPENAI_API_KEY!,
        model: 'text-embedding-3-small',
      }),
    }),
  ],
  db: { provider: 'postgresql' },
  lists: {
    KnowledgeBase: list({
      fields: {
        title: text({ validation: { isRequired: true } }),
        content: searchable(text({ validation: { isRequired: true } }), {
          provider: 'openai',
          dimensions: 1536,
        }),
        category: select({
          options: [
            { label: 'Product', value: 'product' },
            { label: 'Support', value: 'support' },
            { label: 'Technical', value: 'technical' },
          ],
        }),
        published: checkbox({ defaultValue: true }),
      },
    }),
  },
})

Step 2: Create Search Utility

typescript

// lib/knowledge-search.ts
'use server'

import { createEmbeddingProvider } from '@opensaas/stack-rag/providers'
import { getContext } from '@/.opensaas/context'

export async function searchKnowledge(
  query: string,
  options: { limit?: number; minScore?: number } = {},
) {
  const { limit = 3, minScore = 0.25 } = options
  const context = await getContext()

  const provider = createEmbeddingProvider({
    type: 'openai',
    apiKey: process.env.OPENAI_API_KEY!,
  })
  const queryVector = await provider.embed(query)

  const matches = await context.db.KnowledgeBase.where({
    published: { equals: true },
  }).nearest('contentEmbedding', queryVector, { limit, minScore })

  return matches.map((match) => ({
    id: match.item.id,
    title: match.item.title,
    content: match.item.content,
    score: match.score,
  }))
}

Step 3: Create Chat API Route

typescript

// app/api/chat/route.ts
import { streamText } from 'ai'
import { openai } from '@ai-sdk/openai'
import { searchKnowledge } from '@/lib/knowledge-search'

type ChatMessage = { role: 'user' | 'assistant'; content: string }

export async function POST(req: Request) {
  const { messages }: { messages: ChatMessage[] } = await req.json()
  const lastMessage = messages[messages.length - 1]

  const searchResults = await searchKnowledge(lastMessage.content, {
    limit: 3,
    minScore: 0.25,
  })

  let systemMessage = 'You are a helpful AI assistant.'

  if (searchResults.length > 0) {
    systemMessage += '\n\nRelevant information from knowledge base:\n\n'
    searchResults.forEach((result, i) => {
      systemMessage += `[${i + 1}] ${result.title}\n${result.content}\n\n`
    })
    systemMessage += 'Use this information to provide accurate, informed responses.'
  }

  const result = streamText({
    model: openai('gpt-4o-mini'),
    system: systemMessage,
    messages,
  })

  return result.toDataStreamResponse({
    data: {
      sources: searchResults.map((r) => ({
        id: r.id,
        title: r.title,
        score: r.score,
      })),
    },
  })
}

Step 4: Create Chat UI

typescript

// app/chat/page.tsx
'use client'

import { useChat } from 'ai/react'

export default function ChatPage() {
  const { messages, input, handleInputChange, handleSubmit, data } = useChat({
    api: '/api/chat',
  })

  return (
    <div className="container mx-auto max-w-4xl py-8">
      <div className="border rounded-lg h-[600px] flex flex-col">
        {/* Messages */}
        <div className="flex-1 overflow-y-auto p-4 space-y-4">
          {messages.map((message) => (
            <div
              key={message.id}
              className={`flex ${message.role === 'user' ? 'justify-end' : 'justify-start'}`}
            >
              <div
                className={`max-w-[80%] rounded-lg px-4 py-2 ${
                  message.role === 'user'
                    ? 'bg-blue-500 text-white'
                    : 'bg-gray-200 text-gray-900'
                }`}
              >
                <p>{message.content}</p>
              </div>
            </div>
          ))}
        </div>

        {/* Sources */}
        {data?.sources && data.sources.length > 0 && (
          <div className="border-t p-4 bg-gray-50">
            <p className="text-sm font-semibold mb-2">Sources:</p>
            <div className="space-y-1">
              {data.sources.map((source, i) => (
                <p key={i} className="text-sm text-gray-600">
                  {source.title} (Score: {(source.score * 100).toFixed(1)}%)
                </p>
              ))}
            </div>
          </div>
        )}

        {/* Input */}
        <form onSubmit={handleSubmit} className="border-t p-4">
          <input
            value={input}
            onChange={handleInputChange}
            placeholder="Ask a question..."
            className="w-full px-4 py-2 border rounded"
          />
        </form>
      </div>
    </div>
  )
}

Step 5: Seed Knowledge Base

The script runs with no session, so it reaches the database through sudo(), which bypasses access control:

typescript

// scripts/seed-knowledge.ts
import { getContext } from '@/.opensaas/context'

const articles = [
  {
    title: 'Getting Started Guide',
    content: 'Stack is a config-first framework for building admin-heavy applications...',
    category: 'product',
  },
  {
    title: 'Access Control System',
    content: 'The access control system automatically secures all database operations...',
    category: 'technical',
  },
]

async function seed() {
  const context = (await getContext()).sudo()

  for (const article of articles) {
    await context.db.KnowledgeBase.create({ data: article })
    console.log(`Created: ${article.title}`)
  }
}

seed()

Run the seed script:

bash

npx tsx scripts/seed-knowledge.ts

Embeddings will be automatically generated for each article!

Testing RAG Functionality

Test Embedding Generation

typescript

// scripts/test-embeddings.ts
import { createEmbeddingProvider } from '@opensaas/stack-rag/providers'

async function test() {
  const provider = createEmbeddingProvider({
    type: 'openai',
    apiKey: process.env.OPENAI_API_KEY!,
  })

  const embedding = await provider.embed('Hello world')

  console.log('Embedding generated!')
  console.log('Dimensions:', embedding.length)
  console.log('First 5 values:', embedding.slice(0, 5))
}

test()

Test Semantic Search

typescript

// scripts/test-search.ts
import { getContext } from '@/.opensaas/context'
import { createEmbeddingProvider } from '@opensaas/stack-rag/providers'

async function test() {
  const context = await getContext()
  const query = 'How does access control work?'

  console.log(`Searching for: "${query}"`)

  const provider = createEmbeddingProvider({
    type: 'openai',
    apiKey: process.env.OPENAI_API_KEY!,
  })
  const queryVector = await provider.embed(query)

  const matches = await context.db.Article.nearest('contentEmbedding', queryVector, {
    limit: 5,
  })

  console.log(`Found ${matches.length} results:`)
  matches.forEach((match, i) => {
    console.log(`${i + 1}. ${String(match.item.title)} (Score: ${match.score.toFixed(3)})`)
  })
}

test()

Troubleshooting

pgvector Not Available

Error:

typescript

MIGRATION.RUNNER_FAILED ... could not open extension control file ... vector.control

Solution:

pgvector is not present on the server. Make it available, then re-run pnpm db:update with pnpm dev up in another terminal (see Migrations and the dev loop). In a deployment, re-run prisma db migrate instead:

Docker: use the pgvector/pgvector:pg16 image
Homebrew: brew install pgvector
apt: sudo apt install postgresql-16-pgvector
Managed Postgres: Neon, Supabase and RDS offer it; check your provider's extension list

Permission Denied Enabling pgvector

Error:

typescript

ERROR: permission denied to create extension "vector"

Solution:

Grant the migrating role the privilege, or have someone who already holds it pre-create the extension once — see Cost: pgvector needs a privilege you may not have.

OpenAI Rate Limit Errors

The provider wraps every OpenAI failure, so a 429 reaches you as OpenAI embedding generation failed: followed by the SDK's own message (429 Rate limit reached for ...).

Solution:

Wait a moment and retry
Use the batchProcess() utility with rate limiting
Upgrade your OpenAI plan
Consider Ollama for development

Embeddings Not Generating

Symptoms:

contentEmbedding field is always null
No API calls being made

Solutions:

Check API key is set: console.log(process.env.OPENAI_API_KEY)
Verify field configuration has autoGenerate: true or uses searchable()
Check server console for error messages
Ensure content field has a value (required for embedding generation)

Slow Search Performance

Symptoms:

Searches take several seconds
Database CPU usage high

Solutions:

Declare an index on the embedding field:

typescript

contentEmbedding: embedding({
  sourceField: 'content',
  index: { method: 'hnsw', m: 16, efConstruction: 64 },
})

See Indexes for what the pack builds today.

Reduce the result limit:

typescript

context.db.Article.nearest('contentEmbedding', queryVector, { limit: 5 })

Bound the search with minScore, which is lowered into the query as a distance bound rather than filtered afterwards:
typescript
```
context.db.Article.nearest('contentEmbedding', queryVector, { minScore: 0.25 })
```

Database Connection Issues

Error:

typescript

Can't reach database server

Solutions:

Verify database is running: psql -l or docker ps
Check DATABASE_URL is correct
Ensure database exists: psql -l | grep your_database
Check firewall/network settings

Next Steps

RAG Package Reference - Complete API documentation
RAG Advanced Patterns - Custom providers, performance tuning, and advanced use cases
Example: RAG OpenAI Chatbot - Production-ready chatbot implementation

You now have everything you need to implement RAG in your Stack application!