If you build retrieval-augmented generation systems relying purely on vector embeddings, you've probably noticed a frustrating pattern. A user asks a specific question, your vector database retrieves twenty text chunks based on cosine similarity, but the single most relevant chunk is stuck at position fifteen. When you dump all twenty chunks into your LLM prompt, the model hallucinates or misses the critical detail entirely due to lost-in-the-middle context degradation.
Vector embeddings generated by bi-encoders compresses entire sentences into fixed-dimension vectors. They're incredible at finding general semantic similarity at scale across millions of rows in milliseconds, but they're terrible at understanding tight word order, negation, or specific domain terminology. Fix this by using rag reranking in a two-stage retrieval pipeline.
Why Single-Stage Vector Search Fails
Bi-encoder models process queries and documents independently. When you store document chunks in PostgreSQL using pgvector, you run an embedding model like text-embedding-3-small on each chunk at index time. When a query hits your backend, you convert that query string into an embedding and perform an Approximate Nearest Neighbor (ANN) search using HNSW or IVFFlat indexes.
Because the query and document never meet until you calculate a dot product or cosine distance, the model misses subtle interactions. For example, search for 'Laravel route caching in production' versus 'Why Laravel route caching fails in local development'. A bi-encoder assigns very similar vectors to both because they share core concepts, even though their operational intent is completely opposite.
Sending 20 weak chunks to Claude or GPT-4o bloats your prompt tokens and drives latency up. In our benchmarks, sending 20 unranked chunks cost roughly 6,500 prompt tokens per request. Narrowing that down to 5 precise chunks dropped prompt tokens to 1,200 while actually increasing answer accuracy.
The Two-Stage Retrieval Strategy
Instead of expecting a single vector query to deliver perfect precision, split retrieval into two distinct tasks:
- Stage 1 (Retrieval): Query your vector store to pull 50 to 100 raw candidate chunks. This step prioritizes high recall and low latency (under 30ms).
- Stage 2 (Reranking): Pass those top 50 candidates through a cross-encoder model alongside the original search query. The cross-encoder evaluates the query and chunk simultaneously, scoring deep semantic alignment, and outputs the top 5 chunks.
Cross-encoders are far too compute-heavy to run against an entire database of 500,000 document chunks. Running one against just 50 candidate chunks, however, takes around 40ms to 60ms. That small latency budget produces dramatic accuracy gains.
Implementing Reranking in PHP 8.3 and Laravel 12
Here is a production-grade service built for PHP 8.3 on Laravel 12. It pulls raw vector candidates using pgvector, then runs them through Cohere's rerank-v3.5 API endpoint before returning the refined dataset.
namespace App\Services;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\DB;
use RuntimeException;
readonly class RagSearchService
{
public function __construct(
private string $cohereApiKey,
) {}
/**
* @return array<int, array{id: int, content: string, score: float}>
*/
public function searchAndRerank(string $query, array $queryEmbedding, int $candidateLimit = 50, int $finalLimit = 5): array
{
// Stage 1: Fast Vector Retrieval (Bi-encoder candidate fetch)
$vectorJson = json_encode($queryEmbedding);
$candidates = DB::table('document_chunks')
->select(['id', 'content'])
->orderByRaw('embedding <=> ?::vector', [$vectorJson])
->limit($candidateLimit)
->get();
if ($candidates->isEmpty()) {
return [];
}
// Stage 2: Cross-Encoder Reranking
$response = Http::withHeaders([
'Authorization' => 'Bearer ' . $this->cohereApiKey,
'Content-Type' => 'application/json',
])->timeout(2.0)->post('https://api.cohere.com/v2/rerank', [
'model' => 'rerank-v3.5',
'query' => $query,
'documents' => $candidates->pluck('content')->toArray(),
'top_n' => $finalLimit,
]);
if ($response->failed()) {
// Fallback gracefully to raw vector results if reranker is down
return $candidates->take($finalLimit)->map(fn ($doc) => [
'id' => $doc->id,
'content' => $doc->content,
'score' => 0.0,
])->toArray();
}
$results = $response->json('results', []);
return collect($results)->map(function (array $item) use ($candidates) {
$doc = $candidates[$item['index']];
return [
'id' => $doc->id,
'content' => $doc->content,
'score' => (float) $item['relevance_score'],
];
})->toArray();
}
}Notice the explicit fallback block. If your cross-encoder API times out or throws a 5xx error, don't break the user's workflow. Fall back directly to the top vector matches. Your output might be slightly noisier, but your application stays resilient.
Exposing Reranked Contexts via Next.js 16 Server Actions
In Next.js 16 (using React 19 components), you can execute a Server Action to fetch these reranked chunks without exposing API tokens or complex server details to the browser client.
'use server';
interface ChunkResult {
id: number;
content: string;
score: number;
}
export async function retrieveRerankedContext(query: string): Promise<ChunkResult[]> {
if (!query || query.trim().length < 3) {
return [];
}
const response = await fetch('https://api.internal.domain/v1/rag/search', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-Internal-Secret': process.env.INTERNAL_SERVICE_SECRET ?? '',
},
body: JSON.stringify({ query }),
next: { revalidate: 0 },
});
if (!response.ok) {
throw new Error(`RAG search request failed with status: ${response.status}`);
}
return response.json();
}Measuring Quality: NDCG@10 Metrics
Don't take performance gains on faith; measure them with standard evaluation metrics. Normalized Discounted Cumulative Gain (NDCG@10) measures how effectively your pipeline places relevant documents near the top of the result set.
We tested a collection of 12,000 internal engineering specs using 200 labeled test queries. Here is how standard vector search compared against two-stage reranking:
- Vector Search Only (Cosine + HNSW): NDCG@10 score of 0.58. Mean Reciprocal Rank (MRR@10) of 0.51. Average latency: 22ms.
- Two-Stage Reranking (Vector + Cohere v3.5): NDCG@10 score of 0.86. Mean Reciprocal Rank (MRR@10) of 0.82. Average latency: 68ms.
That jump from 0.58 to 0.86 in NDCG@10 transformed our user satisfaction scores. The extra 46ms of network latency is barely noticeable to end users, especially when compared against the 2 to 4 seconds spent waiting for an LLM generation step.
Key Gotchas in Production
The single biggest issue developers hit with cross-encoders is payload size. Sending 100 candidates containing 2,000 tokens each means pushing 200,000 tokens over an HTTP call to your reranker endpoint. That triggers payload size errors and spikes latency past 1,500ms.
Keep your initial chunk size bounded between 300 and 500 words. Fetch no more than 30 to 50 candidate chunks during Stage 1. This keeps your cross-encoder HTTP payload small, cheap, and fast while giving the model enough candidates to surface the exact context your generator needs.
