Testing LLM Integration Without Flaky CI Runs

Your suite was green yesterday. Today it failed because OpenAI tweaked gpt-4o under the hood, changing "Here is your summary:" to "Summary:". If your tests check prose formatting returned by a large language model, you're building a flaky pipeline that will consume your afternoon on random commits.

The Three Anti-Patterns Breaking Your Test Pipeline

When developers start adding LLM features to Laravel or Next.js applications, they usually fall into one of three traps:

• Hitting live endpoints in CI: Making actual HTTP requests to OpenAI or Anthropic during pest or vitest runs slows your builds from 300ms to 8,000ms per test. It burns API credits, hits rate limits during parallel test execution, and breaks when third-party servers drop packets.
• Asserting on raw text: Testing whether the output contains specific phrases like "Apologies for the inconvenience" guarantees failure. Models are non-deterministic by default. A minor update to model weights renders exact string matches completely useless.
• Unbound HTTP mocking: Writing custom mocks that return hand-crafted objects which don't match the actual HTTP payload returned by the OpenAI SDK creates false security. When OpenAI updates their SDK or API parameters, your mock keeps passing while production crashes.

Record Real HTTP Interactions into Fixtures

The most reliable way to isolate your test suite while keeping mocks accurate is recording actual HTTP interactions. In PHP 8.3 and Laravel 12, you can combine fake HTTP responses with strictly captured JSON files saved directly in your source tree.

Always enforce Http::preventStrayRequests() in your test setup. This guarantees that if a developer writes an LLM wrapper that accidentally leaks a real network call, Laravel aborts the test immediately instead of billing your API key.

Here is a complete Laravel 12 test verifying an LLM-powered feedback classification service without hitting external network routes:

<?php

namespace Tests\Feature;

use App\Services\OpenAISummarizer;
use Illuminate\Support\Facades\Http;
use Tests\TestCase;

class OpenAISummarizerTest extends TestCase
{
    public function test_it_summarizes_customer_feedback_into_structured_json(): void
    {
        Http::preventStrayRequests();

        Http::fake([
            'api.openai.com/v1/chat/completions' => Http::response(
                file_get_contents(base_path('tests/Fixtures/openai_summary_response.json')),
                200
            ),
        ]);

        $summarizer = new OpenAISummarizer();
        $result = $summarizer->summarize('The checkout form crashed twice when submitting my credit card.');

        Http::assertSent(function ($request) {
            return $request->url() === 'https://api.openai.com/v1/chat/completions'
                && $request['temperature'] === 0.0
                && $request['seed'] === 42
                && $request['response_format']['type'] === 'json_schema';
        });

        $this->assertEquals('bug_report', $result['category']);
        $this->assertGreaterThanOrEqual(1, $result['severity']);
        $this->assertIsArray($result['action_items']);
    }
}

By loading static JSON responses recorded directly from an actual OpenAI API payload, your suite runs in under 15ms. Notice how the test verifies the request options rather than just checking the response. It verifies that your application explicitly passes temperature: 0 and a fixed seed value.

Lock Down Variance with Seeds and Temperature Zero

Setting temperature to 0 forces the model to choose the most likely candidate token at every step. However, temperature 0 alone doesn't make modern mixture-of-experts models completely deterministic across different server clusters. OpenAI introduced the seed parameter specifically to address this issue.

When you supply an integer seed (such as 42), OpenAI makes a best effort to return the exact same token sequence for identical prompts. The API response includes a system_fingerprint header or body field. If this fingerprint changes in OpenAI's backend, you know model weights updated on their end, explaining why the generated tokens changed.

In your application code, pass these settings whenever you run operational prompts:

$response = $this->client->chat()->create([
    'model' => 'gpt-4o-mini',
    'temperature' => 0.0,
    'seed' => 1337,
    'messages' => [
        ['role' => 'system', 'content' => 'Extract key entities from the user text.'],
        ['role' => 'user', 'content' => $userInput],
    ],
]);

If you don't enforce these parameters inside your service layer, your test environment won't behave consistently across developer workstations and CI runners.

Assert on JSON Schemas, Not Text Prose

Never ask an LLM to "return a friendly summary" and then check if the text looks good in PHPUnit. If you need structural data, use OpenAI's Structured Outputs feature or strict JSON mode paired with a parser like Zod in Next.js 16 or Webmozart Assert in PHP.

Instead of testing the phrasing, test the shape and types of the extracted payload:

• Is category restricted to an expected enum value (e.g., bug_report, feature_request, billing)?
• Is confidence_score a float between 0.0 and 1.0?
• Is action_items an array containing at least one item when issues are reported?

If you need to verify natural language output, test properties like non-empty string length or character bounds rather than exact word sequences. For example, asserting mb_strlen($result['summary']) > 20 validates that the model produced a non-trivial response without breaking when the phrasing changes from "User reported a bug" to "Customer encountered an error".

Mocking LLMs in Next.js 16 Server Actions with MSW

When testing Server Actions or API routes in Next.js 16, Mock Service Worker (MSW v2) provides an HTTP interceptor at the Node process level. It intercepts fetch requests made by official SDKs without requiring you to override package internals.

Here is a Vitest integration test for a Next.js 16 Server Action that parses support tickets into Zod schemas using MSW to serve fixtures:

import { http, HttpResponse } from 'msw';
import { setupServer } from 'msw/node';
import { processSupportTicket } from '@/actions/process-ticket';
import fixtureResponse from './fixtures/openai-ticket-response.json';

const server = setupServer(
  http.post('https://api.openai.com/v1/chat/completions', async ({ request }) => {
    const body = (await request.json()) as Record<string, any>;

    if (body.temperature !== 0) {
      return new HttpResponse(JSON.stringify({ error: 'Temperature must be 0' }), {
        status: 400,
      });
    }

    return HttpResponse.json(fixtureResponse);
  })
);

beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());
afterAll(() => server.close());

test('extracts structured ticket metadata successfully', async () => {
  const result = await processSupportTicket('Cannot reset password on login screen');

  expect(result.success).toBe(true);
  expect(result.data).toEqual({
    topic: 'authentication',
    urgent: false,
    suggestedResponse: expect.any(String),
  });
});

This approach gives you total isolation. The test executes inside Node without touching OpenAI servers, completes in 8ms, and strictly validates that your Server Action correctly handles structured responses.

Gotchas That Will Break Production

Several edge cases frequently slip past test suites when developers rely exclusively on static mocks:

1. Token Limit Truncation: If a user inputs a 50,000-word document, the model output might truncate midway through a JSON response, causing syntax errors in production. Test this specific failure state by creating a fixture file containing a truncated JSON payload and ensuring your service throws a custom domain exception (like LLMResponseTruncatedException) rather than a raw syntax error.
2. Rate Limiting Exceptions: Your code must handle HTTP 429 status codes gracefully. Write a dedicated test where your mock returns 429 with a Retry-After header, and assert that your application retries using exponential backoff or dispatches a delayed queue job in Laravel.
3. Schema Mismatches After System Prompt Changes: When engineers adjust system prompts, recorded fixtures become stale. Establish a scheduled nightly CI job that sets an environment variable to run tests against the live API, updating fixture files automatically when schemas or model behaviors change.

Focus your unit and integration tests on verifying your system's response to structured outputs, rate limits, and network failures. Leave the prompt evaluation and prose grading to offline evaluation frameworks running outside your standard CI loop.