Embedding Large Language Models in Laravel: Patterns, Prompt Management, and Cost Control

Why Add LLMs to a Laravel Product?

Large language models (LLMs) have moved from research demos to everyday SaaS features – think auto‑summaries, smart suggestions, or conversational support. For a startup, the value is clear: a differentiated user experience without building a model from scratch. Laravel already offers a solid HTTP client, queue system, and service container, making it a natural host for LLM calls. The challenge is not whether to call an API, but how to do it reliably, cheaply, and with code that stays readable as prompts evolve.

Architectural Patterns for LLM Calls

1. Synchronous vs. Asynchronous

• Synchronous calls are fine for low‑latency UI actions (e.g., autocomplete). Use Laravel's Http::post() directly and return the result.
• Asynchronous calls are better for heavy tasks (e.g., document analysis). Push the request onto a queue, let a worker fetch the response, and notify the client via websockets or email.

2. Service‑Layer Wrapper

Encapsulate every LLM interaction behind a dedicated service class. This isolates HTTP details, makes testing easy, and lets you swap providers later.

<?php
namespace App\Services;

use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Cache;

class LlmClient
{
    protected string $endpoint;
    protected string $apiKey;
    protected int $timeout;

    public function __construct()
    {
        $this->endpoint = config('services.openai.endpoint');
        $this->apiKey   = config('services.openai.key');
        $this->timeout  = 30; // seconds
    }

    /**
     * Send a prompt and return the raw model output.
     */
    public function complete(string $prompt, array $options = []): string
    {
        $payload = array_merge([
            'model' => $options['model'] ?? 'gpt-4o-mini',
            'prompt' => $prompt,
            'max_tokens' => $options['max_tokens'] ?? 512,
        ], $options);

        // Simple cache key – prompts are deterministic for most use‑cases
        $cacheKey = 'llm:' . md5($payload['prompt'] . $payload['model']);
        return Cache::remember($cacheKey, now()->addMinutes(10), function () use ($payload) {
            $response = Http::withHeaders([
                    'Authorization' => "Bearer {$this->apiKey}",
                ])
                ->timeout($this->timeout)
                ->post($this->endpoint, $payload)
                ->throw()
                ->json();
            return $response['choices'][0]['text'] ?? '';
        });
    }
}
?>

3. Queue Job Example

<?php
namespace App\Jobs;

use App\Services\LlmClient;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Support\Facades\Log;

class GenerateReport implements ShouldQueue
{
    use InteractsWithQueue, Queueable;

    protected string $content;

    public function __construct(string $content)
    {
        $this->content = $content;
    }

    public function handle(LlmClient $client)
    {
        $prompt = "Summarize the following text in three bullet points:\n\n" . $this->content;
        $summary = $client->complete($prompt, ['max_tokens' => 150]);
        // Persist or send the summary…
        Log::info('Report generated', ['summary' => $summary]);
    }
}
?>

Prompt Engineering and Versioning

Keep Prompts as First‑Class Assets

Store prompts in version‑controlled files (e.g., resources/prompts/report_summary.txt). That way a change shows up in a pull request, and you can roll back easily.

$prompt = view('prompts.report_summary', ['content' => $text])->render();

Parameterize for Flexibility

Avoid hard‑coding style instructions inside code. Pass them as configuration values or function arguments. When you need to tweak tone or length, you only edit the prompt file, not the service class.

Semantic Versioning for Prompts

Treat a prompt change as a breaking change if it alters the output contract. Tag releases of the prompt folder (e.g., v1.2.0) and store the version alongside cached responses. This prevents a cached result from a previous prompt version being reused inadvertently.

$cacheKey = "llm:v{$promptVersion}:" . md5($payload['prompt']);

Caching, Rate Limiting, and Cost Control

1. Response Caching

Many LLM calls are deterministic: the same prompt yields the same answer. Cache aggressively for read‑only scenarios (FAQ generation, product descriptions). Use a short TTL (5–15 minutes) for dynamic content to avoid stale data.

2. Token‑Based Quotas

Track tokens consumed per user or per month. Laravel's Cache::increment can maintain a simple counter.

$tokens = $response['usage']['total_tokens'] ?? 0;
Cache::increment("user:{$userId}:tokens", $tokens);

When a quota is exceeded, gracefully fall back to a static answer or a cheaper model.

3. Model Selection Strategy

Start with the cheapest model that meets quality requirements (e.g., gpt-4o-mini). Only promote to a higher‑tier model for edge cases where the lower tier fails a simple validation check.

if (strlen($summary) < 50) {
    $summary = $client->complete($prompt, ['model' => 'gpt-4o']);
}

4. Batch Requests

If you need to process many short prompts (e.g., bulk tag generation), batch them into a single API call when the provider supports it. This reduces per‑request overhead and can lower total token cost.

Observability, Monitoring, and Safe Rollback

1. Structured Logging – Log request ID, model, token usage, and latency.
2. Metrics – Export llm_latency_ms, llm_tokens_used, and llm_errors to Prometheus or Laravel Telescope.
3. Alerting – Trigger alerts if error rate exceeds 1 % or cost spikes >20 % week‑over‑week.
4. Feature Flags – Wrap LLM calls behind a flag (features.llm_enabled). If a provider outage occurs, you can turn the feature off without a deploy.
5. A/B Testing – Serve two prompt versions to a subset of users and compare engagement metrics. Store the variant in a cookie or a user attribute.

Takeaway Checklist

• Wrap every LLM interaction in a service class.
• Store prompts as version‑controlled files and include a version identifier in cache keys.
• Cache deterministic responses and set sensible TTLs.
• Track token usage per user and enforce quotas.
• Start cheap, upgrade models only when validation fails.
• Instrument latency, token count, and errors; set up alerts.
• Feature‑flag the whole capability for quick rollback.

By treating LLMs as an external service with the same rigor you apply to any third‑party API, you gain the benefits of AI‑driven features without surprise bill shocks or flaky user experiences. The pattern scales from a single “smart reply” button to enterprise‑grade document analysis pipelines, all within the familiar Laravel ecosystem.