Serverless runtime hardening

Production security and reliability policies for Vercel deployment

Overview

This runbook defines production and preview runtime policies for OOOC Fête Finder on Vercel. These policies ensure secure, reliable operation in ephemeral serverless environments.

Runtime policy

OOOC Fête Finder enforces the following runtime rules:

1. PostgreSQL-backed storage

   KV/runtime store in deployed serverless environments must be PostgreSQL-backed. File system storage is not persistent and must not be relied upon for event or coordinate data.

2. Database health

   DATABASE_URL must be healthy and accessible from all serverless functions. Application startup validates database connectivity.

3. Token-protected endpoints

   Cron routes and deploy revalidation routes must be token-protected using environment variable secrets.

4. No disk persistence

   Runtime disk persistence is not relied on for event or coordinate storage. All data persists in PostgreSQL.

Data mode configuration

The application operates in one of three modes:

// Environment: DATA_MODE
type DataMode = "remote" | "local" | "test";

Remote mode (production)

DATA_MODE=remote
DATABASE_URL=postgresql://...

• Primary source: PostgreSQL event store
• Fallback source: Local CSV (data/events.csv) if store read fails
• Google Sheets: Admin backup/preview only
• Required for production/preview environments

Local mode (development)

DATA_MODE=local

• Primary source: Local CSV (data/events.csv)
• No database required
• Useful for offline development
• Never use in production

Local mode is for development only. Deployed environments must use DATA_MODE=remote.

Test mode

DATA_MODE=test

• Uses in-memory mock data
• No external dependencies
• Used by test suite

Event and revalidation flow

The application follows this data flow in production:

1. Live reads

   runtime-service → DataManager → PostgreSQL event store

   // features/data-management/runtime-service.ts
   export const getEventsData = cache(async () => {
     return await DataManager.getProcessedEvents();
   });
   
2. Homepage publish

   Admin saves events → PostgreSQL → Revalidate path/tag

   // Admin panel: "Save and Revalidate Homepage"
   await LocalEventStore.saveCsv(csv);
   await forceRefreshEventsData();
   revalidatePath("/");
   revalidateTag("events-data");
   
3. Deploy hook revalidation

   Deploy completes → Webhook → POST /api/revalidate/deploy

   Requires Authorization: Bearer <DEPLOY_REVALIDATE_SECRET>

Cron endpoints

Configured in vercel.json:

{
  "crons": [
    { "path": "/api/cron/cleanup-admin-sessions", "schedule": "0 4 * * *" },
    { "path": "/api/cron/cleanup-rate-limits", "schedule": "10 4 * * *" },
    { "path": "/api/cron/backup-event-store", "schedule": "20 4 * * *" }
  ]
}

Cleanup admin sessions

Endpoint: GET /api/cron/cleanup-admin-sessions
Schedule: Daily at 04:00 UTC
Purpose: Remove expired admin authentication sessions

curl https://your-domain.com/api/cron/cleanup-admin-sessions \
  -H "Authorization: Bearer <CRON_SECRET>"

Cleanup rate limits

Endpoint: GET /api/cron/cleanup-rate-limits
Schedule: Daily at 04:10 UTC
Purpose: Prune expired rate limit buckets from KV store

curl https://your-domain.com/api/cron/cleanup-rate-limits \
  -H "Authorization: Bearer <CRON_SECRET>"

Backup event store

Endpoint: GET /api/cron/backup-event-store
Schedule: Daily at 04:20 UTC
Purpose: Create automatic snapshot of event store and featured schedule

curl https://your-domain.com/api/cron/backup-event-store \
  -H "Authorization: Bearer <CRON_SECRET>"

All cron endpoints require Authorization: Bearer <CRON_SECRET> header. Set CRON_SECRET in both Preview and Production environments.

Deploy revalidation endpoint

Endpoint: POST /api/revalidate/deploy (or GET)
Purpose: Invalidate cache after deployments

curl -X POST https://your-domain.com/api/revalidate/deploy \
  -H "Authorization: Bearer <DEPLOY_REVALIDATE_SECRET>"

Response on success:

{
  "success": true,
  "message": "Events data refreshed",
  "count": 123,
  "source": "postgres-store"
}

Configure webhook

1. Add secret to environment

   Generate and add DEPLOY_REVALIDATE_SECRET:

   openssl rand -base64 48
   
2. Create Vercel Deploy Hook

   Project Settings > Git > Deploy Hooks > Create Hook

   • Name: “Post-Deploy Revalidate”
   • Branch: “main”
3. Configure webhook in CI/CD

   After successful deploy, call:

   curl -X POST "https://your-domain.com/api/revalidate/deploy" \
     -H "Authorization: Bearer $DEPLOY_REVALIDATE_SECRET"
   

Security hardening

Environment variable validation

The application validates required variables at startup:

// lib/env.ts (using @t3-oss/env-nextjs)
import { createEnv } from "@t3-oss/env-nextjs";
import { z } from "zod";

export const env = createEnv({
  server: {
    DATABASE_URL: z.string().url(),
    AUTH_SECRET: z.string().min(32),
    DATA_MODE: z.enum(["remote", "local", "test"]),
    CRON_SECRET: z.string().optional(),
    DEPLOY_REVALIDATE_SECRET: z.string().optional(),
  },
  runtimeEnv: process.env,
});

If validation fails, the application will not start.

Cron authentication

All cron routes verify the bearer token:

// app/api/cron/*/route.ts
export async function GET(request: NextRequest) {
  const secret = process.env.CRON_SECRET?.trim();
  const auth = request.headers.get("authorization");
  const token = auth?.startsWith("Bearer ") ? auth.slice(7).trim() : "";
  
  if (!secret || token !== secret) {
    return NextResponse.json(
      { error: "Unauthorized" },
      { status: 401 }
    );
  }
  
  // Process cron job...
}

Never expose CRON_SECRET or DEPLOY_REVALIDATE_SECRET in client-side code or logs.

Admin authentication

Admin endpoints require x-admin-key header:

// Admin route protection
const adminKey = request.headers.get("x-admin-key");
const expectedKey = process.env.ADMIN_KEY?.trim();

if (!expectedKey || adminKey !== expectedKey) {
  return new Response("Unauthorized", { status: 401 });
}

Rate limiting

Public endpoints use in-memory rate limiting:

// Rate limit: 100 requests per 15 minutes per IP
const rateLimit = {
  windowMs: 15 * 60 * 1000,
  max: 100,
};

Node runtime enforcement

Some routes require Node.js runtime (not Edge):

// app/api/revalidate/deploy/route.ts
export const runtime = "nodejs";

// Required for:
// - Database connections (postgres package)
// - File system access
// - Crypto operations

Edge Runtime is faster but has limitations. Use Node runtime for routes that need full Node.js APIs.

Failure checks

If runtime errors appear in preview/production:

1. Verify DATABASE_URL

   Confirm DATABASE_URL exists in both Preview and Production environments in Vercel dashboard.

2. Verify DATA_MODE

   Confirm DATA_MODE=remote is explicitly set (not relying on default).

3. Verify secrets

   Confirm CRON_SECRET and DEPLOY_REVALIDATE_SECRET are set where needed:

   # Check via Vercel CLI
   vercel env ls
   
4. Check runtime configuration

   Verify route handlers use Node runtime where expected:

   export const runtime = "nodejs";
   
5. Review logs

   Check Vercel function logs for specific error messages:

   Vercel Dashboard > Deployments > [Select deployment] > Logs
   

Cache control headers

The application sets appropriate cache headers:

API routes (no caching)

// lib/http/cache-control.ts
export const NO_STORE_HEADERS = {
  "Cache-Control": "private, no-cache, no-store, must-revalidate",
  "Pragma": "no-cache",
  "Expires": "0",
};

Used for:

• Admin endpoints
• Cron jobs
• Revalidation endpoints
• User session endpoints

OG image route (aggressive caching)

vercel.json

{
  "headers": [
    {
      "source": "/api/og",
      "headers": [
        {
          "key": "Cache-Control",
          "value": "s-maxage=86400, stale-while-revalidate=604800"
        }
      ]
    }
  ]
}

Caches for:

• 24 hours fresh (s-maxage=86400)
• 7 days stale-while-revalidate

Monitoring and alerts

Health check script

Run comprehensive diagnostics:

DATABASE_URL="your-url" \
ADMIN_KEY="your-key" \
BASE_URL="https://your-domain.com" \
pnpm health:check

Outputs:

• PostgreSQL connectivity
• KV store key count
• Event table row counts
• Admin health endpoint response
• Postgres KV endpoint response

Vercel metrics

Monitor in Vercel dashboard:

• Function execution duration
• Function error rate
• Function invocation count
• Edge cache hit ratio
• Bandwidth usage

Custom logging

The application logs key events:

// lib/platform/logger.ts
import { log } from "@/lib/platform/logger";

log.info("cache", "Events loaded", { source: "store", count: 123 });
log.warn("geocoding", "Using fallback (API unavailable)");
log.error("database", "Connection failed", undefined, error);

See Logging and observability for details.

Disaster recovery

In case of critical failure:

1. Emergency fallback to local mode

   Only as last resort:

   # Temporarily set DATA_MODE=local
   # Ensure data/events.csv is current
   # This bypasses database entirely
   
2. Restore from backup

   1. Navigate to /admin/operations
   2. Restore latest backup or selected snapshot
   3. Revalidate homepage
   4. Verify runtime source
3. Database recovery

   If database is corrupted:

   # Re-bootstrap from local CSV
   DATABASE_URL="your-url" pnpm bootstrap:postgres-store
   

Setting DATA_MODE=local in production is an emergency measure only. Restore DATA_MODE=remote as soon as possible.

Production checklist

Before going live, verify:

• DATA_MODE=remote in Production and Preview
• DATABASE_URL set and tested
• AUTH_SECRET is at least 32 characters
• ADMIN_KEY is set and documented
• CRON_SECRET is set and matches Vercel cron config
• All three cron jobs are registered in Vercel
• Deploy revalidation webhook is configured
• Health check script runs successfully
• Backup cron job has executed at least once
• Admin panel is accessible and shows healthy status
• Homepage loads events from PostgreSQL (check runtime source)

Next steps

• Configure logging for production monitoring
• Set up backups for disaster recovery
• Deploy to production with confidence