Next.js

Content Moderation in Next.js

Drop-in moderation for App Router projects.

Route Handlers, Server Actions, and edge-compatible SDK usage.

1. Install and create a policy

Run npm install @vettly/sdk. Create a policy in the Vettly dashboard, copy its ID, and set VETTLY_API_KEY and VETTLY_POLICY_ID in .env.local and your deployment environment. Never prefix the key with NEXT_PUBLIC_.

  • Choose explicit allow, warn, flag, and block behavior for your application.
  • Test benign, blocked, and ambiguous comments in the playground before publishing content.

2. Moderate before saving or publishing

The Route Handler below validates input and keeps content unpublished on non-allow decisions or service errors. Authenticate the user and enforce per-user rate limits before making the paid API call. This example evaluates content; add persistence only after an allow decision.

3. Use Server Actions for form submissions

The Server Action below uses the same SDK and policy. Add your session and authorization checks before moderation, then save only when the returned action permits publication. A Server Component can render the form, but moderation belongs in the submission handler, not a render with side effects.

4. Deploy and verify the failure paths

The SDK uses fetch. Set runtime to edge on a Route Handler when deploying to an edge runtime, or omit it for the default Node.js runtime. Server Actions use the runtime of their route segment. Keep API credentials in server-side deployment secrets.

  • Submit a benign comment and confirm an allow decision.
  • Submit a policy test case and confirm it is never published when blocked or flagged.
  • Remove the API key and simulate service failure: return 503 without saving content.
  • Check malformed JSON and oversized input return 400 before calling Vettly.
  • Record decision IDs with saved content; avoid logging secrets or full private messages.
app/actions.ts: Server Action
typescript
'use server';
import { createClient } from '@vettly/sdk';

export async function moderateComment(formData: FormData) {
  // Authenticate and rate-limit the current user here before calling Vettly.
  const content = formData.get('content');
  if (typeof content !== 'string' || !content.trim() || content.length > 5000) {
    return { error: 'Provide 1–5000 characters' };
  }
  const apiKey = process.env.VETTLY_API_KEY;
  const policyId = process.env.VETTLY_POLICY_ID;
  if (!apiKey || !policyId) return { error: 'Moderation unavailable' };
  try {
    const result = await createClient(apiKey, { timeout: 5000, maxRetries: 0 })
      .check({ content, contentType: 'text', policyId });
    // Save only after an allow decision; route other actions to your review UX.
    return { action: result.action, decisionId: result.decisionId };
  } catch {
    return { error: 'Try again later; your comment has not been published' };
  }
}

What it detects

  • • Comment forms
  • • AI chatbot inputs and outputs
  • • User profile fields
  • • Image uploads
  • • Marketplace listings
  • • Custom rules

Why developers choose Vettly

  • • Works in App Router and Pages Router
  • • Edge-compatible SDK
  • • TypeScript-first with full type safety
  • • Pairs cleanly with Server Actions
Example request
bash
// app/api/moderate/route.ts
import { createClient } from '@vettly/sdk';

export const runtime = 'edge';

export async function POST(req: Request): Promise<Response> {
  // Add your session authorization and per-user rate limit here.
  let body: unknown;
  try { body = await req.json(); }
  catch { return Response.json({ error: 'Invalid JSON' }, { status: 400 }); }
  if (!body || typeof body !== 'object' || !('content' in body) ||
      typeof body.content !== 'string' || !body.content.trim() ||
      body.content.length > 5000) {
    return Response.json({ error: 'Provide 1–5000 characters' }, { status: 400 });
  }
  const apiKey = process.env.VETTLY_API_KEY;
  const policyId = process.env.VETTLY_POLICY_ID;
  if (!apiKey || !policyId) {
    return Response.json({ error: 'Moderation unavailable' }, { status: 503 });
  }
  try {
    const client = createClient(apiKey, { timeout: 5000, maxRetries: 0 });
    const result = await client.check({ content: body.content, contentType: 'text', policyId });
    // Never publish flagged or warned content automatically in this example.
    return Response.json({ action: result.action, decisionId: result.decisionId },
      { status: result.action === 'allow' ? 200 : 422 });
  } catch {
    // Keep content unpublished when moderation cannot complete.
    return Response.json({ error: 'Try again later' }, { status: 503 });
  }
}
Example response
json
{
  "flagged": true,
  "action": "block",
  "categories": {
    "harassment": 0.93,
    "hate": 0.02
  },
  "policy": "default",
  "latency_ms": 142
}

Compared to rolling your own classifier

Vettly returns decisions, not raw scores - your route handler stays a few lines instead of a classification pipeline.

See the AI chatbot pattern

Sources and documentation

Keep exploring

AI Chatbot Moderation API

Moderate inputs and LLM outputs in real time. Block prompt injection, NSFW content, and policy violations before users see them.

Vettly vs OpenAI Moderation

Multi-modal coverage, dashboards, and provider freedom vs text-only checks.

Content Moderation API

One endpoint for text, image, and video moderation.

Content Moderation in Node.js

Add content moderation to any Node.js service in minutes. Works with Express, Fastify, NestJS, Hono, and any HTTP framework.

Get an API key

Start with 15,000 free decisions per month. No credit card required.

Get an API key