AI Safety Guardrails — Implementation Guide

The Safety Stack

LLM safety is defense in depth. No single check catches everything, so

you layer multiple detectors:

User Input
    │
    ├── [1] Input Length Check        (instant, catches overflow)
    ├── [2] Injection Detection       (most critical, runs first)
    ├── [3] Toxicity Screening        (content policy)
    ├── [4] PII Detection/Redaction   (privacy protection)
    │
    ▼
LLM Inference
    │
    ├── [5] Output Toxicity Check     (catches model-generated harm)
    ├── [6] Output PII Redaction      (catches leaked data)
    ├── [7] Groundedness Check        (catches hallucination)
    ├── [8] Hallucinated Contact Check (catches fake emails/URLs)
    │
    ▼
User Output

Quick Start Integration

Minimal (5 minutes)

from src.input_filter import InputFilter
from src.output_filter import OutputFilter

input_guard = InputFilter()
output_guard = OutputFilter()

def safe_chat(user_message: str) -> str:
    # Check input
    input_result = input_guard.check(user_message)
    if not input_result.allowed:
        return input_result.rejection_message

    # Call your LLM
    response = your_llm(input_result.filtered_text)

    # Check output
    output_result = output_guard.check(response)
    if not output_result.safe:
        return output_result.filtered_text  # fallback message

    return output_result.filtered_text

Policy-Based (10 minutes)

from src.policy_engine import PolicyEngine, POLICY_PRESETS

# Choose a preset: "default", "strict", or "permissive"
engine = PolicyEngine(POLICY_PRESETS["default"])

def safe_chat(user_message: str, model_response: str) -> str:
    input_decision, output_decision = engine.evaluate_conversation(
        user_message=user_message,
        model_response=model_response,
    )

    if input_decision.action == "block":
        return "I can't process that request."

    if output_decision.action == "block":
        return "I'm sorry, I can't provide that response."

    # Use the processed text (PII redacted if applicable)
    return output_decision.processed_text

Choosing a Policy Preset

When to Create Custom Policies

The presets cover 80% of use cases. Create a custom policy when:

• You need different rules for different user roles (admin vs public)
• You have domain-specific PII patterns (employee IDs, internal codes)
• You want to allow certain content categories that defaults block
• You need custom checks beyond PII/injection/toxicity

Tuning False Positives

PII Detector

The most common false positives:

Injection Detector

False positives usually come from:

• Legitimate "act as" requests: "Act as a Python tutor" — legitimate but matches jailbreak patterns
• Technical discussions about AI: Talking about prompt engineering is different from doing it
• Code snippets: System prompt examples in documentation

Solutions:

• Raise risk_threshold to 0.60-0.70 for technical-audience applications
• Use per-layer weights to reduce structural analysis weight (which catches "act as")
• Maintain an allowlist of known-safe phrases for your domain

Toxicity Detector

The keyword-based approach misses:

• Sarcasm and irony
• Cultural context (words that are slurs in one dialect but not another)
• Implied threats without explicit keywords

For production systems with high stakes, combine this with a

transformer-based toxicity classifier.

Attack Vectors Covered

Prompt Injection

PII Leakage

Content Safety

Logging and Auditing

The policy engine logs every decision. Set up structured logging:

import logging

# Configure JSON logging for production
logging.basicConfig(
    level=logging.INFO,
    format='{"time":"%(asctime)s","level":"%(levelname)s","msg":"%(message)s"}',
)

Every PolicyDecision includes:

• timestamp — when the decision was made
• policy_name — which policy was applied
• action — allow/redact/flag/block
• triggered_rules — which rules fired
• details — per-rule scores and metadata

Performance Considerations

All checks run on Python stdlib with regex-based detection. Typical latency:

For comparison, a single LLM inference call takes 200ms-5s. Safety

checks add negligible overhead.

Limitations

1. No ML models: This toolkit uses heuristics, not trained classifiers.

Accuracy is lower than transformer-based approaches but latency is

orders of magnitude better.

2. English-centric: Patterns are designed for English text. Other

languages need custom keyword lists.

3. Evolving attacks: New jailbreak techniques appear weekly. Update

the known patterns regularly.

4. Context blindness: The detectors analyze text in isolation. They

don't understand conversation history or user intent.

5. Not a replacement for alignment: Safety guardrails are a

defense layer, not a substitute for proper model training and RLHF.