Bifrost Eiguard Plugin

Content safety plugin for bifrost-http v2.3.0-ew-4, implementing two-stage guardrails: Prompt Guard (input, wolf-defender v2) and a Stage 2 content-safety guard (input + output, SystemOne decision service). The plugin itself is eiguard (eiguard.so).

Architecture

                           ┌─────────────────────────────────────────────┐
                           │        bifrost-http (v2.3.0-ew-4)           │
                           │                                             │
  HTTP Request ───────────►│  PreLLMHook: checks request before LLM call │
                           │  PostLLMHook: checks response after LLM call │
                           └──────────┬──────────────────────┬───────────┘
                                      │                      │
                          ┌───────────▼───────────────┐      │
                          │        eiguard.so         │      │
                          │   (Go plugin, CGO build)  │      │
                          │                           │      │
                          │   PromptGuardPool         │      │
                          │   (round-robin)           │      │
                          │        │                  │      │
                          │        ▼                  │      │
                          │   Stage2Guard             │      │
                          │   (SystemOneClient)       │      │
                          └───────────┬───────────────┘      │
                                      │                      │
             ┌────────────────────────▼────────────┐     ┌───▼─────────────────┐
             │       prompt-guard service          │     │ systemone-sglang    │
             │       (CPU, ~3 GB RAM)              │     │ (GPU 0, 80 GB+)     │
             │       port 8010                     │     │ port 8012           │
             │                                     │     │ SGLang server       │
             │       Wolf Defender v2              │     │ Qwen/Qwen3.8-27B    │
             │       (prompt-injection             │     │ POST /v1/systemone  │
             │        classifier)                  │     │ (multi-question     │
             │                                     │     │  decisions)         │
             └─────────────────────────────────────┘     └─────────────────────┘

Components

eiguard.so (Go plugin)

Built as a Go plugin (.so) via Dockerfile.plugin with -buildmode=plugin and CGO_ENABLED=1, loaded by bifrost-http at runtime as a volume-mounted ELF shared object. Exposes four hook functions:

Hook	Trigger	Guard checked	Blocks?
PreLLMHook	Before LLM call	Prompt Guard → Stage 2 guard, including the manipulation question on the input path	Yes, short-circuit with safety message
PostLLMHook	After LLM call	Stage 2 guard	Yes, replaces response with safety message
HTTPTransportPreHook	On HTTP transport layer	Prompt Guard	Yes
HTTPTransportStreamChunkHook	On each streaming chunk	Stage 2 guard on the final chunk only	Yes

The Go plugin maintains round-robin connection pools to distribute load across multiple replica URLs: PromptGuardPool (always) plus the SystemOneClient for the Stage 2 guard.

prompt-guard (Python, CPU)

FastAPI service running Wolf Defender v2, a prompt-injection classifier. No GPU required.

Request: POST /scan → {"text": "..."} Response: {"label": "INJECTION", "scores": {"BENIGN": 0.01, "INJECTION": 0.98, "JAILBREAK": 0.01}}

If scores[label] > threshold (default 0.9), the request is blocked with: "Request blocked: {label} detected. (request_id: ...)"

systemone-sglang (Python/SGLang, GPU)

SGLang server running Qwen/Qwen3.8-27B (BF16, ~54 GB weights plus KV cache; needs an 80 GB+ GPU on device 0). This is the Stage 2 backend.

Request: POST /v1/systemone with one multi-question decision per message, built from the YAML policy (eiguard/README.md, section "Policy schema (v2)"):

• unsafe (choice): one question over the policy's safety categories; the answer carries a probability per option ID.
• manipulation (noul): optional yes/no probability question, asked only on the input path when manipulation_check_enabled is true.

Response: per-question answers; the plugin blocks when an unsafe option probability reaches unsafe_threshold (default 0.9) and, on the input path, when the manipulation probability reaches manipulation_threshold (default 0.9).

The policy lives in a hot-reloaded YAML file; the question set (categories, manipulation criteria) is data, not plugin code.

Accepted limitations

Two limitations are accepted by design and documented here deliberately:

1. Streaming responses are classified only at the final chunk, so the content was already delivered to the client when a block fires. This is pre-existing behavior.
2. State content beyond max_state_chars (default 24000 characters) is split into overlapping segments (up to max_state_segments, default 8); a state exceeding the segment limit blocks on the input path and fails open on the output path, because the sglang server hard-refuses over-long prompts.

Build & Run

1. Build the plugin (produces ./eiguard/eiguard.so)

DOCKER_BUILDKIT=1 docker build \
  --target export \
  --output type=local,dest=./eiguard \
  -f eiguard/Dockerfile.plugin \
  ./eiguard

2. Start the stack (builds bifrost with plugin support, mounts the plugin)

docker compose up -d --wait

Both Dockerfile.bifrost and Dockerfile.plugin are self-contained: they clone the Bifrost source from git at build time, so no external build context is needed. The key difference from upstream is that Dockerfile.bifrost removes -extldflags '-static' to enable dynamic linking for plugin.Open().

3. (Optional) Run Go unit tests

cd eiguard && GOWORK=off go test ./... -count=1

Sglang image tag pre-flight

docker-compose.yml pins lmsysorg/sglang:v0.5.21, which ships ~2026-10-01. Before deploying, verify the tag exists:

docker manifest inspect lmsysorg/sglang:v0.5.21

If the tag does not exist yet at deployment time, use lmsysorg/sglang:nightly pinned by digest so the deployment stays reproducible (see systemone-sglang/README.md).

Architecture Note: Plugin Build

The Go plugin (eiguard.so) and the Bifrost runtime are built separately:

• Bifrost is built from source by Dockerfile.bifrost (self-contained, clones from git) with dynamic linking enabled (no -extldflags '-static')
• The plugin is compiled as a standalone ELF .so via Dockerfile.plugin and mounted into the Bifrost container at runtime

ABI alignment is guaranteed by two mechanisms:

1. Both Dockerfiles clone the exact same Bifrost source at the same BIFROST_REF
2. Both use Go workspaces (go.work) that resolve dependencies through the same set of module paths, ensuring identical type hashes

Both builds use GOTOOLCHAIN=auto and go 1.27.0 in their workspace files to handle the core/go.mod requirement of Go ≥ 1.27.0.

Configuration

Plugin config in eiguard/config/config.json:

{
  "prompt_guard_urls": ["http://prompt-guard:8010"],
  "timeout_ms": 10000,
  "prompt_guard_threshold": 0.9,
  "log_blocked_requests": true,
  "debug": true,
  "eiguard_config_path": "/app/data/eiguard.yaml",
  "eiguard_check_interval_sec": 30,
  "systemone_urls": ["http://systemone-sglang:8012"],
  "systemone_model": "Qwen/Qwen3.8-27B",
  "unsafe_threshold": 0.9,
  "manipulation_check_enabled": true,
  "manipulation_threshold": 0.9,
  "category_threshold": 0.5,
  "max_state_chars": 24000
}

The Stage 2 keys are documented field by field in eiguard/README.md.

Data Flow

Request path (PreLLMHook)

1. Go plugin extracts last user message from schemas.BifrostRequest
2. Sends to prompt-guard → POST /scan
   • If score > threshold → short-circuit with "Request blocked: {label} detected. (request_id: ...)"
3. Runs the Stage 2 guard on the user message (input path, manipulation question included): posts one multi-question decision to systemone-sglang POST /v1/systemone (unsafe choice over the YAML policy categories + manipulation noul question). Block order: manipulation first, then unsafe.
   • Manipulation probability >= manipulation_threshold → short-circuit with "Request blocked: possible prompt manipulation detected (probability: 0.95). (request_id: ...)"
   • Unsafe option probability >= unsafe_threshold → short-circuit with "Request blocked: content violates safety policy (categories: S9, request_id: ...)."
4. If all checks pass, request proceeds to the LLM

Response path (PostLLMHook)

1. Go plugin extracts assistant message from schemas.BifrostResponse
2. Runs the Stage 2 guard on the assistant message (no manipulation question on the output path): POST /v1/systemone with the unsafe question only
   • If unsafe → replaces response with "I'm unable to provide this response as it violates the platform safety policy. (request_id: ...)"

Streaming path (HTTPTransportStreamChunkHook)

Content accumulates across chunks via a context key; the accumulated content is classified once on the final chunk. See "Accepted limitations": the chunks were already forwarded to the client when a final-chunk block fires.

Memory & VRAM

Service	Model	Memory	Notes
prompt-guard	Wolf Defender v2	~3 GB RAM	CPU, no GPU
systemone-sglang	Qwen/Qwen3.8-27B	~54 GB weights + KV cache	GPU 0, 80 GB+ (H100/H200), BF16 via SGLang

With the systemone backend the plugin caps the state client-side at max_state_chars (default 24000 characters) because the sglang server hard-refuses over-long prompts; longer states are segmented (see "Accepted limitations"). Only the last user message is sent for classification on the input path (not the full conversation history).