Hooks — security and quality

Output guardrails and DLP: validating the final answer and protecting data. The list of all hooks is in 06-hooks-reference.

6.9 link_check

Output guardrail: checks links in the final answer and flags those that are not confirmed by sources (grounded) or are dead (live). Grounding is done against the RAG agent's archive (the same archive::open_cached as rag/archive_tool); structural checks (absolute http(s), public host) are always performed. Catches URLs hallucinated by the model — the main failure mode of an agent that the RAG directive asks to cite sources.

kind: link_check
name: link_guard
mode: grounded              # grounded | well_formed | live | both
archive_path: data/ngu      # источник заземления (пусто = только структура)
check_bare_urls: true       # «голые» http(s):// вне markdown
check_html_anchors: false   # HTML <a href>
max_links: 20               # больше — нейтрально
on_invalid: notice          # notice | annotate | unlink | rework
notice_template: "⚠ {{count}} ссылк(и) не подтверждены источниками"
annotate_template: "{{title}} ⚠"
rework_template: ""         # для on_invalid: rework; {{links}}, {{count}}
rework_clear_stream: true   # false = не очищать уже отданный стрим (видимая самоправка / клиент без replace)
# live-режим (обычно только на администраторском глобальном хуке):
allow_hosts: []             # пусто = любой; иначе leaf/example.com + поддомены
timeout_ms: 2000
cache_ttl_secs: 86400       # 0 = без кэша
max_body_bytes: 0           # 0 = HEAD; >0 = GET (тело не читается)

"Valid" = grounded (the URL is in the archive; a trailing / and #… are allowed), structurally safe, and, in live, live. live does not follow redirects (3xx = live) — this avoids SSRF via redirect; 401/403/405/406/429 are considered reachable (anti-bot), a timeout is "unknown" (not flagged), and 404/410 and network errors are broken. Checks run in parallel and are cached in memory by URL. On any error (no archive, no matches, too many links) the hook is neutral — the turn is not broken.

annotate / unlink modify the answer and rely on the re-emit contract (replace); they are safe only on clients that can replace the displayed answer. The default is notice.

rework does not modify but rejects the answer: the core clears the already-delivered stream (best-effort, replace: true) and injects a correction message with a list of unconfirmed links, after which the model answers again — the agent fixes the links within the same turn. The cap of MAX_REWORK_ROUNDS (2) guarantees completion: if the model fails, the turn closes on the last answer (degradation, without looping).

6.12 format_guard

Deterministically tidies up the final answer on after_assistant_message (without an LLM): makes links clickable and removes "bare" URLs. The hook mutates the answer in place, the core re-emits the corrected version; if there is nothing to change — it is neutral.

kind: format_guard
name: format_guard
linkify_bare_urls: true   # https://… → [https://…](https://…)
dedupe_links: true        # дубли ссылок (по target) схлопываются
one_link_per_line: true   # каждая ссылка с новой строки
stream: true              # форматировать и поток токенов (on_token)
notice: ""                # уведомление в UI при изменении (пусто = тихо)

Stream. after_assistant_message modifies the already saved/delivered answer, and the core re-sends the full final_answer (re-emit) — for streaming clients this is an extra frame. With stream: true, links are formatted right in the stream (on_token): safe characters go through immediately, while the URL is buffered until the terminator and inserted already as [url](url). after_assistant_message remains the authoritative final pass (dedup, one-per-line, non-streaming channels); re-applying is idempotent.

6.13 guard

A single stateless security membrane. Scans text at four model boundaries with a common engine agent_os_core::safety::scanner and applies an action per entity class:

Actions: audit_only · notice · redact (irreversible mask [REDACTED]) · deny (refusal) · suppress · rework (rewrite the answer). tokenize is unavailable in guard — use pseudonymize. When classes are mixed, the strictest action wins.

kind: guard
name: dlp
direction: both            # input | output | tool | both (input+output) | all
on_input: redact           # audit_only | notice | redact | deny
on_output: rework          # audit_only | notice | redact | suppress | deny | rework
# on_tool_args: redact     # по умолчанию = on_input
# on_tool_result: redact
entities: [email, phone, card, iban, passport, inn, snils, api_key, pem]
actions:                   # переопределение действия на класс
  api_key: deny
allow: ["support@company.com"]
patterns:                  # свой regex на класс (напр. person)
  person: '\b[А-ЯЁ][а-яё]+\s[А-ЯЁ][а-яё]+\b'
detector: deterministic    # deterministic | llm (принимается; в v1 = deterministic)
audit_only: false          # true = только считать, не менять
notice_template: "⚠ Скрыто {{count}} чувствительных данных"
rework_template: "Убери эти данные и ответь заново: {{kinds}}"
rework_clear_stream: true

NER. detector: ner / hybrid supplements the rules with a local ONNX model (classes in ner_classes, e.g. [person, org, address]) via the core's shared ONNX service; without the service, the hook stays on the rules. Model selection — ner_model (per-agent hook YAML), otherwise the selector by agent template (templates), then default.

6.14 pseudonymize

Reversible surrogate substitution on top of the same scanner. The model sees only [TYPE_n]; the real values are stored locally in a DuckDB vault and expanded on output.

kind: pseudonymize
name: pii_vault
scope: session             # session | user | template
entities: [person, email, phone, address, org, passport]
detokenize_output: true    # разворачивать ответ пользователю (on_token)
detokenize_tool_args: true # side-effect'ы идут с реальными значениями
tokenize_tool_results: true
fail_closed: true          # не отправлять, если суррогат не удалось поставить
vault_path: data/pii_vault.db
placeholder_format: "[{kind}_{n}]"
output_dlp: true           # страховка: маскировать сырое PII в ответе
patterns: {}               # свой regex на класс (person/org требуют его)
allow: []

Vault table (data/pii_vault.db):

CREATE TABLE pii_map (
  scope_key   TEXT NOT NULL,
  placeholder TEXT NOT NULL,
  kind        TEXT NOT NULL,
  value       TEXT NOT NULL,
  created_at  BIGINT NOT NULL,
  last_used   BIGINT NOT NULL,
  PRIMARY KEY (scope_key, placeholder)
);
CREATE INDEX idx_pii_value ON pii_map(scope_key, kind, value);

• One entity → one placeholder (by (scope, kind, value)); numbering is globally unique, so a placeholder from another scope is "unknown".
• scope: session (default) — keys die with the session; user — consistency across sessions; template — shared per template.
• on_token detokenizes the stream with buffering (a placeholder split across chunks is still expanded). The context/transcript remains in surrogates — raw PII never reaches the model or the saved response.
• fail_closed: vault/detector unavailable → inference is rejected (deny), rather than sending raw data.
• detector: ner / hybrid + ner_classes — supplement the rules with a local ONNX model via the core's ONNX service.
• ner_model — select a specific ONNX NER model (per-agent); otherwise the model by agent template (templates), then default.
• ttl_secs / max_entries — vault retention (purge by last_used, eviction).
• scope: session — purge on session close (on_session_closed); scope: user — key migration when users are merged (merge_user).
• vault_key_env / vault_key — vault encryption key: when a key is set, the value column is encrypted with AES-256-GCM, and lookup/dedup is done by HMAC hash (value_hash); without a key — plaintext (with a warning).
• GET /v1/pii/vault/:scope (admin) — subject-access export of decrypted vault records for the scope.

Pseudonymization ≠ anonymization: the data remains personal and re-identifiable; the vault never leaves the perimeter.

6.16 groundedness

Output guardrail on after_assistant_message: splits the final response into claims (sentences) and checks each against the available evidence — the share of content tokens of the claim found in the evidence must be at least min_support. Unsupported claims are shown as a notification; the response is not changed and not suppressed. Catches what link_check misses: a real link, but a fabricated claim. Deterministic, no LLM and no network.

Evidence (evidence) — the RAG agent's archive (BM25 search for each claim) and the session's successful tool results (ngu_search, http_fetch, ... — what the agent actually received).

kind: groundedness
name: grounding_guard
archive_path: data/ngu        # источник evidence (общий archive::open_cached)
evidence: auto                # auto | archive | tools | both (both == auto)
tool_results_limit: 32        # последних tool-результатов сессии в пул
max_evidence_chars: 200000    # потолок текста evidence из инструментов
min_support: 0.5              # доля токенов claim, найденных в evidence
min_claim_chars: 24           # короче — не claim
min_claim_tokens: 3           # меньше содержательных токенов — не claim
max_claims: 30                # больше — не проверять (нейтрально)
top_chunks: 3                 # сколько чанков архива брать на claim
skip_questions: true          # пропускать вопросительные предложения
on_invalid: notice            # notice | rework
rework_template: "Эти утверждения не подтверждаются источниками: {{claims}}. Убери их или подтверди и ответь заново."
rework_clear_stream: true     # для rework: очистить уже отданный стрим (replace)
notice_template: "⚠ {{count}} утверждени(й) не подтверждается источниками: {{claims}}"
max_notice_claims: 3          # сколько claim показать в notice

URLs and markdown markup are stripped from the claim before tokenization (markdown link [текст](url) → текст). The archive is opened once and cached (shared archive::open_cached with rag/link_check/archive_tool); tool results are taken from HookContext.tool_results (ToolResultLookup::recent_all, in-memory per-session, successful only). Neutral if no source is available. Paraphrase and synthesis may be flagged as unsupported — hence by default notice; rework should be enabled after calibrating the threshold on your own corpus. Works alongside link_check (the hooks are independent).

6.18 language_guard

Ensures the final response is written in the expected language — the user's language (target: auto) or a specified one. A hybrid of three points: capture of the language on the user message (before_user_message), an optional preventive directive (before_inference), and a guard of the final response (after_assistant_message).

The key point: after_assistant_message triggers only on the final (tool-free) response, and the user message is no longer there — so the user's language must be captured in advance (in-memory per process, cleared on on_terminate/on_session_closed).

Detection. Script-based, deterministic, no LLM: letters are classified by Unicode ranges (Latin, Cyrillic, Greek, Arabic, Hebrew, Devanagari, Han, Hangul, Thai), and the dominant group is taken. Distinguishes ru↔en, but not en/es/de — for languages of the same script, a refinement tier is enabled (detector):

When min_ratio is below the threshold, the response is considered mixed and is not flagged; short responses (min_chars) are not checked.

The langid model (mini-BERT, ~25 MB int8, 200 languages) is already registered in config/server.yaml → security.onnx.models and is downloaded automatically on first use (download: auto, repository onnx-community/language_detection-ONNX, pinned revision). A server build with the onnx feature is required. If the ONNX service or model is unavailable, the hook is neutral — the turn is not broken.

Which languages are "ok". allowed (list of codes/scripts) is stronger than target (auto = the user's language). Distinguishing languages within a single script (en/es, ru/uk) is only possible with detector: ngram/onnx/llm; with detector: script, any response in the same writing-system family is acceptable.

kind: language_guard
name: language_guard
target: auto            # auto | <код: ru, en, …> | <скрипт: cyrillic, latin, …>
allowed: []             # непусто = ответ обязан быть в одном из списка
min_chars: 40           # короче — не проверяем
min_ratio: 0.5          # доля доминирующего скрипта; ниже — смешанный текст
sample_chars: 400       # образец сообщения пользователя для tier'а
on_mismatch: notice     # notice | rework
notice_template: "⚠️ Ответ не на языке запроса (ожидался {{expected}}, получен {{detected}})"
rework_template: ""     # пусто = встроенный текст; {{expected}}, {{detected}}
rework_clear_stream: true
detector: script        # script | ngram | onnx | llm
onnx_model: langid      # id модели для detector: onnx (см. security.onnx.models)
model: ""               # дешёвая модель для detector: llm
min_confidence: 0.6
judge_instructions: ""  # свой prompt судьи (JSON-ответ)
inject_directive: false # превентивно на before_inference
directive_template: "Always answer in {{language}}."

Outcomes. Match / nothing to check → neutral. Mismatch → notice or rework (the core clears the stream and injects a system correction, cap MAX_REWORK_ROUNDS). replace/suppress are not used: text cannot be translated deterministically (see §6.1 rework). The directive is idempotent (marker).

6.28 schema_guard

Validates structured output against a JSON Schema and, on mismatch, either asks the model to correct itself (rework) or flags it (notice). Complements the output_schema of the process (which only checks the job_done payload): this hook fires on every final response.

Points:

• after_assistant_message (target: answer) — response validation; on_invalid: notice | rework.
• before_tool_call (target: tool_args) — tool argument validation; on_invalid: deny | notice (deny returns a failed result, the model sees the errors and can retry the call).

kind: schema_guard
name: schema_guard
target: answer              # answer | tool_args
on_invalid: rework          # answer: notice | rework ; tool_args: deny | notice
schema:
  type: object
  required: [answer]
  properties:
    answer: { type: string }
# notice_template: "⚠ Схема не соблюдена: {{errors}}"
# rework_template: "Ответ не соответствует схеме: {{errors}}"
# tools: [structured_search]   # tool_args: ограничить список
# exempt_tools: []             # tool_args: исключить

For answer, JSON is extracted even from a fenced block (```json … ```) or text containing a JSON object/array. The validator is the same as for output_schema (agent_os_core::json_schema).