Operations

🔵 Launch, testing, benchmarks, deployment, and monitoring — using only compiled binaries.

9.1 CLI Commands

Everything is done with a single binary agent-os with subcommands (subcommands are extensible by modules via the agent_os_cli registry):

9.2 Starting the Server

./agent-os serve
AGENT_OS_BIND=0.0.0.0:8080 ./agent-os serve        # сменить адрес привязки

Loads .env, config/server.yaml, config/agents/**, config/budgets.yaml, config/web/**. See 03-server-configuration.

9.3 Testing Tools

agent-os tool config/agents/ngu/tools/ngu_search.yaml '{"query":"поступление"}'
agent-os tool config/agents/drom/tools/drom_find_brands.yaml '{"query":"газ"}'

Prints the passed arguments, call_template, output for the LLM, and result_template. Non-zero exit code on error.

9.4 Testing Hooks

agent-os hook config/agents/ngu/hooks/faq_cache.yaml user "Как поступить на фит?"
agent-os hook config/agents/ngu/hooks/faq_cache.yaml turn "вопрос" "ответ" --db /tmp/faq.db
agent-os hook config/scripts/hooks/moderation.yaml user "Привет" --live   # реальный LLM-судья

Commands: info, user "<msg>", turn "<q>" "<a>", loop "<reason>". Flags: --db, --live, --agent, --template, --user, --session.

9.5 Benchmarks

agent-os bench runs scenarios from config/agents/<agent>/bench/scenarios.yaml.

# против работающего сервера
agent-os bench --url=https://HOST --agent=drom --max=5

# поднять собственный локальный сервер (нужен DEEPSEEK_API_KEY в .env)
agent-os bench --port=3001 --max=3

# фильтр по имени сценария (подстрока / regex)
agent-os bench --url=https://HOST --name="Camry|BMW" --max=10

# параллельность + таймаут
agent-os bench --url=https://HOST --agent=drom --parallel=3 --timeout=90

Flags:

Results are written to bench_report_<timestamp>.json in the working folder, with counters passed/failed and per-turn failures.

Scenario Format

name: "Drom Agent"
model: "drom-agent"

scenarios:
  - name: "Поиск Camry"
    turns:
      - user: "Найди Toyota Camry"
        assert:
          contains: [camry]
          min_links: 2
          valid_urls: true

Checks: contains (substrings), min_links / min_urls (number of markdown links), valid_urls (all links http/https).

9.6 Crawling a Site into an Archive

agent-os scrape https://nsu.ru -w 8 -d 3 -n 500          # обход
agent-os scrape --analyze nsu_full.archive               # анализ повторяющихся блоков

Flags: -w/--workers, -d/--max-depth, -n/--max-pages, -r/--rate-limit (ms), -o/--output (archive path), -m/--navmap, --timeout (sec), --no-same-domain.

The archive (a directory with meta.json, docs.bin, terms.bin) is what archive_tool looks for.

9.7 Monitoring and Logs

Reading Traces: agent-os trace and HTTP API

While the server is running, the DuckDB files are locked by its connections, so they cannot be opened from outside. For this there is a read-only HTTP API and CLI on top of the same engine trace_tool:

agent-os trace sessions [--agent TEMPLATE] [--only-errors] [--limit N]
agent-os trace transcript <session_id> [--search QUERY] [--limit N]
agent-os trace events <session_id> [--limit N]
agent-os trace search <query> [--limit N]

The CLI talks to GET /v1/trace/{sessions,events,transcript,search} (admin gate). Base URL — --url / AGENT_OS_URL (default http://127.0.0.1:3000), token — --token / AGENT_OS_ADMIN_TOKEN / the first full token from config/server.yaml. The same endpoints are convenient to call from scripts and the admin UI.

Using trace-agent

Template trace-agent (config/agents/trace/agent.yaml) — a diagnostic assistant on top of logs/events.db. Ask it (in chat) "Which sessions had errors?", "How much did agent X spend?", "Which searches returned zero results?". Its tools trace_* (sessions, events, transcript, search, process) are described in trace_tool.

Dashboard

The page /dashboard (and its API GET /v1/dashboard) gives a summary of the server:

• System — CPU, memory, uptime, build version;
• Provider — current model and base URL, concurrency limit;
• State — number of active/total sessions, active processes, templates, budget groups, tokens spent;
• Auth — whether admin auth is enabled, number of tokens/users;
• Series — time series of activity/tokens from the event store.

9.8 Deploying Config Changes

Config-only changes (agents, tools, hooks, budgets, web content) do not require a rebuild — copy the changed config/ and data/ to the server and restart (or rely on live reload for config/agents/** / config/web/**):

# загрузить
scp -r config/ data/ user@host:/path/to/agent_os/

# перезапустить
ssh user@host "kill \$(pgrep agent-os); cd /path/to/agent_os && \
  set -a && source .env && set +a && nohup ./agent-os serve > agent_os.log 2>&1 &"

# проверить
curl -sk https://HOST/health

If a new tool/hook engine was added (a new kind: that required Rust code), the binary must be rebuilt — but adding agents/tools/hooks on existing engines never requires this.

Deploying Config from a Git Repository

Instead of copying files, point the server at a config repository — it will clone and pin it at startup (or will poll):

# зафиксировать тег при старте
AGENT_OS_CONFIG_REPO=https://github.com/org/agent-config \
AGENT_OS_CONFIG_REF=v1.2.0 \
./agent-os serve

# или опрашивать репозиторий и hot-reload при изменении
AGENT_OS_CONFIG_REPO=https://github.com/org/agent-config \
AGENT_OS_CONFIG_POLL=60 \
./agent-os serve

The allowed commit is logged at startup and is visible at /health in the field config (source, url, ref, commit, poll_secs). To roll out a change — commit/tag in the config repository and restart (pin mode) or wait for the next poll tick. See §3.1.

9.9 TLS

Set a PEM certificate/key via AGENT_OS_TLS_CERT / AGENT_OS_TLS_KEY; otherwise — plain HTTP (for example behind a reverse proxy).