--- name: hermes-ops-and-deployment description: 'Use when operating Hermes-adjacent infrastructure: cron job cost control, web_search/web_extract provider configuration, Railway CLI/API deployment inspection, and budget-aware service/provider choices. Covers live config verification, kill switches, provider splits, GraphQL/API gotchas, and post-action smoke checks.' version: 1.0.0 author: Hermes Agent license: MIT metadata: hermes: tags: [hermes, devops, cron, cost-control, web-providers, railway, deployment, providers] related_skills: [hermes-agent, vps-app-deployment, terminal-docs-research-fallback, cron-job-cost-design, hermes-web-providers, railway-cli] --- # Hermes Ops and Deployment ## Overview Use this umbrella for operational work around Hermes and its adjacent deployment/provider stack: scheduled jobs, web tool backends, Railway projects, cost controls, and smoke-test discipline. The absorbed skills are preserved under `references/absorbed/` for exact commands, schemas, scripts, and incident notes. ## When to Use - Creating, auditing, pausing, or debugging Hermes cron jobs. - Investigating unexpected LLM/API credit burn or 429/fallback-model incidents. - Configuring `web_search` / `web_extract` providers, adding a web provider plugin, or designing tiered fallbacks. - Inspecting Railway projects/services/deployments/domains from the VPS. - Choosing budget-aware provider/service setups and verifying live config. - Performing non-destructive runtime/deployment/operations audits of Hermes-adjacent VPS apps; see `vps-app-deployment` → `references/non-destructive-runtime-audit.md` for the detailed PM2/nginx/ports/health/log/storage/container evidence pattern. ## Cron Job Cost Control Treat cron jobs by cost class: - `no_agent=True` scripts: zero LLM tokens; best for deterministic scans/watchdogs/status snapshots. - Sparse LLM jobs: daily/hourly summaries or reasoning passes. - Frequent LLM jobs: high cost; minimize tools and cadence. - Frequent LLM jobs with delegation: highest risk; use finite repeats or deterministic gates. Emergency playbook: list jobs first, categorize by schedule/model/toolsets/script/delegation, pause rather than remove unless asked, verify with a final list. Real project progress usually lives in the app DB/state file, not only cron transcripts. When Alex asks whether one cron job caused token/quota burn, do historical attribution before resuming anything: query `~/.hermes/state.db` sessions whose IDs match `cron__%`, aggregate input/output/cache-read tokens by job, and label any billable estimate as a proxy because cache reads and subscription-backed providers may not map cleanly to dollars. For a measured single run of a paused recurring job, capture a baseline, trigger `cronjob(action='run')`, pause the job again as soon as the new session appears so recurrence is not left enabled, wait for completion, then compare usage/output. If provider/session token counters are zero, compare saved session JSON payload shape (`~/.hermes/sessions/session_cron__*.json`: file size, message count, tool-result chars, tool-call argument chars) rather than calling it free. See `references/cron-token-attribution.md` for exact queries, report shape, JSON-payload fallback, and the Codex cron zero-token accounting caveat. When Alex asks to speed up/slow down a project cron worker, update the whole project cadence deliberately: list jobs first, identify the primary worker plus any companion no-agent budget watcher, update both schedules, tighten the worker prompt/toolsets for token diet if cadence increases, lower watcher thresholds if needed, and verify with a final cron list. See `references/project-cron-cadence-and-token-diet.md` for the concrete pattern and HD Wiki example. Full preserved detail: `references/absorbed/cron-job-cost-design/SKILL.md`, including the triage script and state-shape references. For project-specific autonomous agents on remote VPSes, use `references/token-cautious-remote-hermes-project-loops.md`: dedicated Hermes user, project skill, Telegram allowlist, systemd user timer, max-run/STOP controls, reduced toolsets, and per-run iteration/time caps. When handing an active project from a parent session to that dedicated agent, also use `references/token-cautious-remote-hermes-project-takeover.md`: durable handoff/state files, a bounded acknowledgement pass, explicit takeover loop priority, and ad-hoc verification of the handoff wiring. ## Web Search / Extract Providers Hermes selects one backend per capability: `web.search_backend`, `web.extract_backend`, then shared `web.backend`, then auto-detect. SearXNG/DDGS/Brave are search-only; Firecrawl/Tavily/Exa/Parallel can extract; Jina/Crawl4AI/Playwright-direct require custom plugin work unless already installed. There is no config-only tiered fallback chain; write a provider plugin if the user wants `Jina → Crawl4AI → Firecrawl`. Remember `web_extract` also uses an auxiliary summarizer model, which is separate from the fetcher and can dominate cost. Full preserved detail: `references/absorbed/hermes-web-providers/SKILL.md`. ## Railway CLI/API Operations Use `RAILWAY_API_TOKEN` for account tokens; `RAILWAY_TOKEN` is for project tokens and causes misleading auth failures in cross-project work. `railway list` is project-level; service/deployment/source/domain detail usually requires the Railway GraphQL API. Service source/domains/deployments live under `serviceInstances.edges.node`, not directly on `Service`. When identifying the source of truth among similar services, check latest successful deployment plus custom domains; latest failed deploy is not what production serves. Full preserved detail: `references/absorbed/railway-cli/SKILL.md`. ## Stale Tracker / Topology Assumption Audits When auditing a live planning surface for stale architecture assumptions (for example Viewer pages, persistent checklists, cron prompts, repo docs, or memory notes): 1. Establish the canonical source of truth first (ADR/status doc/memory entry) and state it explicitly. 2. Inspect the published surface, not only repo markdown. For Viewer-backed pages, query `/home/avalon/apps/viewer/viewer.sqlite` (`pages`, `checklist_items`) because public HTML/checklist rows can lag or diverge from source docs. 3. Audit Hermes cron from `hermes cron list --all` plus `~/.hermes/cron/jobs.json`; check prompts for stale assumptions and whether a job refreshes all relevant published slugs, not just one page. 4. Search durable trackers broadly: current repo docs, generated/archived research pages, skill/reference notes, memories, and recent cron/session artifacts when relevant. 5. Distinguish **canonical current guidance**, **historical/superseded research**, and **useful guard text**. Do not treat all matches as bugs; call out whether to rewrite, add a supersession banner, or preserve as a warning. 6. Report exact identifiers for remediation: Viewer slug, checklist `item_id`, cron job id/name/schedule, file path + line, and the stale phrase. 7. For read-only audits, explicitly say no files/services were changed and note any pre-existing dirty git state if observed. ## Operational Verification Checklist - [ ] Live config/env/token shape is verified instead of assumed. - [ ] Destructive cron/deployment operations are scoped and verified after execution. - [ ] Cost implications are estimated in runs/day and fanout, not vibes. - [ ] Provider capabilities match intended use (search-only vs extract-capable). - [ ] Deployment/service source-of-truth is verified through API data and public smoke checks. - [ ] Published planning/tracker surfaces are checked against their canonical source of truth, including Viewer SQLite/checklist state when Viewer is involved. - [ ] Non-destructive audits avoid restarts/reloads/builds/deploys/edits and use only read-only health checks; report exact PM2 ids/names, ports, paths, statuses, and timestamps. - [ ] Secrets are never echoed into chat/logs. ## Common Pitfalls - Using usage counters as proof a cron/skill/provider is safe or important. - Leaving high-cadence LLM/delegation jobs running after an experiment. - Expecting web-provider config to express fallback chains. - Confusing Railway account tokens with project tokens. - Reporting deployment success without checking the public health/domain route.