Rocky Cockpit

If you can read this, the dashboard loaded. Generated 2026-06-21 13:45:01 UTC · refreshes every 60s
LAN: http://192.168.50.10:8787/ · Tailscale: http://100.98.146.114:8787/

OpenClaw Worker

spend-brake-active
OpenClaw spend brake active; default model is openai-codex/gpt-5.4

ROG Mirror

rc=0
mirror ok / no error

Bridge

inbox 7 · running 1
done 9 · failed 10

Worker Queue

idle_collect_context
no inbox work; continue health/report collection

Daily Report

07:30 Kuwait
Telegram concise report + approval queue

Remote Access

Tailscale private
Use http://100.98.146.114:8787/ when away from home

Approval queue

Public/remote access
Needs your approval before tunnel/DNS/port-forward.
Paid model/API upgrade
Wait for repeated rate-limit evidence.
Production/service changes
No auto-change; review first.

Current mission

# Cockpit Current State

Updated: 2026-05-04 14:50 Asia/Kuwait

## Operating mode

Rocky/Hermes is the user-facing chief orchestrator. OpenClaw is now connected as an internal worker path through a bounded file-bridge loop. The loop is safe-by-default: dry-run/report-only, no secrets, no service changes, no deletion, no trading execution.

## Closed workload loop

Active scheduled jobs:

1. Daily Chief Orchestrator Report
   - Job id: a325b28023e0
   - Schedule: 07:30 daily
   - Delivery: Ali Telegram chat
   - Role: concise human-facing daily summary and approval queue.

2. Autonomous OpenClaw Cockpit Loop
   - Job id: f89a47fb2d24
   - Schedule: every 4 hours
   - Delivery: local only
   - Script: ~/.hermes/scripts/openclaw_autonomous_loop.py
   - Role: collect health, attempt bounded OpenClaw worker job unless backoff is active, update state/reports, mirror non-secret cockpit files to ROG.

## OpenClaw worker status

OpenClaw direct model-run is wired into the loop but currently unstable under long prompts / possible subscription or fallback limits.

Latest classification: backoff-active after timeout/fallback issue.

Observed failure detail:
- Router internal timeout hit after 120s.
- stderr included Ollama fallback unavailable at http://127.0.0.1:11434.
- This may be caused by Codex/OpenAI subscription/rate limits, auth/session pressure, or OpenClaw fallback routing.

Current mitigation:
- Do not hammer OpenClaw repeatedly.
- Backoff for 3 hours after timeout/rate/fallback failures.
- Hermes/tool-first reporting remains reliable.
- OpenClaw will be retried automatically by the 4-hour loop.

## Next build priority

Visual cockpit dashboard MVP:
- local first
- self-updating from cockpit state/reports/bridge files
- overview cards and drill-down links
- approval queue
- service pipeline status
- hosted on ROG LAN/private access first
- no public exposure/tunnel/port-forward without Ali approval

## Approval gates remain

Requires Ali approval:
- paid API/subscription changes
- public exposure / DNS / tunnel / port forward
- credential copying/rotation/exposure
- destructive data operations
- production service changes
- live trading
- XRP policy changes

## Hermes approval mode

Updated: 2026-05-04

Hermes approval mode has been set to `smart` in `/Users/alharbi/.hermes/config.yaml`.

Meaning:
- low-risk commands may be auto-approved by Hermes auxiliary judgment
- high-risk/destructive/public/paid/credential/production/live-trading/XRP actions remain gated by Ali approval policy
- gradual movement toward YOLO/off will be considered only after the cockpit, backup, and approval/audit loop prove reliable

## 2026-05-04 continuation after storage recovery

Ali freed Mac storage by safely removing downloaded update data and photos. Verified current Mac disk: `/` and Data have ~86GiB available; Data volume ~58% used.

Dashboard build continued and is live on ROG with pages:
- `/` overview
- `/services.html`
- `/agents.html`
- `/approvals.html`
- `/pipeline.html`
- `/reports.html`
- `/basic.html` fallback

OpenClaw usage status: direct OpenClaw Codex route produced `OPENCLAW_OK` in stdout, but the CLI failed to exit cleanly before timeout and emitted Ollama fallback-unavailable warnings. Router was patched to treat useful stdout + timeout as a partial result instead of total failure while preserving stderr for review. Continue bounded OpenClaw use with backoff; do not hammer subscription.

## 2026-05-06 Symphony-style worker queue

Rocky is now the default decider for the Symphony-style cockpit worker queue. Ali should not need to manage ticket churn. Internal tasks can be created, merged, prioritized, routed, reviewed, or marked stale by Rocky without asking unless a hard approval gate is crossed.

Added cockpit artifacts:
- `SYMPHONY_QUEUE_POLICY.md`
- `tasks/QUEUE.md`
- `state/symphony-queue.json`
- `scripts/symphony-decider.py`
- dashboard page `/queue.html`

Anti-churn defaults: max 6 worker turns, max 2 review/fix cycles, same-group concurrency 1, then Rocky must accept, narrow, or block with a short reason.

ROG health snapshot

=== timestamp ===
2026-06-21 12:27:20 +03 +0300

=== host ===
 Static hostname: rog
       Icon name: computer-laptop
         Chassis: laptop 💻
      Machine ID: 1b06c820d83a4a95bbc9587bfb191699
         Boot ID: a0b2f31bd1c34b4cbc19a01f8e5e2aab
Operating System: Ubuntu 24.04.4 LTS
          Kernel: Linux 6.8.0-124-generic
    Architecture: x86-64
 Hardware Vendor: ASUSTeK Computer Inc.
  Hardware Model: G74Sx
Firmware Version: G74Sx.202
   Firmware Date: Mon 2011-08-15
    Firmware Age: 14y 10month 6d

=== load/memory ===
 12:27:21 up 8 days,  2:03,  0 user,  load average: 1.27, 1.35, 1.67
               total        used        free      shared  buff/cache   available
Mem:            11Gi       7.2Gi       384Mi       118Mi       4.5Gi       4.5Gi
Swap:          4.0Gi       4.0Gi       248Ki

=== disks ===
Filesystem                         Size  Used Avail Use% Mounted on
/dev/mapper/ubuntu--vg-ubuntu--lv  914G  132G  744G  15% /
/dev/sdb1                          916G   16G  855G   2% /data

=== docker ===
NAMES                    STATUS                IMAGE
qbittorrent-mam          Up 8 days             065792d2b11f
radarr                   Up 8 days             lscr.io/linuxserver/radarr:latest
qbittorrent              Up 8 days             lscr.io/linuxserver/qbittorrent:latest
prowlarr                 Up 8 days             lscr.io/linuxserver/prowlarr:latest
sonarr                   Up 8 days             lscr.io/linuxserver/sonarr:latest
gluetun                  Up 8 days (healthy)   qmcgaw/gluetun:latest
scrypted                 Up 8 days             koush/scrypted:latest
go2rtc                   Up 8 days             alexxit/go2rtc:latest
omada-controller         Up 8 days (healthy)   mbentley/omada-controller:6.2.10.17
honcho-api-1             Up 8 days (healthy)   honcho-api
honcho-deriver-1         Up 8 days             honcho-deriver
honcho-redis-1           Up 8 days (healthy)   redis:8.2
honcho-database-1        Up 8 days (healthy)   pgvector/pgvector:pg15
homeassistant            Up 8 days             ghcr.io/home-assistant/home-assistant:stable
moh-scheduler-preview    Up 8 days (healthy)   nginx:1.27-alpine
alarian-alsarraf-pilot   Up 8 days             node:22-alpine
open-webui               Up 8 days (healthy)   ghcr.io/open-webui/open-webui:main
adguardhome              Up 8 days             adguard/adguardhome:latest
glance                   Up 8 days             glanceapp/glance:latest
paperless-ngx            Up 8 days (healthy)   ghcr.io/paperless-ngx/paperless-ngx:latest
paperless-db             Up 8 days             postgres:15-alpine
paperless-redis          Up 8 days             redis:7-alpine
paperless-gotenberg      Up 8 days             gotenberg/gotenberg:8
paperless-tika           Up 8 days             apache/tika:latest
scrutiny                 Up 8 days             ghcr.io/analogj/scrutiny:master-omnibus
dozzle                   Up 8 days             amir20/dozzle:latest
searxng                  Up 8 days             searxng/searxng:latest
kiwix-serve              Up 8 days             ghcr.io/kiwix/kiwix-serve:latest
overseerr                Up 8 days             lscr.io/linuxserver/overseerr:latest
n8n                      Up 8 days             n8nio/n8n:latest
homepage                 Up 8 days (healthy)   ghcr.io/gethomepage/homepage:latest
calibre-web              Up 8 days             lscr.io/linuxserver/calibre-web:latest
calibre                  Up 8 days             lscr.io/linuxserver/calibre:latest
mousesearch              Up 8 days             sevenlayercookie/mousesearch:latest
homebridge               Up 8 days             homebridge/homebridge:latest
flaresolverr             Up 8 days             ghcr.io/flaresolverr/flaresolverr:latest

=== binaries ===
Hermes Agent v0.12.0 (2026.4.30)
Project: /home/rog/.hermes/hermes-agent
Python: 3.11.15
OpenAI SDK: 2.32.0
Up to date
/usr/bin/openclaw
v22.22.0
Python 3.12.3

=== user services matching hermes/openclaw ===
  openclaw-node.service     loaded active running OpenClaw Node Host (v2026.4.14)

=== openclaw node status ===
● openclaw-node.service - OpenClaw Node Host (v2026.4.14)
     Loaded: loaded (/home/rog/.config/systemd/user/openclaw-node.service; enabled; preset: enabled)
     Active: active (running) since Sun 2026-06-21 12:25:12 +03; 2min 11s ago
   Main PID: 38176 (openclaw-node)
      Tasks: 11 (limit: 14223)
     Memory: 229.7M (peak: 346.4M)
        CPU: 10.293s
     CGroup: /user.slice/user-1000.slice/[email protected]/app.slice/openclaw-node.service
             └─38176 openclaw-node

Jun 21 12:25:12 rog systemd[1611]: openclaw-node.service: Scheduled restart job, restart counter is at 5457.
Jun 21 12:25:12 rog systemd[1611]: Started openclaw-node.service - OpenClaw Node Host (v2026.4.14).