brahman/crates/modules/shuma/docs/DEVELOPMENT.md

# Development guide — shipote

Cómo compilar, correr, testear y debuggear shipote desde la raíz del monorepo (`/home/sergio/brahman`).

## Compilar

Todos los crates de shipote y su soporte:
```sh
cargo build -p shipote-daemon -p shipote-cli -p shipote-shell
```

Sólo lo esencial (daemon + cli, sin GUI):
```sh
cargo build -p shipote-daemon -p shipote-cli
```

Workspace completo:
```sh
cargo check --workspace
```

## Correr el daemon

```sh
SHIPOTE_LOG=info ./target/debug/shipote-daemon
```

Filtros útiles:
```sh
SHIPOTE_LOG=warn,shipote_core=info ./target/debug/shipote-daemon          # menos ruido
SHIPOTE_LOG=warn,audit=info ./target/debug/shipote-daemon                 # sólo audit
SHIPOTE_LOG=info,shipote_core::pipeline=debug ./target/debug/shipote-daemon # debug del pipeline
```

El daemon:
- Escucha `$XDG_RUNTIME_DIR/shipote.sock` (fallback `/run/user/$UID/shipote.sock`).
- Restaura snapshot de `$XDG_STATE_HOME/shipote/state.json` (fallback `$HOME/.local/state/shipote/state.json`).
- Relanza pipelines `live` (con `restart_on_failure=true`) automáticamente.
- Reaper cada 500 ms (cosecha zombies + drena pending restarts + chequea quota breaches).
- SIGTERM/SIGINT → snapshot + `stop_with_grace(1s)` de todos los workspaces + exit.

## Correr el shell (GUI)

```sh
cargo run -p shipote-shell
```

Polling cada 2 s al daemon. Si el daemon no está corriendo, muestra "Daemon DOWN" banner rojo. Sparkline de RSS por workspace. Cards: Estado, Capabilities, Workspaces, Comandos, Saved pipelines, Flow channels, Quota breaches, Live tail.

Requiere DISPLAY (X11/Wayland) — no corre en sandbox sin gráfico.

## Correr el CLI

```sh
./target/debug/shipote ping
./target/debug/shipote health
./target/debug/shipote workspace create demo.toml
```

Ver [CLI.md](CLI.md) para referencia completa.

## Tests

Suite completa (85 tests al cierre de Fase R):
```sh
cargo test -p ente-incarnate \
           -p shipote-card \
           -p shipote-protocol \
           -p shipote-discern \
           -p shipote-core \
           -p yahweh-provider-fs \
           -p nouser-core \
           --no-fail-fast
```

Breakdown:
| Crate | Tests |
|---|---|
| `ente-incarnate` | 16 (clone+ns, stdio, pre_exec, caps detect, cgroup paths) |
| `nouser-core` | 27 (pre-existentes — discerner integrado sin regresiones) |
| `shipote-card` | 8 (roundtrip TOML/JSON, compile-to-card, intersect_soma) |
| `shipote-core` | 26 (workspaces, pipeline fan-in/out, flow_channel replay, stats history, restart, quota enforce, snapshot dirty-skip, ...) |
| `shipote-discern` | 5 (MagicBytes, JsonProbe, CardProbe, TomlProbe, Utf8Probe) |
| `yahweh-provider-fs` | 3 (discern_head + integración) |

Tests específicos útiles:
```sh
# Fan-in (merger):
cargo test -p shipote-core --lib pipeline_fanin -- --nocapture

# Replay buffer:
cargo test -p shipote-core --lib replay -- --nocapture

# Quota enforcement:
cargo test -p shipote-core --lib quota_enforce -- --nocapture

# Snapshot incremental:
cargo test -p shipote-core --lib save_snapshot_skips
```

> Algunos tests del pipeline (fan-in/fan-out) son tokio-async y necesitan `--test-threads=1` cuando todos los tests corren juntos. La suite por crate ya funciona bien.

## Smoke E2E manual

Demo end-to-end de los features principales:

```sh
# 1. Limpiar estado previo
rm -f $HOME/.local/state/shipote/state.json
pkill -f shipote-daemon

# 2. Arrancar daemon (en background o terminal aparte)
SHIPOTE_LOG=info ./target/debug/shipote-daemon &
sleep 0.3

# 3. Crear workspace
cat > /tmp/ws.toml <<'EOF'
label = "demo"
on_exit = "reap"
EOF
WS=$(./target/debug/shipote workspace create /tmp/ws.toml)
echo "workspace: $WS"

# 4. Health check
./target/debug/shipote health

# 5. Run comando con log capture
./target/debug/shipote run -w $WS /bin/echo -- "hello shipote"
sleep 0.3
CMD=$(./target/debug/shipote commands $WS | head -1 | awk '{print $1}')
./target/debug/shipote logs $WS $CMD

# 6. Pipeline con tap (data plane real)
cat > /tmp/pipe.toml <<EOF
label = "demo-pipe"
workspace = "$WS"
discern = { sample_bytes = 4096, enrich_producer = true }

[[nodes]]
label = "p1"
payload.Native = { exec = "/bin/echo", argv = ['{"hello": 1}'], envp = [] }

[[nodes]]
label = "p2"
payload.Native = { exec = "/bin/cat", argv = [], envp = [] }

[[edges]]
from = 0
from_output = "stdout"
to = 1
to_input = "stdin"
EOF
./target/debug/shipote pipeline run /tmp/pipe.toml --tap

# 7. Flow throughput
./target/debug/shipote flow throughput

# 8. Stats con CPU%
./target/debug/shipote workspace stats $WS

# 9. SIGTERM (drain + snapshot)
pkill -TERM -f shipote-daemon
```

## Debug del daemon

### Logs del daemon en archivo
```sh
SHIPOTE_LOG=info ./target/debug/shipote-daemon 2>&1 | tee /tmp/shipote-daemon.log
```

### Filtrar audit log
```sh
SHIPOTE_LOG=warn,audit=info ./target/debug/shipote-daemon 2>&1 | grep audit
```

### Verificar caps runtime
```sh
shipote caps
```
Te dice si `user_ns` está bloqueado por sysctl/LSM y si tu cgroup está delegado.

### Inspeccionar el snapshot
```sh
cat $HOME/.local/state/shipote/state.json | jq .
```

JSON con: version, timestamp_ms, workspaces, saved_pipelines, live_pipelines, stats_history persistida.

### Debugging FDs
```sh
ls -la /proc/$(pgrep -x shipote-daemon)/fd | head -20
```
Si ves muchos `pipe:[N]` o `socket:[N]` huérfanos, hay leak en algún spawn.

## Arquitectura del repositorio

```
crates/
├── apps/
│   ├── shipote-daemon/     ← binario long-running
│   ├── shipote-cli/        ← binario `shipote`
│   └── shipote-shell/      ← GUI GPUI
├── modules/shipote/
│   ├── shipote-card/       ← tipos WorkspaceSpec/PipelineSpec/...
│   ├── shipote-protocol/   ← wire postcard
│   ├── shipote-discern/    ← MagicBytes/Json/Toml/Card/Utf8
│   ├── shipote-core/       ← WorkspaceManager + pipeline + flow_channel + ...
│   ├── README.md           ← entry point (este archivo es vecino)
│   └── docs/
│       ├── ARCHITECTURE.md
│       ├── CLI.md
│       ├── RECIPES.md
│       └── DEVELOPMENT.md  ← estás acá
└── shared/
    └── ente-incarnate/     ← extraído de ente-soma; reusable por shipote y otros
```

## Memoria del proyecto

Toda la historia de fases (F a R) está documentada en:
```
/home/sergio/.claude/projects/-home-sergio-brahman/memory/project_shipote.md
```

Esa memoria persiste entre conversaciones con Claude Code. Si arrancás una sesión nueva, Claude la consulta automáticamente.

## Issues conocidos

### El remote `origin` está mal configurado
`https:/sergio:...` con un solo `/`. Para subir los 10 commits locales:
```sh
git remote set-url origin https://sergio:<password>@gitea.gioser.net/sergio/brahman
git push origin main
```

### El daemon no se conecta al broker
Si el Init (`brahman-init.sock`) no está corriendo, el sidecar loguea `no conectado` y el daemon sigue standalone. Esto es **diseñado** (graceful degradation), no un bug. Los announcements de edge-card al broker tampoco llegan en este modo.

### Pipeline restart pierde ULIDs
Cada relaunch genera un pipeline_id nuevo. Trackers externos que dependen del ULID se rompen. Workaround: usar el `label` del pipeline (estable entre restarts).

### `tokio::test` + `waitpid` síncrono
Algunos tests del pipeline necesitan `--test-threads=1` porque el waitpid síncrono dentro de un `current_thread` runtime bloquea el reactor antes de que las tareas spawneadas corran.