faro/PLAN.md

# Faro: Docker Engine Management UI

> Living project plan. Add ideas to the backlog, record architectural choices in
> the decision log, and promote accepted work into a milestone.

## 1. Product vision

Faro is a small, self-hosted control plane for managing one or more Docker
Engines. It provides a clear web UI, an automation-friendly API, and CLI tools,
with special attention to easy deployment, reliable backups, and operational
visibility.

The initial product should be useful to an individual operator or a small team.
It is not intended to replace a full container orchestrator such as Kubernetes.

## 2. Product principles

- **Easy to deploy:** a single server binary and container image, with sensible
  defaults and a documented Docker Compose installation.
- **API first:** every management operation available in the UI is exposed by a
  stable, documented API and can be automated.
- **Safe by default:** destructive actions require explicit confirmation;
  credentials and Docker access are narrowly scoped.
- **Recoverable:** configuration, metadata, and named volumes can be backed up,
  verified, and restored through repeatable workflows.
- **Visible:** users can quickly understand engine health, resource usage,
  container state, recent events, and job outcomes.
- **Small operational footprint:** avoid required external services for the
  single-node installation.

## 3. Users and core journeys

### Primary users

- A self-hosting user managing one or more Docker hosts.
- A small operations team that needs shared visibility and controlled access.
- An automation author using the REST API or CLI.

### Core journeys

1. Deploy Faro with Docker Compose and complete first-run setup.
2. Register a local or remote Docker Engine and verify connectivity.
3. View engine, container, image, network, and volume status.
4. Start, stop, restart, inspect, and view logs for a container.
5. Create and restore a backup of Faro state and selected Docker volumes.
6. Diagnose failures through events, job history, logs, and health information.
7. Perform the same common operations using the CLI or API.

## 4. Initial scope

### MVP

- Single Faro server instance.
- Local Docker socket and remote Docker Engine connections over TLS/SSH.
- Engine list and health summary.
- Read-only inventory for containers, images, volumes, and networks.
- Container lifecycle operations and streaming logs.
- Engine event stream and basic CPU, memory, disk, and container metrics.
- Scheduled and on-demand backups of Faro state and selected named volumes.
- Local filesystem backup target, with a provider interface for object storage.
- Backup retention, integrity verification, and guided restore.
- REST API with an OpenAPI specification.
- CLI for engines, containers, backups, health, and authentication.
- Web UI built with Vue 3 and Vuetify0.
- Web UI page for listing containers from the Docker Engine API.
- Initial administrator account, API tokens, and a basic audit log.
- Docker image, Compose file, health check, and upgrade documentation.

### Later

- Multi-user roles and granular permissions.
- S3-compatible and other remote backup targets.
- Compose application/stack management.
- Image updates, registry management, and vulnerability information.
- Notifications and alert routing.
- High availability and multiple Faro server replicas.
- Agent-based engine connectivity for restricted networks.
- Swarm-specific management.

### Explicitly out of scope for the first release

- Kubernetes or non-Docker runtimes.
- A general-purpose terminal in the browser.
- Full orchestration or scheduling across engines.
- Building a container registry.

## 5. Proposed architecture

```text
Browser (Vue + Vuetify)        CLI
          |                     |
          +------ HTTPS API ----+
                     |
              Faro server (Go)
         +-----------+------------+
         |           |            |
      SQLite      Job runner   Event/metric cache
                     |
              Docker client layer
           +---------+---------+
           |                   |
     Local socket       Remote engines
                         (TLS or SSH)
```

### Backend

- Go service organized as a modular monolith.
- Versioned REST endpoints under `/api/v1`.
- OpenAPI is the API contract and drives documentation/client generation where
  practical.
- Docker Engine SDK behind an internal interface so connectivity and tests can
  use alternate implementations.
- SQLite for the default installation; keep persistence boundaries clean enough
  to support PostgreSQL later if demand justifies it.
- Persistent job records for backups and other long-running operations.
- Server-Sent Events (SSE) for logs, events, job progress, and live status;
  introduce WebSockets only if bidirectional streaming becomes necessary.
- Structured JSON logs and Prometheus-format application metrics.
- Embed the production frontend in the Go binary for a simple single-artifact
  deployment; allow separate frontend/backend processes in development.

### Frontend

- Vue 3, TypeScript, Vite, Vuetify 3, Vue Router, and Pinia.
- Generated or typed API client based on OpenAPI.
- Primary views: overview, engines, engine detail, containers, container detail,
  storage, backups, jobs, audit log, and settings.
- Responsive layout, keyboard-accessible actions, clear empty/error/loading
  states, and a dark theme.

### CLI

- Go CLI shipped as a separate `faroctl` binary.
- Reuse generated API types/client rather than connecting directly to Docker.
- Human-readable tables by default; `--output json` for automation.
- Configuration profiles for server URL and credentials.
- Stable exit codes, non-interactive flags, and shell completions.

## 6. Key domain areas

### Engine connections

- Store engine name, endpoint type, labels, connection state, and last check.
- Support local Unix socket, TLS-protected TCP, and SSH connection strategies.
- Encrypt stored credentials at rest with a user-supplied master key.
- Never expose Docker credentials or raw private keys through API responses.
- Use timeouts, reconnect backoff, and explicit capability detection.

### Backups and restores

- Treat backups as durable jobs with progress, logs, status, and cancellation.
- Back up Faro's database/configuration separately from Docker volume data.
- Quiesce supported workloads with optional pre/post hooks; clearly label crash-
  consistent backups when a workload is not paused.
- Stream volume archives without staging the full archive in memory.
- Produce a manifest containing versions, contents, timestamps, checksums, and
  source engine identity.
- Verify checksums after creation and before restore.
- Apply retention by count and/or age, with a dry-run preview.
- Restore to an alternate volume name by default; overwriting an existing volume
  requires explicit confirmation.
- Document recovery when the Faro service itself is unavailable.

### Visibility

- Engine availability and Docker version.
- Container state, health check, restart count, uptime, and resource usage.
- Host CPU, memory, filesystem, and Docker storage usage.
- Recent Docker events and Faro audit events.
- Backup/job duration, result, bytes processed, and last successful run.
- Correlation/request IDs across API errors, jobs, and structured logs.

### Security

- Document that access to the Docker socket is effectively host-level control.
- Run the server as a non-root user where the connection method permits it.
- Password hashing with a modern memory-hard algorithm; short-lived sessions and
  revocable API tokens.
- CSRF protection for cookie-authenticated browser requests, strict CORS, secure
  headers, request size limits, and rate limiting on authentication endpoints.
- Audit authentication, engine changes, lifecycle actions, backup restores, and
  token changes without logging secrets.
- Pin minimal container base images and publish an SBOM and checksums for
  releases.

## 7. API outline

The exact resources will be defined in OpenAPI before implementation. Candidate
resource groups:

- `/api/v1/session`, `/api/v1/tokens`
- `/api/v1/engines`
- `/api/v1/engines/{engineId}/containers`
- `/api/v1/engines/{engineId}/images`
- `/api/v1/engines/{engineId}/volumes`
- `/api/v1/engines/{engineId}/networks`
- `/api/v1/backup-targets`, `/api/v1/backup-policies`, `/api/v1/backups`
- `/api/v1/jobs`, `/api/v1/events`, `/api/v1/audit-events`
- `/api/v1/health`, `/api/v1/version`, `/metrics`

API conventions to decide early:

- Resource IDs, pagination, filtering, sorting, and timestamp format.
- Standard error envelope with a stable machine-readable error code.
- Idempotency behavior for mutating and long-running requests.
- Optimistic concurrency or preconditions for configuration changes.
- SSE event envelope, resume behavior, and connection limits.
- Compatibility and deprecation policy for `/api/v1`.

## 8. Delivery milestones

Each milestone should end with a runnable increment, documentation, and tests.

### M0 — Validate and scaffold

- [ ] Confirm target users and the exact MVP scope.
- [ ] Resolve the open decisions listed below.
- [ ] Write threat model for Docker access, credentials, and restores.
- [ ] Create repository layout for server, CLI, web app, API spec, and docs.
- [ ] Establish formatting, linting, unit tests, and CI.
- [ ] Add a development Compose environment and sample Docker Engine.
- [ ] Add a minimal end-to-end smoke test.

**Exit:** one command starts the development stack; CI builds and tests all
components.

### M1 — Engine connectivity and read-only inventory

- [ ] Implement configuration, database migrations, health, and version APIs.
- [ ] Add local socket engine registration and connectivity checks.
- [ ] Add remote TLS and/or SSH connectivity based on the M0 decision.
- [ ] Implement read-only container, image, volume, and network APIs.
- [ ] Build overview, engine list, and engine detail UI.
- [ ] Add equivalent `faroctl engine` and inventory commands.
- [ ] Add integration tests against supported Docker versions.

**Exit:** a user can register an engine and inspect its resources through the
web UI, CLI, and API.

### M2 — Container operations and live visibility

- [ ] Add start, stop, restart, and inspect operations with audit records.
- [ ] Add log streaming with bounded history and redaction guidance.
- [ ] Consume Docker events and expose an SSE stream.
- [ ] Collect and display basic engine/container resource metrics.
- [ ] Add operation confirmation, actionable errors, and reconnect states.
- [ ] Add permission and failure-path tests.

**Exit:** a user can safely operate and troubleshoot containers in near real
time.

### M3 — Backup and restore

- [ ] Implement persistent background jobs and progress streaming.
- [ ] Implement local filesystem target and backup manifests.
- [ ] Add named-volume backup, checksums, retention, and verification.
- [ ] Add scheduled backup policies with timezone handling.
- [ ] Add guided restore, collision protection, and restore validation.
- [ ] Back up and restore Faro's own state.
- [ ] Run documented disaster-recovery tests on clean infrastructure.

**Exit:** scheduled backups and tested restores work from the UI, CLI, and API.

### M4 — Authentication, hardening, and release

- [ ] Implement first-run admin setup, sessions, and API tokens.
- [ ] Encrypt stored engine credentials and define key rotation/recovery.
- [ ] Complete audit log UI/API and security controls.
- [ ] Add retention/cleanup for events, metrics, jobs, and audit data.
- [ ] Load-test event streams, logs, and representative engine counts.
- [ ] Build a minimal production image, Compose example, and upgrade flow.
- [ ] Publish operator, backup recovery, API, and CLI documentation.
- [ ] Add release automation, SBOM, checksums, and signed artifacts.

**Exit:** the first supported release can be installed, upgraded, monitored, and
recovered using published documentation.

## 9. Testing strategy

- Go unit tests for domain logic, validation, retention, and authorization.
- API contract tests against the OpenAPI schema.
- Integration tests using disposable Docker Engines and temporary volumes.
- Frontend component tests for important state and permission variants.
- Playwright end-to-end tests for the web UI, including setup, engine
  registration, container operations, backup, and restore.
- Restore tests must compare checksums and application-level sample data.
- Compatibility matrix for supported Docker Engine and browser versions.
- Security checks for dependencies, container images, secrets, and common web
  vulnerabilities.

## 10. Deployment and operations

Start with three supported modes:

1. Docker Compose with Faro connecting to the local Docker socket.
2. Docker Compose with Faro managing remote engines over TLS/SSH.
3. Standalone binaries for the server and CLI.

Required operational features:

- Environment variables and a configuration file, with documented precedence.
- Persistent data directory and an explicit master-key mechanism.
- Liveness and readiness endpoints.
- Graceful shutdown for HTTP streams and active jobs.
- Schema migration and downgrade/rollback guidance.
- Configurable structured logs and Prometheus metrics.
- Versioned release notes with breaking-change and backup warnings.

## 11. Open decisions

Record the result and rationale in the decision log.

- [ ] **D-001:** Confirm Vue 3 + Vuetify 3 (assuming “Vuetify0” was a typo).
- [ ] **D-002:** Choose the Go HTTP router and OpenAPI generation approach.
- [ ] **D-003:** Choose database access/migration libraries.
- [ ] **D-004:** Decide whether the MVP includes both TLS and SSH remote engines.
- [ ] **D-005:** Define the authentication bootstrap and master-key experience.
- [ ] **D-006:** Decide whether backups run directly through the Docker API or via
  a short-lived helper container.
- [ ] **D-007:** Define supported Docker Engine versions and maximum tested scale.
- [ ] **D-008:** Define license and release/distribution channels.
- [ ] **D-009:** Choose a metrics retention model: live-only, local time series,
  or integration with an external metrics system.

## 12. Success measures

Initial targets; adjust after a prototype and user feedback.

- A new user can deploy Faro and connect the local engine in under 10 minutes.
- Common UI operations are also possible through documented CLI/API commands.
- Engine disconnects and failed jobs are visible with an actionable reason.
- A backup can be verified and restored on a clean host using only documented
  steps.
- The idle server has a small, measured CPU and memory footprint.
- Upgrades preserve configuration and include an explicit recovery path.

## 13. Risks and mitigations

| Risk | Mitigation |
| --- | --- |
| Docker access permits host compromise | Strong warnings, narrow connection options, authentication, audit, and deployment hardening |
| Volume backups are inconsistent | Pre/post hooks, optional pause, manifests, verification, and clearly stated consistency level |
| Remote connection setup is difficult | Connection wizard, validation endpoint, actionable diagnostics, TLS/SSH examples |
| Metrics storage increases product complexity | Start with bounded retention and an export endpoint; defer a full time-series system |
| UI, CLI, and API behavior diverge | API-first implementation and a shared generated client |
| Restore destroys existing data | Restore to a new name by default, preview changes, require explicit overwrite confirmation |

## 14. Decision log

Add one row whenever an open decision is resolved.

| ID | Date | Decision | Rationale | Status |
| --- | --- | --- | --- | --- |
| D-001 | TBD | Vue/Vuetify version | Awaiting confirmation | Proposed |

## 15. Backlog and idea inbox

Add unrefined ideas here without disrupting the milestones. Give each accepted
item an owner and promote it to a milestone when it is ready.

| ID | Idea | Why it matters | Priority | Owner | Status |
| --- | --- | --- | --- | --- | --- |
| I-001 | S3-compatible backup storage | Keeps backups off-host | Later | — | Idea |
| I-002 | Compose stack management | Groups related containers into applications | Later | — | Idea |
| I-003 | Notifications for engine/backup failures | Reduces time to detection | Later | — | Idea |

## 16. Change log

| Date | Change |
| --- | --- |
| 2026-08-06 | Initial project plan created. |