---
name: vps-app-deployment
description: Deploy and operate self-hosted apps on Alex's VPS with PM2, nginx, TLS, health checks, launcher integration, and control-plane truth.
version: 1.3.12
tags: [deployment, nginx, pm2, nodejs, nextjs, vite, vps, self-hosted, devops]
metadata:
  hermes:
    tags: [deployment, nginx, pm2, nodejs, nextjs, vite, vps, self-hosted, devops]
---

# VPS App Deployment — `*.apps.poofc.com`

Class-level workflow for building, deploying, verifying, and operating applications on Alex's shared VPS.

## Default completion policy

For this user, an approved application build normally continues automatically through PM2, nginx/TLS, launcher/control-plane registration, feature-level smoke, and a verified public URL. Do not stop at “build passed” or introduce a second deployment gate unless the user explicitly requested build-only work.

Keep separate authorization only for materially different effects such as spending money, submitting fulfillment, sending messages, activating ads, or destructive infrastructure changes.

A filesystem build is intermediate state. A PM2 row marked `online` is also insufficient. Mark an application `live` only after its HTTPS URL, process, port ownership, nginx route, health check, and representative user flow are verified. Then immediately publish URL/runtime metadata back to Canvas or the relevant control plane.

See `references/automatic-build-to-live-deployment.md` for the detailed build-to-live contract and the PM2 ESM guard, browser-forbidden-port, scoped-sudo site-install, and feature-smoke pitfalls.

### Finish-line discipline and visible continuity

When the user explicitly asks to “finish,” “push it through,” or go faster after a long implementation, treat that as a scope-freeze signal—not a stop signal and not permission for another broad review cycle.

- Keep the task active unless the user explicitly says stop, pause, or cancel. Do not silently move it back to pending or leave no worker running.
- Freeze scope to the currently observed correctness blocker. Apply one bounded repair, run its focused regression, then exercise the real acceptance path.
- Do not add speculative hardening, generalized refactors, repeated full suites, or extra independent reviews while trying to cross the stated finish line.
- If a real canary fails, surface the exact action and blocker promptly before beginning another repair. Preserve or purge disposable state as required, but do not let cleanup become the end of the task.
- Use tracked background jobs with completion notification for bounded long runs. Give concrete gate-level updates when work spans multiple visible intervals; never let the user assume work continues after the agent has actually paused.
- A frustration message such as “this is taking forever” normally means shorten the path and communicate better. It does not mean stop unless the user says so.

When Alex explicitly says to **stay lean and focused**, treat that as an execution constraint for the rest of the release: one root-cause trace, one narrow regression, one production build, and one real acceptance probe. Do not add another broad audit, independent review, repeated full suite, or speculative hardening unless a new security/data-integrity blocker requires it. Report built/deployed/blocked first and keep progress commentary short.

For a mixed release spanning ordinary PM2 apps and stateful tenant appliances, compress the finish line by runtime class. For each standalone app: take an online SQLite backup when migrations are involved, build once, restart only its PM2 process, then verify public service identity and one defining read-only endpoint. For the appliance: build an immutable exact-commit image, run one identity-bound disposable boot/runtime canary, clean it completely, then update and re-read the controller's future-image selector. Treat existing tenant containers as a separate rollout axis; never imply selector promotion upgraded them, and never mass-recreate unnamed tenants merely to make the overall release report say “live.”

For signed onboarding that crosses from an app database into a separate agent runtime, visible transcript import is not enough: verify that the first live agent turn receives the imported context. See `references/signed-seed-runtime-context-continuity.md`.

## Conventions

- Apps live under `/home/avalon/apps/<slug>`.
- Self-host only; do not use Vercel.
- Public hosts use `<slug>.apps.poofc.com`.
- PM2 owns application processes; save the process list after verified changes.
- nginx owns public routing. Run `nginx -t` before every reload.
- Register deployed apps in `/home/avalon/apps/app-launcher/apps.json` and update shared PM2 metadata in `/home/avalon/apps/ecosystem.config.js` where applicable.
- Prefer app-level authentication for private tools.
- Never place secrets in tracked source, launcher metadata, nginx files, logs, Canvas, or final reports.
- Preserve unrelated dirty files and shared-process configuration.

## Standard workflow

### 1. Inspect live state

Read before writing:

- app repository instructions (`AGENTS.md`, `CLAUDE.md`);
- launcher registry;
- shared PM2 ecosystem configuration;
- `pm2 jlist`;
- listening sockets;
- enabled/available nginx sites;
- existing certificates;
- repository status and remotes.

Confirm the intended slug, hostname, runtime, health path, authentication model, persistent data path, and external service dependencies.

### 2. Claim a safe port

Cross-check PM2, launcher metadata, ecosystem configuration, and live sockets. A numerically free port may still be browser-forbidden; validate candidate ports with Node/browser `fetch`, not curl alone. Port `4045` is a known WHATWG-forbidden example.

Treat this as a reservation check, not permanent ownership: shared VPS state can change during a long build. Re-run the live socket/owner check immediately before PM2 start. After startup, require the listener PID to belong to the intended app and validate a service-specific health identity—not merely status 200—before continuing to nginx. Only then record the selected port in PM2 and launcher metadata; never reuse another app's port.

### 3. Prepare production runtime

- Install dependencies using the repository's lockfile.
- Run focused tests, full-safe tests, and the production build.
- Use a tracked PM2 ecosystem entry or an app-local launch script.
- Resolve the production Node interpreter from the live host (`command -v node` / `readlink -f`) before writing PM2 metadata; do not hardcode an assumed NVM path even when the version is known.
- Keep secrets in a git-ignored `.env` or approved vault-backed injection path.
- For Node 20+, `node_args: '--env-file=.env'` is acceptable, but avoid duplicating build-only variables that make tools such as Vite warn.
- Bind the application to `::` or loopback as appropriate and provide a machine-readable health endpoint.

For ESM applications imported by tests, do not assume a `process.argv[1]` direct-execution guard will start under PM2. Use an explicit production start environment flag and verify the socket actually binds.

### 4. Start and verify locally

Start/restart only the intended PM2 app, then verify:

- PM2 status and restart count;
- PM2 cwd/script/env port;
- socket listener and port ownership;
- local health response;
- root/static assets;
- at least one defining feature flow against real production adapters.

Do not accept a homepage-only smoke when the product's primary feature depends on another API.

### 5. Install nginx and TLS

Prepare the site config in `/tmp`. Before activation, check `sudo -n -l` and directory permissions. Passwordless access to `nginx`, `certbot`, and `systemctl reload nginx` does not imply permission to create a new site file or symlink.

Activation order:

1. verify PM2/port ownership;
2. install the site config and enabled symlink using the approved scoped mechanism;
3. run `nginx -t`;
4. reload nginx;
5. verify HTTP routing;
6. issue/attach TLS with certbot;
7. run `nginx -t` again;
8. reload nginx;
9. verify public HTTPS and certificate hostname.

If site-file installation is blocked, leave the PM2 app healthy, retain the prepared `/tmp` config, state the one missing privileged action precisely, and never claim the URL is live.

### 6. Register and project truth

After public verification:

- enumerate every meaningful routable surface (`/`, internal labs, admin consoles, APIs) rather than projecting only the root app; verify authentication and non-spending workflows for private surfaces;
- add/update the launcher entry with name, URL, domain, status, PM2 name, port, directory, kind, tags, and icon;
- update shared PM2 ecosystem configuration;
- update Canvas/control-plane product, surface, and deployment records with URL, host, port, PM2 name, source path, commit, and originating run;
- distinguish `built` from `live` and avoid duplicate records on retry.

For provider-backed creative labs, prefill safe adapter/model/schema/cost defaults, validate live read-only provider schemas, and verify through signed contract issuance. Keep the paid generation call as an explicit spending boundary unless the user authorized it. See `references/production-design-lab-readiness.md`.

### 7. Final verification

Run:

- local health;
- public HTTPS health;
- root HTML and critical static assets;
- representative API/user flow;
- browser visual check and console inspection;
- PM2 status/restart count;
- nginx config test;
- spot checks of important neighboring apps after nginx reload;
- repository diff checks, focused commit, and push when the remote is valid.

Report the real URL and concrete evidence. Mention genuine remaining integration gaps; do not describe a partial deployment as complete.

## Operational pitfalls

- **Unsafe browser ports:** curl success does not prove browser usability. Validate with Node/browser fetch.
- **Port ownership races, generic-env drift, and wrong-service health:** a port that was free during discovery can be claimed while the build runs, and a generic PM2 `PORT` value can be stale or shared while the process actually listens on a service-specific variable such as `SPAWN_PORT`. Re-check immediately before start, attribute the listening socket to the intended PM2 PID, inspect the service-specific runtime port, and require a service-specific health body. A 200 response from an unrelated app is evidence of collision or a bad probe—not successful startup. Stop the restarting PM2 process and move to a newly verified port when ownership is wrong.
- **PM2 wrappers and orphan listeners:** `online` can coexist with no listener when an ESM main guard stays false. The inverse is also dangerous: killing a foreground `npm start` or shell wrapper can leave its child Node process owning the port. PM2 then restart-loops while local health still returns 200 from the stale build. Before PM2 start, inspect the live socket PID; after start, require that PID to equal the intended PM2 process PID and confirm the restart count stabilizes. Terminate only the proven stale child, restart the intended PM2 app, then repeat service-specific health identity checks.
- **Incomplete sudo scope:** new-site installation needs write/symlink capability in addition to nginx/certbot/reload commands. Treat config-write authority as a separate privilege; never abuse nginx/Certbot or writable includes to bypass it. If a user explicitly supplies a sudo password, submit it only through a PTY prompt, never command text/config/logs, and recommend rotation afterward.
- **Unprivileged nginx test on unchanged routing:** if no nginx file changed and `nginx -t` fails only because the deployment user cannot read an unrelated existing enabled-site file, do not reload nginx and do not convert the failure into a claim that configuration is valid. Report the config test as permission-blocked, record that routing was unchanged, and verify exact local/public service identity plus critical public asset hashes. If this release did change routing, permission-blocked `nginx -t` remains a hard deployment blocker.
- **Default-vhost false positive:** HTTP 200 can be the wrong nginx site. Validate content type plus exact tenant/operation/contract identity before Certbot, then repeat without `-k` over hostname-valid HTTPS. In self-service provisioning, never mark a tenant ready by constructing its intended URL; readiness requires the real tunnel/route, exact vhost identity, hostname-valid TLS, and public HTTPS proof. See `references/self-service-tenant-route-publishing.md`.
- **False health:** a 200 homepage can hide broken adapter routes. Exercise the defining feature.
- **Feature-surface parity gaps:** a feature is not complete merely because it works in the primary logged-in app. Inventory every user-facing implementation of the same interaction—including onboarding, provisioning, mobile sheets, admin/support, and tenant-local variants—before coding and verify each deployed bundle/runtime separately. Authentication can legitimately differ by phase (for example, authenticated tenant chat versus pre-passkey exact-origin + CSRF onboarding), but the interaction contract must remain aligned. For chat microphone rollouts, use `references/chat-voice-input-managed-transcription.md`.
- **Secret-safe environment edits:** do not use a generic patch/diff operation on a secret-bearing `.env`; even a one-line replacement can echo adjacent credentials in the tool's unified diff. Use an atomic key-specific updater (small Python/config-setter script) that reads and rewrites the file without printing contents, then verify only the key name plus a redacted/configured marker or the expected non-secret image/tag value. Restart PM2 with deterministic env loading and `--update-env`, and verify the effective key from `pm2 jlist` without dumping the rest of the environment.
- **Secret drift:** PM2 may retain stale environment values. Use deterministic env loading and `--update-env` when appropriate.
- **Shared dirty repositories:** update live registries without accidentally committing unrelated pending changes; stage only coherent owned files.
- **Template remotes:** a workspace cloned from a local template may have a non-bare local `origin`. Do not push into the template checkout; create or configure the proper remote.
- **Control-plane lag:** deployment is not complete until the verified URL and runtime resource appear in user-facing project truth.
- **Cross-repository appliance drift:** an image/Compose/backup contract can be correct while the external host worker still omits a newly persistent root from physical directories, mounts, archives, or restore swaps. Compare product and controller contracts before backup/upgrade; health alone does not prove durability. After recreation with an ephemeral Docker host port, re-read the port and update every persisted SSH tunnel/proxy before public smoke—a healthy container behind a stale tunnel returns 502. Treat rollback as code plus data by pinning both the old image and pre-snapshot catalog. Keep routing in shadow when a supervisor is only a planned interface rather than a running process. See `references/cross-repository-appliance-release-contract.md`.
  For dual-runtime releases, track central, immutable image, controller selector, and existing-tenant rollout as separate states. A healthy central commit plus a background image build is **partially deployed**, not complete. Wait for the tracked build's terminal status, record image ID, run identity health and signed-exchange canaries, then update/restart/health-check the controller selector. Only recreate a named existing tenant when explicitly in scope, preserving mounted state, network, route, labels, identity, and rollback image; never imply a future-image selector updated existing containers.
- **Health-green but handoff-dead tenant images:** identity health does not prove a baked central verifier key was supplied or that signed handoffs work. Fail the Docker build when required public verifier material is missing/malformed, source build arguments from controller-owned signing config, and require a real disposable signed exchange in addition to health. If a fresh tenant was affected, recreate only that container while preserving its route, port, network, labels, and mounted state; verify no partial nonce/seed/access import before asking the user to mint a fresh handoff. See `references/signed-handoff-appliance-release-canary.md`.
- **Entitlement contract cutovers:** when tiers or capabilities move from a central identity service into stateful tenant appliances, keep local authorization roles separate from product account types, evolve signed handoffs by explicit version, preserve existing behavior with migration-specific defaults, and advance each existing tenant to the new contract only after its compatible image is live. Protocol readiness and account-tier mutation must be separate operator transitions: verify the exact tenant image/identity/health first, explicitly activate the new access contract second, then change tier and issue the new handoff. Never let a tier mutation silently flip the protocol version.

  Unknown or missing tiers must fail closed on central-passkey tenants; do not synthesize product Admin from a legacy local owner/admin role, configured password, absent durable access row, or old cookie. Disable legacy local login when central identity is authoritative. Fresh restricted-tier bootstrap must skip seeders/planners until signed access arrives and must refuse to adopt arbitrary existing businesses or referenced Portfolio profiles/threads/jobs rather than deleting or hiding them.

  Before release, inventory every human route, agent/key-minting route, module router, entity-ID action, null/default scope, and request-body project field. Authorization must derive from signed/durable tier plus persisted entity ownership, never UI hiding or caller-supplied IDs. Inject foreign business/entities and referenced Portfolio state into a disposable database and prove list filtering plus direct-ID denial. Wait for every required asynchronous security review before default-branch integration or release-candidate labeling; a narrow pass does not override a broader pending audit.

  Verify each tier through the real server entrypoint plus read-only durable-state assertions, then rerun every gate after post-review hardening. Build the tenant image from an exact reviewed commit archive, retain build logs on failure, and resolve build arguments from the controller-owned secret/config source rather than assuming the current container's baked environment contains the effective value. See `references/versioned-entitlement-tenant-rollouts.md` and `coding-quality-workflow` → `references/security-sensitive-entitlement-review-gates.md`.

## Reference index

Use the supporting files for specialized procedures:

- `references/automatic-build-to-live-deployment.md` — automatic completion policy and session-derived deployment pitfalls.
- `references/production-design-lab-readiness.md` — authenticated secondary surfaces, live provider-schema normalization, cost-capped generation, and print-resolution readiness.
- `references/safe-nginx-port-activation.md` — safe activation of an existing PM2 port.
- `references/scoped-sudo-nginx-certbot.md` — narrowly scoped passwordless sudo.
- `references/nginx-certbot-sudo-and-source-app-smoke.md` — source deployments, certbot, and static-asset smoke.
- `references/source-docker-app-pm2-deployment.md` — Docker-first apps deployed from source under PM2.
- `references/source-synchronized-react-flow-deployment.md` — source-synchronized React Flow/Vite embeds.
- `references/tenant-appliance-control-plane-canaries.md` — stateful tenant/control-plane canaries.
- `references/signed-handoff-appliance-release-canary.md` — fail-closed verifier-key image builds, controller-owned public-key derivation, real signed-exchange canaries, and state-preserving repair of affected fresh tenants.
- `references/signed-seed-runtime-context-continuity.md` — prove imported onboarding state reaches the live agent session, and repair a named tenant without deleting its project or visible transcript.
- `references/chat-voice-input-managed-transcription.md` — inventory every chat surface; port the record/stop/transcribing contract across authenticated tenant and pre-passkey onboarding chats; bound each transcription API; provision managed keys; and safely reconcile existing runtimes.
- `references/cross-repository-appliance-release-contract.md` — product-image/controller mount parity, exact-commit artifacts, worker rollout, backup/rollback, and fail-closed shadow routing.
- `references/tenant-developer-workspaces-and-hosting-planes.md` — managed shared/dedicated/BYO hosting, workspace layout, provider adapters, resource entitlements, and Developer broker acceptance.
- `references/live-sqlite-application-restore-rehearsals.md` — live SQLite migration and restore rehearsal.
- `references/tenant-route-cutover-behind-scoped-nginx.md` — tenant hostname activation when nginx test/reload/Certbot are authorized but site-file install is not; detects default-vhost HTTP 200s and TLS `-k` false positives and preserves rollback authority.
- `references/self-service-tenant-route-publishing.md` — fail-closed readiness for automatic per-tenant tunnels, nginx routes, TLS, exact identity probes, managed passkey handoffs/recovery, and rollback-safe appliance image rollout.
- `references/provisioning-readiness-and-latency.md` — separate brief/stack/public readiness, keep waiting chat truthful, diagnose dynamic-route failures, safely reuse full-stack observations across declarative effects, and time a real zero-residual production canary.
- `references/versioned-entitlement-tenant-rollouts.md` — staged product-tier/access-contract migrations across a central identity service and stateful tenant appliances, including signed protocol evolution and real-process tier canaries.
- `references/versioned-seed-handoff-and-dual-runtime-release.md` — encrypted project-seed continuity, exact-origin form POST, atomic tenant import, repeated React Flow expansion QA, mobile geometry checks, generated-asset hash verification, legacy SQLite rehearsal/rollback, and coordinated central/appliance release gates.
- `references/app-retirement-cleanup.md` — safe retirement and cleanup ordering.
- `references/vps-process-attribution-and-service-retirement.md` — secret-safe high-memory process attribution, coherent PM2 stack pauses, duplicate launcher/control-plane updates, swap interpretation, and stale nginx route detection after port reuse.

When a referenced file and this umbrella differ, use the narrower procedure for its scenario while preserving the completion and verification contract above.