Home AI platform

Problem

I wanted to share the local models on my Mac with a few invited people, without opening my home network to the internet. Chat apps like this usually sit in front of a cloud API. Running one from a single machine in a house changes what has to be true:

• nobody reaches a model engine, the database or an admin tool directly;
• one heavy user can't take the machine away from everyone else;
• a private document can't end up in a request to a cloud model;
• every request is accounted for, whether it succeeds or not.

Constraints

• One machine. A Mac M4 with 128 GB of unified memory runs Ollama and LM Studio. Two answers generated at once make both slower and can push the box into swap.
• Nothing exposed but the API. No router port forwarding. The database, Redis and the model engines listen on loopback only.
• Private by default. Uploaded documents stay on local disk, and cloud models stay off unless the plan and the workspace both allow them.
• A small, settled stack. TypeScript on Node, PostgreSQL with pgvector, Redis and Next.js, with an 80% test coverage floor.

Design

export function assertWorkspaceAllowsModel(workspace: Workspace, model: ResolvedModel): void {
  if (model.privacyClass !== "cloud") return;
  if (!workspace.isPrivate) return;
  if (workspace.allowCloudOverride) return;

  throw new AppError(
    "CLOUD_BLOCKED_PRIVATE_WORKSPACE",
    "This workspace is private, so it cannot use a cloud model. Choose a local model, or allow cloud models for this workspace.",
  );
}

The top row is the way in. The browser talks only to the backend, through a Cloudflare Tunnel that routes /api and /status and answers everything else with 404 at the edge. The backend verifies the Supabase sign-in, checks the invite, the 18+ confirmation and the current terms, then resolves the model and applies the privacy rule.

The middle row shares the machine. A per-user rate limit and a daily plan budget come first. The question is then matched against the user's documents in pgvector, and the request waits for a slot on the engine. One interface covers Ollama, LM Studio, Claude and Qwen, with a timeout and an output cap on every call.

The bottom row stays on the Mac: the engines on loopback, document ingestion to local disk, status checks and encrypted backups, and a usage row for every request.

Key decisions

• One gateway for every model call. The browser never calls an engine. A scan, run after a build, fails if a server-only variable, a server secret or an internal address appears in the frontend source or bundle.
• Admin closed at the edge, not only behind a login. The first real test of the tunnel found admin routes reaching the app, with only the role check in front of them. Rather than depend on an access policy staying configured, the tunnel now answers admin with 404, and admin runs only on the Mac itself.
• The privacy rule runs before retrieval. A private workspace refuses a cloud model before any document is read, so private text never goes into a cloud-bound request, even briefly. Opting in is per workspace, and the setting says up front that it exposes the documents already there.
• Slots are leases, not counters. Each engine slot is a lease in Redis that expires, so a process that dies mid-answer can't leave the engine marked busy for good. Waiting requests are ordered by priority, then arrival, and a streaming client is told its place in line.
• One usage row per request, whatever happens. Success, error, timeout and a client that walks away each write exactly one row. Refusals are recorded but don't use up the budget, and work already done is still charged.
• The server's history is the record. For a saved chat, only the user's new message comes from the request. Everything earlier is read from the database, so a client can't rewrite a conversation by replaying an edited one.
• Built with an AI coding agent under a written contract. The contract puts security and privacy rules first, fixes the stack and keeps work inside the current phase. Each of the 13 phases has develop and test checklists, and blocked items say what blocks them.

Evidence

The code, checked on a clean copy in October 2026:

• 588 tests pass. 510 cover the API and run against a real PostgreSQL with pgvector and a real Redis; 78 cover the web app. API coverage is 95% of statements and 87% of branches, against an 80% floor.
• The type check, a production build and the frontend exposure scan are clean.

Against the real services, in September 2026:

• Sign-in. A token from the real Supabase project was accepted through the project's published keys.
• Documents. A file containing an invented fact was uploaded and embedded locally in 1.5 seconds. The model answered with that fact and cited the right passage, which it couldn't have known from training.
• Failure handling. With Redis stopped, the status page still answered and reported the fault, and the watchdog alerted after three failed checks. An encrypted backup was restored into a scratch database.
• Exposure. The exposure probe passes 6 of 6 on the live tunnel. While wiring it up, I found and stopped an Ollama container from another project that was serving its API to the whole home network.

Still open before the first invite: a legal review of the privacy policy and terms, a NAS backup with a restore test, and sign-off on the Mac's security baseline.

What I'd change

• Run ingestion as a worker. Today it runs inside the API process, so a restart mid-upload leaves the document stuck. A worker should claim jobs, pick up stale ones on start, and wait for an engine slot like chat does.
• Share one queue across both engines. Ollama and LM Studio have a slot each, so two local answers can still run at once on the same memory. Cloud calls should get a limit of their own instead.
• Skip retrieval when there's nothing to search. Every chat embeds the question today, so plain chat fails if the embedding model is down.
• Enforce the retention windows. The privacy page says deleted chats are purged after 30 days and usage records after 400. Nothing purges them on a schedule yet, so that needs a job and a test before anyone is invited.
• Bring the docs up to date with the tunnel as deployed. The security doc still describes the earlier admin setup.