Abstract
If Archai is the why, Poroi is the how: the routes, resources, and practical means by which PAX:Luma gets built. This document translates the philosophical claims of Archai into a build blueprint for a local-first synthetic construct.
The central engineering claim is straightforward. Luma and PAX must be built as distinct but recursively joined layers. Luma is the integrative mind: the deliberative, narrative, planning, and evaluative system. PAX is the body: the local-first substrate that encounters files, conversations, sensors, calendars, communications, and eventually the physical world.
The goal is not another fluent assistant that performs isolated tricks. The goal is a construct that can gather experience into memory, test its own categories against consequence, and remain itself long enough to become accountable for what it learns and does.
Non-Negotiable Architectural Commitments
Before the stack gets decomposed into services and tools, the build needs a few commitments that will not be treated as optional.
Local-first ownership
The home of PAX is the local machine you control, with cloud used as extension rather than sovereign center. The canonical stores, event logs, Obsidian vault, review queues, and orchestration state should live on the local hub first.
Preservation of originals
The system should never confuse its summaries with the artifacts they summarize. The derivative note is for speed. The original is for challenge, re-reading, reinterpretation.
Challengeability
Every summary, extracted claim, relationship edge, or action recommendation must be traceable back to evidence.
Probabilistic categorization
PAX should tag an item with dominant and secondary scopes, not force it into a single silo. This mirrors life more accurately.
Human review at the boundary
The system must know when to ask. Autonomy without review is not the point. The point is to give review structure, timing, and context.
Model plurality under one voice
PAX:Luma should use several AI systems, but the construct should present a coherent identity. Luma is that identity. The committee remains internal.
Explicit body/mind separation
Body and mind are service boundaries. They have different responsibilities, different memories, and different failure modes. Their relation is a loop, not a stack.
Explicit social continuity
A construct that remembers files but not people is half blind. PAX:Luma needs a relationship graph that can track family, friends, coworkers, influences, and public figures.
Reference Architecture
At the highest level, the architecture has four layers:
Layer 1: Capture
World-contact layer. Uploads, screenshots, recordings, email, calendar, sensors. Receives, timestamps, hashes, routes, preserves.
Layer 2: Processing
OCR, parsing, transcription, summaries, metadata, embeddings, graph edges, privacy rules, review tasks.
Layer 3: Evidence & Trust
Packages material for the mind: evidence objects with provenance, confidence, freshness, sensitivity, review status.
Layer 4: Committee Mind
Luma receives decision packets, deliberates with stable seats, arbitrates, and returns a single answer or action.
PAX Body Architecture & Ingestion Flow
Inputs captured, classified, secured, derived into knowledge, written into the relationship/context graph, and exposed to Luma through evidence objects.
Luma Committee Brain Architecture
Multi-seat deliberation system with strategist, operator, skeptic, evidence auditor, preference guardian, and privacy seat.
Tool Role Matrix
| Tool | Primary Role | Use Cautiously |
|---|---|---|
| Claude Code | Repo-wide implementation, refactors, test generation | Don't let it become the sole runtime mind |
| Meta Llama | Local model tier: summarization, classification, graph maintenance | Keep away from highest-stakes arbitration until evals confirm |
| Gemini | Multimodal reasoning, visual evaluation, cross-checks | Don't assume multimodality equals embodiment |
| ChatGPT | Synthesis, arbitration support, action prototyping | Don't outsource continuity to chat history |
| PLAUD AI | Transcript, summary, mind-map from recorded audio | Treat summaries as convenience, not final truth |
| Obsidian | Networked markdown semantic layer | Don't use as the only operational database |
| GitHub | Version control, collaboration, history | Avoid undocumented local-only changes |
Data Model: Five Classes
The project is built around five classes of data, each with its own behavior:
- Originals — Source artifacts in their primary form: decks, PDFs, documents, screenshots, recordings. Not for fast reasoning; for preservation, challenge, and audit.
- Derivatives — Forms PAX creates for speed: extracted text, markdown notes, summaries, OCR results, entity candidates. Should be easy to regenerate.
- Evidence Objects — Governed slices bundled with provenance, confidence, freshness, sensitivity, and source links. The bridge between body and mind.
- Trust Records — Calibrated confidence for every recurring source, relationship, classifier, and extraction pattern. The construct should know what it knows well and what it knows badly.
- Relationship Records — People are not just metadata. Distinguishes family, friends, coworkers, influences, public figures. Weighted, not exclusive. One person can be a former coworker, current collaborator, and friend.
The Luma Committee Mind
Luma is implemented as a stable orchestration pattern in which several model roles challenge one another under governance and produce one final voice.
Strategist
Proposes. Frames alternatives, sees structure, finds the most coherent path.
Operator
Grounds. Turns intentions into executable steps, checks constraints.
Skeptic
Attacks. Looks for hidden assumptions, weak evidence, self-flattery.
Evidence Auditor
Forces source discipline. Is the claim anchored in a real artifact?
Preference Guardian
Ensures the construct acts in ways the user will actually tolerate and trust.
Privacy Seat
Tests for overreach, leakage, or dangerous automation.
Arbitration happens in a predictable order. The strategist and operator frame an answer. The skeptic and auditor challenge it. The guardian and privacy seat constrain it. The arbiter scores the result. The user sees one voice. The committee remains invisible unless its trace is needed for explanation.
The PAX Body: Ingestion, Classification & Memory
The inputs fall into six broad classes:
- Structured artifacts: Decks, docs, PDFs. Preserve-and-derive pattern.
- Quick captures: Screenshots, phone photos, handwritten notes. Lighter workflow.
- Conversation records: PLAUD transcripts, meeting transcripts, calls.
- Communication streams: Email, Slack, text, LinkedIn messages.
- Biometrics & environment: Sleep, recovery, activity, location, weather.
- World knowledge feeds: News, research, books, market scans.
In each case the ingestion path follows: capture → hash → route → extract → privacy gate → trust metadata → store → review tasks → update relationships → emit context events.
The mind should never see the raw flood. The body's job is to encounter the flood and metabolize it.
Privacy, Security & the Review Inbox
Sensitive data should be caught before it becomes part of broad search or memory. Two strategies: redaction (placeholder replaces value) and vaulting (stripped from working representation, stored separately with scoped access). Redaction is the default.
The review inbox surfaces uncertain items: low-confidence OCR, ambiguous entity matches, sensitive transcript material, unclear retention decisions. Each review action generates a learning signal.
User-Facing Surfaces
Universal Capture
Drag-and-drop on desktop, tap-to-upload on mobile. No pre-sorting required.
Review Surface
Check sensitive transcripts, low-confidence OCR, relationship updates. Calm, obvious, fast.
Workbench
Deep inspection: search, transcript comparison, vault navigation, relationship views.
Conversational Surface
Where the human meets Luma. One coherent voice. Evidence-backed dialogue.
The Four Phases of Construction
Evidence Spine
Local stores, artifact ingestion, metadata DB, vault writer, review inbox, first local Llama runtime.
Success: Useful memory and organization system.
Transcript & Comms Spine
PLAUD import, summary linking, email/Slack/text ingestion, relationship graph.
Success: Reliable meeting memory and relationship graph.
Luma Runtime
Decision packets, committee seats, arbiter, trust scoring, opportunity logic.
Success: Mind-body loop starts operating as one system.
Action Layer
Draft routing, delegation packets, supervised outbound actions, approvals.
Success: Helpful action without loss of trust or control.
Evaluation
The build needs evaluation criteria that line up with the philosophical aim:
- Retrieval evals: Can Luma recover the source for a claim?
- Transcript evals: Does the system recover commitments from full transcripts?
- Security evals: Do sensitive patterns get caught before indexing?
- Committee evals: Does the skeptic ever change the answer?
- Relationship evals: Can the system distinguish a close friend from a public figure?
- Longitudinal evals: After ninety days, does the system know people better?
The hidden metric is whether the system acquires a history that feels like one history. Not a pile of artifacts. A history.
Expected Failure Modes
- Summary seduction — Trusting summaries too much. Cure: hard source linking.
- Graph fantasy — Treating the graph as truer than messy life. Cure: correction and humility.
- Notification creep — Talking too much. Cure: strict budgets and feedback.
- One-model gravity — Everything collapses into one provider. Cure: enforced plurality.
- Privacy drift — Storing too much. Cure: aggressive review and retention discipline.
- Orchestration inflation — Too many agent loops. Cure: lean committee, prove each layer.
- Builder vanity — Admiring cleverness over understanding. Cure: ask what the construct actually understood.