0
Kai Detmers
← Project index
Native knowledge system · iOSv0.3 · incubator

Synthesis— voice in, project memory out.

Synthesis treats speech as input to a knowledge pipeline rather than a transcript archive: capture a thought, extract typed blocks, route them by meaning and update the relevant project's living synthesis.

Stack

ClientSwiftUI · SwiftData
CaptureApple Speech · AVFoundation
System entryAppIntents · ActivityKit · WidgetKit
Knowledgetyped extraction · embeddings · rerank · synthesis
GatewayCloudflare Worker · App Attest
StateInbox · projects · recordings · structured blocks

How it evolved

Reconstructed from the zero-friction capture spec, commits and current source. The chronology is simplified into product milestones.

01
Optimize for capture before organization.

The first job is getting a thought out of the user's head with almost no navigation. Voice, system shortcuts and a small acquisition surface matter more than a sophisticated project browser at this stage.

02
Make failure impossible to ignore.

Failed-recording strips, debug views, orphan-audio cleanup and explicit capture state appeared because invisible loss is catastrophic in a memory product. Reliability became UI, not just logging.

03
Turn transcripts into typed knowledge.

Tasks, decisions, questions and ideas became durable objects. Raw speech remains provenance; the rest of the system operates on structured blocks.

04
Add semantic routing with an uncertainty buffer.

Embeddings retrieve likely projects and an LLM reranks ambiguous cases. Low-confidence results go to Inbox instead of silently contaminating the wrong project.

05
Move from filing to project intelligence.

ProjectThinker / ProjectConnector work introduced project suggestions and automatic synthesis updates. The product starts answering “what changed?” rather than merely “where was this note saved?”

06
Harden the edge before distribution.

Backend URL validation, App Attest and safer HTTP provider handling moved the architecture toward an app that can use paid AI infrastructure without embedding reusable credentials.

UI rationale

Capture is a system actionApp Intent, widget and Live Activity surfaces reduce the need to open a navigation hierarchy before speaking. The best capture UI is often outside the main app.
Inbox means “the model is unsure”The Inbox is not a generic dumping ground. It is an explicit uncertainty buffer for items that should not be auto-routed without review.
Failures are persistent objectsA failed recording must remain visible until resolved or dismissed. Silent disappearance would teach users not to trust the product with important thoughts.
Project view is synthesis, not chronologyThe durable interface should foreground current decisions, open tasks and emerging context. A raw transcript timeline is useful evidence but a weak project model.
Suggestions remain reviewableProject updates should appear as understandable changes rather than hidden autonomous mutations. The interface keeps user agency at the boundary where AI changes persistent state.

Pipeline

Capture→Speech/audio becomes raw text with recoverable recording state.
Extract→Model converts transcript into typed task / decision / question / idea blocks.
Route→Embedding retrieval narrows candidates; rerank resolves meaning; uncertainty goes to Inbox.
Synthesize→Accepted blocks update a living project summary and proposed next changes.

Current evidence

0.3current documented product line
22/22tests reported passing in current repository status
Incubatordistribution is not presented as live until the product loop earns it

I did not find a clean product screenshot artifact in the repository that is suitable for public use. The case therefore shows the actual interaction and failure model instead of using a fabricated app-store-style mock.

Related