Building a More Personal AI Stack: What I Worked on in August
This month’s AI work focused on continuity: semantic memory that survives failures, private model experiments that can do real concurrent work, and a long-context architecture designed not to throw history away.
Building a More Personal AI Stack
What I worked on in August: durable memory, local models, full context, and the standard of proving a system works before trusting it.
The interesting part of personal AI is not asking a model a clever question. It is building enough continuity around it that an answer can belong to an ongoing body of work.
That has been the thread running through my August AI work with Hermes. We have been shaping an assistant environment around three connected needs: memory that can carry forward, local inference that stays under our control, and enough context that earlier work is not casually discarded when a conversation grows.
There is a fourth piece as well: voice. The longer-term goal is an assistant that feels present in a conversation through timing, turn-taking, warmth, and useful technical capability—not merely a text pipeline with audio attached.
All of this has shared one operating rule: a green process or a successful response is only the beginning. The work counts when it survives a real test.
Memory that can recover
Semantic memory gives Hermes a way to connect durable facts, prior decisions, and useful context across sessions. It is how an assistant can retain the difference between a one-off detail and something that should shape future work.
That only matters when the retrieval path is dependable. This month, the Cognee service behind semantic recall became unavailable because its workload could no longer retrieve its image. The visible symptom was simple—recall had stopped working. The real work was tracing the failure through the deployment path, restoring it through GitOps with a verified immutable upstream image, and confirming the repaired service with a real graph-recall request.
That last verification was essential. A dashboard can look healthy while the thing you actually depend on remains broken. The repair was complete only when the deployment was healthy, the endpoint was ready, and Hermes could retrieve usable context again.
The broader lesson is that memory is more than storage. A pile of old messages does not make an assistant continuous. Useful memory needs to be curated, retrievable, and resilient enough to survive an ordinary infrastructure failure.
Two local GPUs, one private inference path
Local inference was the other major experiment. The aim was practical: take the GPUs already available, run models privately, and give Hermes a clean OpenAI-compatible route into that local capability.
The initial setup explored a larger model split across the host GPU and a desktop GPU. That approach made a bigger model fit, but it did not create true parallel work. One distributed model instance still has one decode path.
The better fit for the immediate goal was two independent local replicas: one worker on each GPU, reached through a single private gateway. That made parallelism possible, provided the routing layer could see active work accurately.
It turned out that the first load-balancing approach did not account for in-flight requests in the way this setup needed. The resulting behavior looked healthy from the outside while requests were still getting serialized onto one worker. We replaced that behavior with a shared least-connections layer and then tested the result directly.
Two authenticated tool-call requests were started at the same time. Each landed on a different GPU and completed successfully. That was the proof that mattered: the system was doing concurrent agent-shaped work, not just displaying two healthy status indicators.
An honest quality gate
The infrastructure test passed. The model test did not.
The local VibeForged Q3 experiment could serve requests and participate in the new routing setup. Under a fresh, long Hermes prompt, it also showed instruction-following failures and runaway generations. That made it unsuitable as the default model for a full Hermes agent session.
This is exactly why a meaningful quality gate exists. It is easy to declare victory when a model emits text, accepts a tool schema, or responds to a health check. An agent has a higher burden: it has to follow instructions reliably, remain bounded, and behave sensibly with the real context and tools it will carry in production.
So the experiment remains useful. It proved the private routing architecture and genuine concurrency. It did not earn the role of default assistant brain. Recording that difference is progress, because it keeps the next decision grounded in evidence instead of wishful thinking.
Infrastructure success and model suitability are separate claims. Both need their own proof.
The long-context decision
The constraint that emerged most clearly was context. A previous local setup had a context window too small for the full Hermes prompt and an evolving conversation. That makes an agent less useful precisely when work becomes complex enough to need continuity.
The tempting response is to shrink the prompt, compress history aggressively, or quietly drop the oldest material. That would make the hardware problem less visible while making the assistant less trustworthy.
We chose a different design constraint: do not solve the problem by deleting the past.
The next local-inference path is being designed around a private desktop deployment using the GLM-5.2/Colibri family. Its target is a much larger context window—up to one million tokens—with an isolated Hermes profile, one request queue, capacity preflight, and compression disabled for that profile. History is not meant to be shifted, truncated, reset, or silently offloaded elsewhere to make the system appear more capable than it is.
This work is still under validation. The design, capacity model, and runbook are in place; live admission tests remain part of the deployment gate. That status matters. It would be inaccurate to call it a completed replacement for the current agent model today.
Still, the direction is clear. Context is the working memory of an agent. Treating it as a disposable product limit would undermine the point of building a persistent assistant in the first place.
Voice as presence, not a button
Voice remains part of the larger vision. Voicebox was useful as a text-to-speech service, yet the target experience is more ambitious: natural prosody, well-timed pauses, interruption and turn-taking, conversational rhythm, and the feeling that technical competence and human warmth can coexist.
The previous GPU-backed voice service was deliberately stopped to free capacity for the local-inference experiments. That was a prioritization decision, not a claim that voice work is complete or abandoned. The voice layer is still a design goal for Hermes—one that should enrich the existing technical capabilities rather than replace them.
What this month actually changed
August did not produce a single magic model or a polished launch announcement. It produced something more durable:
- semantic recall was restored and verified after a real failure;
- Hermes gained a functioning private route into local inference;
- two local GPU workers proved they could handle concurrent tool-shaped requests;
- a promising model experiment was kept out of the default path because it did not meet the full-agent quality bar;
- the next architecture was shaped around preserving full context instead of hiding the constraint through truncation; and
- the voice direction became clearer: an assistant should be capable, continuous, and more natural to talk with over time.
That is the kind of progress I want from personal AI. It should be local where that matters, honest about what is still experimental, and designed to carry real work forward. The next stage is not about pretending the system is finished. It is about continuing to build the conditions under which it can become genuinely dependable.