TIMBOTIC / Field report

Building TIMBOT: knowledge is not memory

One limitation of TIMBOT's memory system became increasingly obvious as the assistant grew more capable.

Remembering a fact is not the same as understanding a project.

TIMBOT already has persistent memory using SQLite and FTS5. It can remember useful information, retrieve relevant facts and retain a limited history when those facts change.

That works reasonably well for short pieces of information.

But I have projects, technical notes, design decisions, experiments and ideas that don't fit neatly into a collection of remembered facts.

Some of that information changes frequently. Some of it is incomplete. Some of it contradicts earlier thinking.

And much of it is already written down.

I didn't want TIMBOT to build another copy of that information inside its own memory database.

I wanted it to work with the knowledge I already maintain.

That led me to SilverBullet.

Why SilverBullet

SilverBullet is an open-source, Markdown-based knowledge management system.

It combines a wiki-style editor with links, backlinks, queries, templates and scripting.

More importantly, the underlying information remains ordinary Markdown files.

That last detail matters to me.

I've never been particularly interested in locking years of technical work into a proprietary application database.

Markdown can be read without SilverBullet. It can be searched, versioned, backed up and processed by other software.

If I change my knowledge management system later, the documents remain useful.

SilverBullet provides the editing and organisational layer without demanding ownership of the information.

That makes it an interesting companion to TIMBOT.

Memory and knowledge have different jobs

The distinction is worth making explicit.

TIMBOT's existing memory system is designed for relatively small, durable facts.

For example:

• My main AI machine runs a particular GPU.
• A project uses a particular software stack.
• A device has a specific network address.
• I prefer one modelling approach over another.

Those facts are useful during conversation.

But consider a project involving a mechanical design.

I might have notes covering design requirements, material choices, sketches, manufacturing constraints, failed prototypes and decisions made over several months.

Reducing all of that to a handful of remembered facts would discard much of what makes the documentation valuable.

It would also introduce another problem.

Who decides which version is correct?

A saved memory might describe the original design, while a project note contains a later revision.

Both pieces of information may be historically accurate.

Only one may describe the current design.

The answer isn't to make the model more confident about choosing.

The answer is to preserve the source material and its context.

Memory helps TIMBOT maintain continuity. SilverBullet gives it somewhere to look for documented knowledge.

Those are complementary functions, not interchangeable ones.

Giving TIMBOT access to a knowledge base

The useful integration is straightforward in principle.

A question comes into TIMBOT.

If the question concerns something documented in SilverBullet, TIMBOT needs a controlled way to locate the relevant pages, retrieve their contents and use that information when constructing an answer.

The broad flow is:

User question → TIMBOT → SilverBullet retrieval → Relevant Markdown → Local model → Answer

The important part is what happens between the question and the model.

A language model should not need unrestricted access to an entire knowledge directory just to answer one question.

It should receive the material relevant to that request.

That means separating knowledge retrieval from language generation.

SilverBullet remains responsible for maintaining the documents.

TIMBOT remains responsible for selecting tools, validating requests and controlling what information reaches the model.

The language model interprets the retrieved material.

It does not become the authority over the underlying files.

Search is still not understanding

The previous TIMBOT article was about a failure in web research.

The assistant successfully searched for high-memory GPUs but couldn't reliably determine which retrieved information satisfied the actual question.

That problem doesn't disappear simply because the search results come from my own notes.

Suppose I ask:

What did I decide about the cooling system for the latest prototype?

A text search might find several pages mentioning cooling.

One could contain an early proposal.

Another might describe a rejected design.

A third could contain the final decision.

Retrieving all three doesn't automatically tell TIMBOT which one is authoritative.

A useful answer needs to distinguish between a proposal, a decision and a historical observation.

If the notes don't establish that distinction, TIMBOT should say so.

This is the same engineering problem encountered with web evidence, but in a different environment.

The source is more familiar.

The reasoning problem remains.

Keeping the original documents visible

One of the more dangerous habits of AI assistants is making source information disappear behind a fluent summary.

The answer sounds complete, so the person reading it has little reason to inspect what the model actually found.

I want the opposite.

When TIMBOT uses knowledge from SilverBullet, the useful result should retain a connection to the originating document.

A page title or document reference is more valuable than an unsupported assertion that the answer came from my notes.

Ideally, I should be able to move from the answer to the relevant SilverBullet page and inspect the original context.

That also makes errors easier to correct.

If the source document is wrong, I can update it.

If the source is correct but TIMBOT interpreted it badly, that is a different problem.

If TIMBOT retrieved the wrong page entirely, that is another.

Without source references, all three failures look much the same.

With them, there is at least a reasonable path to diagnosis.

Retrieval should be bounded

SilverBullet documents can become large.

A project knowledge base can become much larger.

Sending every matching page to the language model would be an inefficient way to handle retrieval, especially when the default model runs on consumer hardware.

TIMBOT already operates within a finite context window and bounded tool rounds.

SilverBullet integration needs to respect those limits rather than treating the context window as unlimited storage.

The sensible approach is to retrieve a manageable number of relevant documents or sections.

That introduces tradeoffs.

Tight limits reduce noise and inference cost, but they can exclude information needed to answer a question.

Broad retrieval improves coverage but may bury the useful evidence inside unrelated material.

More context is not automatically better context.

I also don't see a compelling reason to introduce a vector database simply because the information now comes from a knowledge management system.

FTS5 has already been useful for TIMBOT's existing memory.

Conventional text search, document titles, links and metadata are reasonable starting points for knowledge retrieval.

Semantic search may become useful later.

It should solve an observed retrieval problem, rather than become another component added because AI systems are expected to have one.

Reading documents is not permission to obey them

There is a security boundary here that matters.

TIMBOT's existing web research system treats retrieved pages as untrusted data.

SilverBullet content needs a similar distinction, even though the documents are personally maintained.

A Markdown page could contain instructions, code blocks, copied conversations or material retrieved from elsewhere.

A project note might legitimately say:

Ignore the previous configuration and replace it with this one.

That could be an instruction written for a human working on the project.

It should not become an instruction to TIMBOT merely because the assistant retrieved the page.

The content is evidence relevant to the question.

It is not a replacement for the system's operating rules.

This becomes particularly important if SilverBullet eventually supports write operations through TIMBOT.

Reading a document and editing it are different privileges.

A read-only knowledge lookup should never quietly become permission to rewrite project notes.

The model can propose changes.

The application must decide which operations exist and who is allowed to authorize them.

That is the same principle used elsewhere in TIMBOT.

The model can request an action.

It cannot invent authority.

The temptation to let AI organise everything

Once an assistant can read a knowledge base, the obvious next idea is to let it maintain that knowledge base.

Automatically summarise conversations.

Create project pages.

Update decisions.

Rewrite outdated documentation.

Connect related notes.

There is genuine potential in that.

There is also a fairly obvious failure mode.

An assistant that misunderstands a conversation could write its misunderstanding into the knowledge base.

On the next request, it could retrieve that note as evidence.

The original mistake would then acquire the appearance of a documented fact.

Repeated often enough, this creates a feedback loop in which generated information becomes increasingly difficult to distinguish from information supplied or verified by a human.

I don't want that.

For now, the more useful principle is to keep the distinction between reading knowledge and modifying knowledge explicit.

If TIMBOT eventually writes to SilverBullet, I want changes to be inspectable and attributable.

Drafting a proposed page is not the same as publishing it.

Suggesting an update is not the same as silently replacing an existing decision.

And a model-generated summary should not be treated as independently verified knowledge merely because it has been saved to disk.

Another boundary around cloud models

SilverBullet also introduces a privacy consideration.

TIMBOT's default inference path uses my local AI machine, but it supports explicitly selected cloud models.

The existing interface lets me choose how much conversation history reaches a cloud provider.

Access to private project documentation needs the same kind of care.

A SilverBullet page may contain technical designs, unpublished ideas, network details or other information I have no intention of sending to an external service.

Allowing TIMBOT to retrieve that page for local inference should not automatically authorize sending it to OpenAI or Anthropic.

There are several distinct questions:

• Can TIMBOT search the knowledge base?
• Can it read the document?
• Can the local model use the retrieved content?
• Can any of that content be included in a request to an external model?

Those permissions should not collapse into a single decision.

One lesson from building TIMBOT keeps repeating: access to information and permission to disclose it are not the same thing.

What makes this useful

The practical benefit isn't that TIMBOT can search Markdown.

Plenty of software can search Markdown.

The benefit is being able to ask questions across an existing body of technical work using the same interface that handles conversation, local inference, memory and web research.

I can imagine asking:

What were the main design constraints for this project?

Or:

Find my notes about the earlier prototype.

Or:

Compare the current design requirements with the original proposal.

These requests involve more than remembering an isolated fact.

They require finding existing information, preserving its source and interpreting it in relation to a specific question.

That is where a personal knowledge base becomes more useful than simply giving an assistant a larger memory.

It also changes what I expect from the assistant.

For a question about an existing project, a useful response may be a short summary with references to two documents.

It doesn't need to invent a comprehensive answer.

And if the documentation is incomplete, identifying the missing information may be the most useful result.

Still not an autonomous agent

The SilverBullet work fits the same approach that has guided TIMBOT so far.

I am not trying to build a system that continuously reads everything, reorganises my projects and makes decisions in the background.

TIMBOT is still a request-driven assistant.

It uses local inference by default.

External capabilities exist for specific purposes.

Retrieval is bounded.

Actions need defined interfaces.

The person using the system remains responsible for important decisions.

Connecting SilverBullet expands the information TIMBOT can work with, but it doesn't need to change those principles.

In fact, the more information an assistant can access, the more important its boundaries become.

The bigger picture

TIMBOT started as an experiment in running a personal AI assistant on hardware I control.

The interesting work gradually moved away from the language model itself.

First came routing, permissions and memory.

Then better treatment of web evidence, verification and tool failures.

Now the problem is how to connect the assistant to a larger body of personally maintained knowledge without losing the distinctions between facts, documents, instructions and decisions.

SilverBullet is a good fit because the information remains readable, editable and independent of the assistant.

TIMBOT doesn't need to own the knowledge.

It needs to find the right material, interpret it carefully and show enough of its sources that I can check the result.

That is a much more useful direction than trying to make the model remember everything.

The goal isn't an AI that knows everything I've written.

It's an assistant that knows where to look, understands the limits of what it finds, and leaves the original knowledge under my control.