Skip to main content

Fuse Software GmbH

(DE | EN)

back

AI Support ·

A 0-to-1 AI Support Agent for Complex Domain Software

Jonas Meintschel

When software is both badly documented and full of domain complexity, an ordinary AI + RAG agent gets nowhere.

You have to build the agent's understanding of the program from zero. The fastest way to get a support chat that actually understands the program is to start from the source code. Source code beats documentation on two counts. It is current, and it shows you what changed between versions.

The code in this Fuse project is written in an old language that few people use today. Language models still read its logic well. Any model scoring above 50% on the Artificial Analysis Intelligence Index handled it. The identifiers in the code — variables, functions, and so on — carry usable implicit knowledge about the application domain. So we add a glossary that explains the domain-specific terms. It sharpens the AI's grasp of the context.

Implementation

From the source code, we build a Markdown documentation of everything the program does. This functional documentation is our cache of what the AI understood. For each part of the program, the agent makes its tool calls once and places the functionality in its domain context once.

The functional documentation is also the layer the chat later uses for RAG — for the vector database and for keyword search.

The documentation needs a sensible structure, and so do the steps the AI takes to write it. Here we chose one document per program screen.

For each screen, an AI agent writes one Markdown file that follows a fixed structure.

Example:

Structure (generalized)## PlacementWhich code files describe this screenWhich menu paths open the screen## Domain ContextPurposeTypical use casesDomain terms## Permissions## Layout## RulesLocksMessagesField validations

Three agents produce the full documentation: a Navigator, a Writer, and a Reviewer. The Navigator finds the source files behind the screen and turns them into findings for every aspect the structure asks for. The Writer builds the screen's documentation from the Navigator's findings alone, then saves it. The Reviewer checks the outline and confirms that every finding made it into the text. The three jobs stay strictly separate. That is what keeps hallucinations out.

Navigator

Prompt (generalized)Collect every raw finding for each section of the structure. Cite the source code as verifiable evidence. Completeness matters most.

Tools:

  • Read a file
  • Write a file
  • Run a bash command
  • Store findings in structured form
  • Look up domain terms in the glossary
  • Query the menu structure from the database
  • Build a table of every input field
  • Pattern search for locking rules, messages, and permissions at screen and object level
  • Look up what a variable means in the database documentation

Writer

Prompt (generalized)Write the screen description for domain users, not engineers. Use the findings and nothing else. Every finding has to appear in the file. Follow the given structure.

Tools:

  • Store the documentation
  • Create a git commit

Reviewer

Prompt (generalized)You read and judge. You do not correct. Your answer is either “PASS” or a list of every gap and error. Check whether the text covers all aspects of the structure.

Tools:

  • Read a file
  • Verify claims against the source code
  • Glossary and database documentation, to check the description is right

Runner

A script drives the documentation run for one screen. The Navigator goes first and produces findings. The Writer drafts the screen description. The Reviewer analyzes the draft. If the description is complete, the screen is done. If not, the Writer gets up to two more rounds to fix the errors from the latest gap list.

A Claude Code agent watches the run across every screen we document. It also tunes API utilization, restarts failed runs, and shows progress against the expected finish time.

Harness

We chose Pi (pi.dev) as the harness, because it leaves us in full control. Each agent gets its own tools and prompt instructions, matched to its job. Claude Code looks after the runner, and nothing else.

Model

We picked the open-weights model DeepSeek V4 Flash 0731. It writes good descriptions, and the price fits the value.

API Provider

Choosing the API provider turned out to be more interesting than choosing the model. These criteria mattered most:

  • Model availability
  • Server location
  • GDPR compliance
  • A hard spending cap you can set
  • API limits on tokens per minute and requests per minute
  • Price
  • Caching, which affects the price

Retrieval

The screen descriptions answer questions from users in a chat interface. A retrieval Pi agent runs the backend behind it.

Prompt (generalized)You answer questions from users of the program. Search the knowledge base first. Invent nothing. If you cannot say anything about something, explain what you do not know. Answer in the user's language and never name source code.

Tools:

  • Search knowledge through the RAG pipeline
  • Permission lookup
  • Glossary

Indexing

A vector database finds the descriptions that are closest in content. The application already runs on PostgreSQL, so we used its pgvector extension. A vector database stores snippets of text, and we cut our chunks along the Markdown headings. We embedded with BGE-M3 on TEI and indexed every document the agents had written. The cross-encoder is bge-reranker-v2-m3, also on TEI.

Alternative Approach

An agent could also answer questions by searching the codebase itself, with the same tools at hand. We ruled that out. Users would wait longer for an answer, and we expect it to cost more to run.

An AI agent documented every screen in the program, in full. We defined the catalog of aspects to describe up front. We also built parsing tools for the specific programming language, to help the agent write. The description then grows in a loop: extend it, check it for completeness, extend it again.

Hybrid RAG reaches the stored knowledge, with BGE-M3 embeddings on TEI. The cross-encoder is bge-reranker-v2-m3, also on TEI. A rigid hybrid retrieval pipeline answered well enough, and a pi.dev agent made it better. The agent gets the pipeline as a tool for searching knowledge. It also gets a tool for querying permissions, so it can answer in the context of the user asking. A third tool handles attachments.

Contact

Jonas Meintschel · j.meintschel@fusesoftware.de