An answer is only as good as the source behind it. So every answer Orrin gives in the console shows where it came from: a numbered citation that opens the exact passage in your documentation.
Your documentation becomes passages
When you add a help page, paste text or upload a file, Orrin splits the text into passages. A new passage starts at every heading, and a long section is cut into passages of a paragraph or two that overlap a little, so a sentence is never cut off from the words around it.
Each passage keeps the heading it sat under. That is why a citation can point to the right section of a long article, not just to the article. Passages belong to one workspace and are never shared with another.
Each question is ranked three ways
Orrin compares every passage with the question three ways. The first is meaning. A small language model, all-MiniLM-L6-v2, turns the question and each passage into a list of numbers, and two texts that mean the same thing get lists that point the same way, even when they share no words. The model runs inside Orrin’s own server, so your documentation is not sent to another company for this.
The second counts exact words, using BM25, a ranking method search engines have long relied on. The third compares the overall mix of words and short phrases. The three are blended, with meaning counting most, so a passage that uses your exact term and a passage that says the same thing in other words can both reach the top.
A gate before any answer
A passage whose meaning is not close enough to the question’s is dropped as weak evidence. If nothing is left, Orrin does not guess. It says the documentation does not cover the question, and in a workspace it records the question as unanswered.
From the passages that pass, Orrin keeps only the sentences whose own meaning is close to the question. It skips headings, stops at two sentences, and puts the number of the source passage after each one. Asked “How do I waive a late fee for a tenant?”, the help center of Brickwren, the example company in our case study, gives this answer:
To waive a late fee, open the tenant's ledger, select the fee and choose Waive fee. [1] You need the Waive fees permission to do this. [1]
One click to check
In the console every citation is a link. It opens the document in a new tab, scrolls to the cited passage and highlights it, so whoever reviews an answer can check it in one click. If the document was reindexed and the passage has changed, the page says so instead of pointing at the wrong text.
What an unanswered question is for
Every question your documentation cannot answer is counted on the Usage page, and any of them can be raised as a support ticket. That list is written by your users, in their own words, which makes it the most useful documentation backlog there is.
Where it falls short
The language model is small. It measures how close two texts are in meaning, not whether a passage truly answers a question, so Orrin sets its bar high: when a question is worded very differently from your documentation, Orrin more often says the documentation does not cover it than risks a wrong answer. Quoted answers also follow the ranking rather than the order of the article, so a quoted sentence occasionally reads oddly out of its context. That is exactly why the citation is there: the source is always one click away.