← Back to Blog

2026-09-22 • 5 min read • Measure

How to Measure Which Documentation an AI Agent Actually Uses

How to compare the pages your MCP server cited with the pages you expected, using MCP Studio's source breakdown and, on Action analytics, the passage.

You can believe the agent is using the help center and be half right. It is using the help center, and it is also citing a launch blog that still describes a flag you removed from the product. Site analytics will not show that. The blog might have almost no page views. The agent retrieved it anyway.

Measuring which documentation an agent uses means reading what the MCP server returned, then laying that list next to the pages you intended.

What the two levels show

The free breakdown is the source and the client. Which document set was consulted, and whether the calls came from Claude, Cursor, or a curl loop you forgot was running. A repository that never appears is not what the agent is using, even if you added it. Label your own tests so they are not the top user.

Action analytics is the page. Core plus Action, $99 a month, shows the page, section, and excerpt for each request. That is the documentation the agent was given. Core at $49 a month adds the question text, so you can see which question pulled which page. Predictive analytics, $149 a month with the tiers below it, adds an A to D grade per source. Use the grade to choose what to open. Read the page before you treat the letter as a verdict.

How much of a call you can see is in What the questions people ask your docs can tell you. A server nobody has called has nothing to measure.

Why usage is not quality

A source can be cited on every call and still be the wrong paragraph. Open the passage. That check is Measure MCP retrieval quality.

One afternoon is a sample. A week of the same top page is a pattern. Recheck after the next batch of requests, not after a single test from your laptop.

A page with no citations is not automatically useless. If related questions are being asked, the page is not in the agent's hands. If nobody asked, the silence is not evidence.

Read the citations in MCP Studio

MCP Studio is the no-code builder on Appa Tools. Open the server on the dashboard. The view names are in the dashboard guide.

  1. Read the free breakdown by source, tool, and client. Mark your own test traffic.
  2. On Action analytics, list the pages cited for the questions you care about. Compare that list to the pages you intended.
  3. Ask by hand when an expected page never appears. If a different page is cited, the heading or a retrieval rule is the next edit. Retrieval rules are how you prefer the page you meant.
  4. Disconnect a stale source or leave it in place with your eyes open. Disconnecting deletes the indexed copy of that source. It does not touch the others.

Check that it works

You added the help center and a launch blog. The free breakdown shows almost all calls on the help center. Action shows the cancellations article cited whenever someone asks about a refund, and the launch blog cited once for the removed flag.

The cancellations article is the page you review when refund policy changes. The blog is the one you disconnect or keep on purpose.

Questions that cite nothing are the other half of this measurement. They are documentation gaps.

If the top page is a surprise

Do not remove it because it was not in the plan. Read the passage. If it is the right answer, the plan was wrong. If it is a stale flag, disconnect that source and ask the question again.

See which pages the agent uses today

Open the server in MCP Studio and read the source breakdown. On Action analytics, open one cited page and compare it to the page you expected. Keep the source that matches the policy. Disconnect the one that does not.

Where to go next