> For the complete documentation index, see [llms.txt](https://constructor-chat.gitbook.io/botmetrica-api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://constructor-chat.gitbook.io/botmetrica-api-docs/how-it-works.md).

# How it works

## LLM-as-a-judge

BotMetrica automatically analyzes AI agent’s conversations with users, detects issues, and classifies them using predefined tags.\
Our LLM judge attaches tags to specific messages and explains **why** each tag was applied.

<figure><img src="/files/uGLnsFRuzt8OjCIT9WWz" alt=""><figcaption><p>The judge sees the AI agent’s context (e.g. retrieved information from a database) as well as the whole conversation</p></figcaption></figure>

### Labeling Reliability

**Key advantage:** automated analysis identifies **more issues with higher accuracy** than manual review.\
At the same time, the **distribution of error types closely matches human labeling**, ensuring consistency and trustworthiness.

<figure><img src="/files/wwKwUGrni76APegBsLde" alt=""><figcaption><p>You can see the reasoning and specific quoted parts of the message on each automatically found issue</p></figcaption></figure>

## Labeling speed comparison

Example from one of our clients: **467 conversations — one month of traffic**

* **Manual human review:** \~11 hours
* **AI review:**\
  \~10 minutes — more than **65 times faster**
  * Finer-grained tagging
  * More detected events and issues
  * Structure comparable to manual labeling
  * Stable, repeatable results across different samples