Response quality scoring and feedback
Omnifox scores your agent's replies and flags weak ones; review them and leave your human feedback.
Watch this step in the video · Reply quality
What quality scoring is
Omnifox automatically reviews your AI agents' replies with a quality judge (another AI that grades them), so you catch weak or risky answers without reading every conversation by hand. Each reply gets an overall score broken down into four dimensions: relevance, safety, coherence and helpfulness. Replies that fall below the threshold get flagged for your review.
How to open the Quality panel
- In the side menu open AI Agents.
- Open the Quality section (page
/ai-agents/{id}/quality). - At the top you'll see the totals: scored replies, flagged %, reviewed % and average overall score, along with the active flagging threshold.
How to review and give feedback
- Use the filters: by agent, by range (1 day, 7 days, 30 days or all), and the Flagged only toggle to focus on what matters most.
- Open a reply to see the per-dimension breakdown and the judge's comment.
- Click Review and set your human verdict: good, bad or neutral. You can add a comment explaining why.
- Save. Your decision is recorded and counts toward the reviewed %.
This human feedback is gold: it confirms (or corrects) the automatic judge and surfaces patterns you can fix in the agent's instructions.
How to verify it's working
After a few hours of traffic, return to the panel: you should see scored replies and, if there were weak cases, some flagged ones. Turn on Flagged only and confirm you can open and rate each one.
If it doesn't work
- No scored replies: the agent hasn't produced replies in that range yet, or scoring is still processing. Widen the range to 30 days.
- Too many flagged: this is usually an instructions or knowledge-base problem, not the judge. Read 5–10 flagged cases and look for the pattern.
- No Quality section: you need the
ai-agents.viewpermission.
Tip: spend 10 minutes a day reviewing only the flagged ones. It's the fastest way to raise your agent's real quality.
Tip: when you mark several replies "bad" for the same reason, edit the agent's instructions and re-measure a week later.
Related articles
Related articles
-
Create your first AI agent
Learn what an AI agent does in Omnifox, how to give it instructions, connect it to a channel and test it in the sandbox before it talks to real customers.
-
How to write good instructions for your AI agent
Learn how to write clear instructions so your AI agent replies with the right tone, limits and actions.
-
AI Agents complete guide
AI Agents in depth, in 65 chapters: creating, instructions, tools, knowledge, behavior, tester, inbox, workflows, analytics and usage, with every setting explained.
-
Connect your AI agent to WhatsApp, Instagram, Messenger and web chat
An agent with no channels serves nobody. Connect it to your WhatsApp, Instagram, Messenger or website chat from the Connections section, then switch it on so it starts replying.
-
Start here: AI Agents in 8 minutes
AI Agents essentials in a short video: creating an agent, its instructions, tools, knowledge, channels, the tester, hours and handing over to a person.