Support QA Reviews
On closed Intercom tickets, an agent scores clarity, tone, and resolution via GPT and logs results to Google Sheets.
What it does
You get instant quality checks on every closed Intercom ticket as the agent scores your support conversations for clarity, tone, and resolution using OpenAI. It automatically logs these structured results to Google Sheets so you can track performance and provide targeted coaching feedback without manual review.
Step by step
- Configure an Intercom webhook listener to trigger on ticket status changes.
- Implement HTTP request logic to fetch full conversation metadata and message history from Intercom API.
- Define a strict JSON schema for the output scores to ensure compatibility with Google Sheets columns.
Where the LLM does the work
- Generate natural language coaching feedback based on specific low-scoring metrics identified in the transcript.
- Evaluate subjective qualities like tone and empathy by analyzing the context of customer-agent interactions.
- Structure raw conversation logs into a coherent narrative summary that highlights key resolution points.
Watch out for
Long transcripts may exceed token limits, so truncate or summarize history before sending to the model.
Ambiguous ticket statuses might trigger false positives, so filter webhooks strictly by closed status and valid IDs.
Tools that fit
OpenAI Chat Model LLM Evaluate response quality and tone
The agent brief
Everything your agent needs, including the gotchas. Copy it and go.
You are helping me build the following AI agent workflow. ## Goal Support QA Reviews: On closed Intercom tickets, an agent scores clarity, tone, and resolution via GPT and logs results to Google Sheets. ## Specification - What it does: You get instant quality checks on every closed Intercom ticket as the agent scores your support conversations for clarity, tone, and resolution using OpenAI. It automatically logs these structured results to Google Sheets so you can track performance and provide targeted coaching feedback without manual review. - Trigger: Triggered by an event (Event · on conversation closed) - Autonomy: Fully hands-off - Expected setup effort: about an afternoon - Tools/services involved: - Intercom: Trigger and fetch conversation data - OpenAI Chat Model: Evaluate response quality and tone - Google Sheets: Log structured QA scores ## Known pitfalls, handle each one explicitly in your implementation 1. Long transcripts may exceed token limits, so truncate or summarize history before sending to the model. 2. Ambiguous ticket statuses might trigger false positives, so filter webhooks strictly by closed status and valid IDs. ## Reference implementation https://n8n.io/workflows/5831 (workflow template) Fetch and inspect this before building. If it matches my stack, adapt it; if not, rebuild the pattern with my tools. ## Process requirements 1. Before building: ask me which of the listed tools I actually use and what my platform is (n8n / Make / code / other). Do not assume. 2. Adapt the pattern to my answers; do not force the reference stack. 3. Address every pitfall above; tell me how you handled each. 4. Provide a test plan I can run before letting this touch real data. 5. Ask before any step that sends messages, modifies data, or spends money. Source: https://usecasesforagents.com/use-case/automate-support-qa-reviews/ via usecasesforagents.com
Frequently asked questions
Can I use a different Support & Ticketing than Intercom?
Yes. Intercom is only the example support & ticketing in this recipe. The same flow works with Hiver, Marker.io, Syncro and WHMCS API. Swap the support & ticketing connection and keep the rest of the setup as written.
Want this running in your business?
This is what I do. I design and build AI agents like this one, and keep them running. If you want it set up for your team instead of doing it yourself, get in touch.
Who it's for
Built for support teams drowning in inbound and looking to triage faster, not just harder.