Playbook

The customer feedback aggregator.

Pull feedback from Intercom, Zendesk, and email into one queryable stream. LLM classifies themes. Routing rules send the right items to the right team.

9 min read

The pain

Why feedback gets lost

Customer says "your mobile app crashes on the checkout screen" in three places: an Intercom chat, a Zendesk ticket, and a direct email to the founder. Three teams see one each. Nobody connects them. The bug ships to mobile in week 4 instead of week 1.

The four ways feedback aggregation breaks:

  • Per-tool dashboards. Intercom has its own tags. Zendesk has its own categories. Email has none. Nothing joins them.
  • Manual triage. Someone reads every ticket and tags it. They miss patterns because each ticket is its own window.
  • No deduplication. The same bug is filed in five tickets in two days. Engineering sees one, fixes it, the others sit unread.
  • No team routing. A pricing complaint goes to support. A bug report goes to support. A feature request goes to support. Nothing routes to the right team automatically.

The architecture

What we're building

One workflow per feedback source, all writing to one normalized table. An LLM step classifies each item. A routing step delivers it to the right team.

  • One source, one trigger. Intercom new conversation = Rotor event. Zendesk new ticket = Rotor event. Email parser = Rotor event.
  • Normalize first. Every source maps to {customer, content, source, sourceId, receivedAt}.
  • Classify with an LLM. One step, one prompt, one structured output: theme, severity, team.
  • Route per theme. Bug to Linear, feature to ProductBoard, billing to RevOps Slack.
workflow({
  id: "ingest-intercom-conversation",
  trigger: { event: "intercom.conversation.created" },

  steps: async ({ event, step }) => {
    const conv = event.data;

    // 1. Normalize
    const item = await step.run("normalize", () => ({
      customer: conv.contacts.contacts[0]?.email,
      content: conv.source.body,
      source: "intercom",
      sourceId: conv.id,
      receivedAt: conv.created_at,
    }));

    // 2. Persist
    const id = await step.run("persist", () =>
      db.feedback.insert(item)
    );

    // 3. Classify with LLM (durable + cached)
    const classification = await step.run("classify", () =>
      llm.complete({
        model: "claude-sonnet-4-6",
        system: CLASSIFICATION_PROMPT,
        prompt: `Classify this feedback:\n\n${item.content}`,
        schema: ClassificationSchema,
      })
    );

    await step.run("update-classification", () =>
      db.feedback.update(id, classification)
    );

    // 4. Dedup against last 7 days
    const matches = await step.run("find-similar", () =>
      db.feedback.semanticSearch(item.content, { days: 7, limit: 5 })
    );

    if (matches.length >= 2) {
      // Likely a duplicate report — group, don't notify
      await step.run("group", () =>
        db.feedback.linkAsDuplicate(id, matches[0].id)
      );
      return { grouped: true };
    }

    // 5. Route to team
    await routeToTeam(step, item, classification);
  },
});

The classifier

One prompt, structured output

const CLASSIFICATION_PROMPT = `
You classify product feedback. Output JSON matching the schema.

Themes: bug, feature_request, billing, onboarding, performance, other
Severity: critical, high, medium, low
Team: engineering, product, revops, support

Rules:
- "Crash" or "doesn't work" → bug, severity high or critical
- "I wish" or "would be great if" → feature_request, severity low
- "charged twice" or "billing" → billing, route to revops
- Default to support if unclear
`;

const ClassificationSchema = z.object({
  theme: z.enum(["bug", "feature_request", "billing", "onboarding", "performance", "other"]),
  severity: z.enum(["critical", "high", "medium", "low"]),
  team: z.enum(["engineering", "product", "revops", "support"]),
  summary: z.string().max(200),
});

Structured output via the model's JSON mode. The schema is the contract. If the LLM hallucinates a new theme, the validation step rejects it and the workflow retries with feedback.

Routing

One handler per team

async function routeToTeam(step, item, c) {
  switch (c.team) {
    case "engineering":
      await step.run("create-linear-issue", () =>
        linear.issues.create({
          team: "Engineering",
          priority: c.severity === "critical" ? 1 : 2,
          title: c.summary,
          description: `Source: ${item.source}\nCustomer: ${item.customer}\n\n${item.content}`,
          labels: [c.theme],
        })
      );
      if (c.severity === "critical") {
        await step.run("alert-eng-oncall", () =>
          slack.send("#eng-oncall", `Critical bug from ${item.customer}: ${c.summary}`)
        );
      }
      break;

    case "product":
      await step.run("create-pb-feedback", () =>
        productboard.insights.create({
          title: c.summary,
          content: item.content,
          customerEmail: item.customer,
        })
      );
      break;

    case "revops":
      await step.run("notify-revops", () =>
        slack.send("#billing", `Billing issue from ${item.customer}: ${c.summary}`)
      );
      break;
  }
}

Edge cases

What goes wrong, and how to handle it

LLM hallucinates a theme. Schema validation rejects. The step retries with the validation error in the prompt. After 2 retries, fall back to theme: "other" and route to support.

Customer not found. Email matches no contact in HubSpot. Persist anyway. The unknown-customer feedback gets flagged for sales to review.

Duplicate floods. Outage causes 200 tickets in an hour. The dedup logic groups them. One Linear issue gets created with a count. Engineering sees scale, not noise.

LLM cost runaway. Add a daily cap. If feedback volume exceeds 1,000/day, switch to a cheaper classifier or batch-classify hourly instead of per-item.

The math

What this costs on Rotor

A team receiving ~500 feedback items/day. Each = 5 step-runs. ~75,000 step-runs/month.

That fits Rotor Pro ($99/mo). Plus LLM costs (~$50/mo at typical volumes for a 500-item/day stream on Claude Sonnet).

Compared to:

  • Productboard's native feedback aggregation: ~$200-500/mo per seat. Not cheap, no LLM classification.
  • DIY on a worker: free in compute, but you maintain the LLM, the schema, the routing, and the dedup. 3+ engineer-weeks before it's stable.

Fork this playbook on Rotor.

$9 to start. 30-day money back. Hard caps protect you from runaway bills.

Start shipping