Conversational AI Designer Interview Questions

A working bank of questions for 2026. Pick four to six per round. Most of the signal is in follow-ups.

The strongest candidates speak in shipped surfaces, real conversation logs, and named tradeoffs. Push for failure stories. If you can't get a specific assistant they shipped, with traffic numbers, the work is probably prototype-stage.

Round 1: Recruiter or hiring-manager screen, 30 minutes

1. Walk me through one conversational experience you've shipped. What was the goal, the design, and the outcome?
2. What does your daily toolset look like — models, platforms, evaluation tools?
3. How would you describe the difference between a chatbot copywriter and a conversational AI designer?
4. What's a conversational failure mode you've personally hit, and what did you change as a result?
5. What's the biggest mistake you've seen teams make in launching an AI assistant?
6. Why this role, why now, and what would success look like for you in 90 days?

What you're listening for: shipped work, real metrics, healthy skepticism, fluency in both design and model vocabulary.

What to flag: demo-only portfolios, platform name-dropping with no design fundamentals, vague refusal logic.

Round 2: Craft and judgment, 60 minutes, live

Discussion plus two short live exercises.

Discussion (25 minutes)

1. Describe the assistant or experience you're proudest of. Walk me through the persona, the flows, and the failure modes.
2. Walk me through a RAG implementation you've shipped. What was the chunking, the retrieval evaluation, what did the assistant do when it didn't know?
3. A user asks the assistant a question outside its scope. Walk me through how you design the refusal.
4. How do you decide when an assistant should hand off to a human? What does that hand-off look like?
5. How do you set up evaluation for a customer-facing assistant?
6. How do you handle prompt injection and sensitive data leakage in production?

Exercise A: Writing (15 minutes)

Brief: a fictional brand's customer support assistant. Provide a rough brand voice description and three sample customer scenarios. Ask the candidate to draft, on the spot:

• The system message.
• The default refusal.
• The escalation hand-off line for one scenario.

Strong signal: brand voice fidelity, structural discipline in the system message, kindness and clarity in refusals and hand-offs.

Exercise B: Flow (20 minutes)

Brief: a returns-and-exchanges assistant for an e-commerce brand. Ask the candidate to whiteboard:

• The happy path.
• The top three failure modes.
• The escalation rules.
• The evaluation scenarios they'd write to cover this surface.

Strong signal: flow fundamentals, awareness of edge cases, named evaluation scenarios.

Round 3: Take-home or paid trial

Pay for it. Cap at 4 to 6 hours. Pick one:

Persona and system message v1. Given a brand brief and three sample interactions, draft the persona, the system message, three example flows, and the refusal and hand-off behavior.

Conversational surface design. Pick one surface (web chat, in-product help, lifecycle, voice). Design it end-to-end: persona, flows, failure modes, evaluation suite, measurement plan.

Evaluation harness. Given an existing assistant transcript set (provided), design a scenario suite and rubric. Score 10 conversations and recommend the top three changes.

Safety and refusal review. Given a set of edge-case prompts (provided), describe how the assistant should behave for each and write the refusals.

Score on: writing voice, flow fundamentals, evaluation rigor, safety instincts, and how cleanly the work could be handed to a real team.

Round 4: Cross-functional panel, 45 minutes

Bring in product, support, content, brand, and one engineering partner.

1. (Product) How would you partner with product on conversational features in-product?
2. (Support) Have you partnered with a support team on an assistant rollout? Walk me through the playbook.
3. (Content) How do you partner with content on the assistant's voice and the underlying knowledge base?
4. (Brand) How do you protect brand voice in conversational surfaces under real load?
5. (Engineering) Have you partnered with engineers on production deployment, observability, and model upgrades?
6. (Risk) How do you approach prompt injection, sensitive data, and disclosure?

Strong signal: translation skills, comfort with hand-offs, instinct for production realities and risk.

Round 5: Leadership and vision, 30–45 minutes

Head of product, head of design, or hiring manager.

1. Where do you think conversational AI is going in the next 12 to 24 months, and how should we prepare?
2. What would you change about our current assistant in your first quarter? (Send public materials in advance.)
3. What do you need from leadership to do your best work?
4. What's an opinion you hold about conversational AI that most of your peers would disagree with?
5. What scares you about this role?

Strong signal: strategic clarity, an actual point of view, candor about needs.

Scoring rubric

A 5 in each dimension:

• Writing voice. System messages, refusals, and hand-offs sound like a real brand under load.
• Conversation design. Strong flow fundamentals. Failure modes named. Escalation logic clear.
• Model and tooling fluency. Honest, current, specific. Articulates tradeoffs without hype.
• RAG and grounding. Has shipped at least one. Speaks fluently to chunking, retrieval evaluation, citations.
• Evaluation discipline. Reads logs at volume. Builds scenario suites. Reports learnings.
• Safety awareness. Practical answers for injection, sensitive data, disclosure, refusals.
• Cross-functional fluency. Translates between design, product, support, content, engineering.

What they'll ask you

Strong candidates will interview you back. Have answers:

• What model APIs and conversational platform are available?
• Who owns the assistant today, and how is it measured?
• How autonomous is the role on persona, refusals, and tooling?
• How is the role measured at six months? At twelve?
• What's leadership's appetite for risk on conversational surfaces?

If you don't have answers, the role isn't ready to open.