Agentic AI Notebook
Back to Projects
AdvancedPhase 10 35 hours(broken down below)

AI Interview Coach

Voice-enabled interview practice agent with real-time feedback, scoring, and personalized improvement plans.

PythonOpenAI Realtime APIWhisperReact

Project walkthrough

AI Interview Coach

1 / 6

Project Goal

Voice-first interview practice with real-time AI and rubric scoring.

  • →Natural voice conversation via Realtime API
  • →Behavioral and technical question banks
  • →5-dimension automated scoring
  • →Personalized coaching plans across sessions

Use ← → arrow keys or buttons to navigate the walkthrough

Time breakdown (35h)

Each phase maps to the estimated hours — follow in order for a realistic build schedule.

Voice pipeline setup

8h
  • •WebSocket proxy with auth and session management
  • •Audio codec handling (PCM 24kHz)
  • •Latency monitoring and reconnect logic

Interview flow & prompts

8h
  • •Role-based question bank (behavioral + technical)
  • •Interviewer persona system prompt
  • •Follow-up question generation based on answer depth

Scoring rubric & feedback

8h
  • •5-dimension rubric with structured JSON scores
  • •Per-answer feedback with specific improvement tips
  • •Aggregate session score and percentile tracking

React UI & session storage

6h
  • •Voice activity indicator and recording controls
  • •Live transcript display
  • •Session history with score trends

Coaching plan generator

5h
  • •Identify weakest rubric dimensions across sessions
  • •Generate weekly practice plan with targeted questions
  • •Export PDF summary for user review

Architecture

The React client streams audio via WebSocket to a FastAPI proxy that forwards to OpenAI Realtime API for low-latency speech-to-speech interaction. After each answer, a scoring service evaluates the transcript against a 5-dimension rubric and generates targeted feedback and a personalized improvement plan.

100%
Loading diagram...

Scroll inside the frame to explore · use + / − to zoom up to 200%

Prerequisites

  • OpenAI Realtime API access and WebSocket fundamentals
  • React with audio recording (MediaRecorder API)
  • Understanding of speech-to-text and text-to-speech pipelines
  • Experience designing evaluation rubrics
  • Basic WebSocket proxy patterns in Python (FastAPI)

Setup steps

  1. Create FastAPI WebSocket proxy to OpenAI Realtime API
  2. Build React UI with microphone capture and audio playback
  3. Define interview rubric: clarity, structure, depth, relevance, confidence
  4. Seed 10 behavioral and 10 technical question banks by role
  5. Store session transcripts and scores in SQLite or Postgres
  6. Test end-to-end voice loop with <500ms perceived latency

Features to build

  • Voice interaction
  • Real-time feedback
  • Score tracking
  • Personalized coaching

Expected result

Conduct a 5-question mock interview via voice, receive real-time follow-ups, then view a scored report across 5 dimensions with specific feedback and a personalized 1-week improvement plan.

Resume bullet points

  • →Built voice-enabled AI interview coach with real-time speech processing
  • →Designed evaluation rubric with automated scoring across 5 dimensions

Interview questions

Why use OpenAI Realtime API instead of Whisper + TTS separately?
Realtime API provides sub-second speech-to-speech with natural turn-taking and interruption handling. Separate STT→LLM→TTS adds 2–4s latency per turn, breaking conversational flow.
How do you design a fair interview scoring rubric?
Define explicit dimensions (clarity, structure, depth, relevance, confidence), use structured JSON scores, calibrate against human-labeled sessions, and avoid scoring on accent or speaking speed.
How would you handle a user with a poor microphone?
Detect low audio quality via SNR metrics, prompt user to check mic settings, offer text-input fallback, and use noise suppression preprocessing before sending to the API.