• As part of Speech Analytics, you will own the human-in-the-loop review processes that measure model accuracy, reliability, and alignment with Chime’s standards for quality and member trust
• Your work provides the trust layer that ensures models behave as expected — identifying gaps, failure modes, and opportunities for improvement
• You’ll partner closely with Speech Analytics, Data teams, Enablement, and Model Owners to ensure AI systems operate safely and consistently in production
• Own the Human-in-the-Loop evaluation process for all AI models supporting Operations
• Run recurring sampling and reviews to assess accuracy, consistency, and failure modes
• Score, tag, and document cases where AI systems misclassify, hallucinate, skip steps, or generate incomplete outputs
• Maintain structured rubrics and guidelines to ensure reviewer alignment and scoring consistency
• Conduct deeper investigations into error patterns and root causes
• Translate insights into recommendations for model owners and partner teams
• Track and report key evaluation metrics such as accuracy, recall, coverage, and error types
• Maintain thorough documentation for evaluation procedures, sampling logic, and scoring definitions
• Collaborate with cross-functional teams to integrate evaluation findings into dashboards and tuning workflows

Support scaling governance processes and strengthening model-health standards across Operations

• Experience reviewing unstructured text and applying rubrics or scorecards
• Understanding of how AI supports operations (classification, summarization, categorization, automation)
• Ability to identify patterns, edge cases, and failure modes from qualitative and quantitative data
• Familiarity with QA frameworks or content-review workflows
• Experience with SQL, Looker, Snowflake (nice to have)
• Strong attention to detail and high consistency standards
• Clear communication and documentation skills

A passion for improving member experience by ensuring AI is safe, fair, and reliable

3–5+ years in QA, evaluation, operational analytics, HITL programs, or model monitoring

• The base salary offered for this role and level of experience will begin at $105,000 and up to $145,000
• Full-time employees are also eligible for a bonus, competitive equity package, and benefits
• The actual base salary offered may be higher, depending on your location, skills, qualifications, and experience

What we offer for our full-time, regular employees 🏢 Our in-office work policy is designed to keep you connected - with four days a week in the office and Fridays from home for those near one of our offices, plus team and company-wide events depending on location

Whether you’re coming in regularly or are part of our fully remote program, you’ll stay engaged with your work and teammates. 💻 In-office perks including backup child, elder, and/or pet care, plus a subsidized commuter benefit to support your regular commute 💰 Competitive salary based on experience ✨ 401k match plus great medical, dental, vision, life, and disability benefits 🏝 Generous vacation policy and company-wide Chime Days, bonus company-wide paid days off 🫂 1% of your time off to support local community organizations of your choice 👟 Annual wellness stipend to use towards eligible wellness related expenses 👶 Up to 24 weeks of paid parental leave for birthing parents and 12 weeks of paid parental leave for non-birthing parents 👪 Access to Maven, a family planning tool, with $15k lifetime reimbursement for egg freezing, fertility treatments, adoption, and more

🎉 In-person and virtual events to connect with your fellow Chimers—think cooking classes, guided meditations, music festivals, mixology classes, paint nights, etc., and delicious snack boxes, too! *Perks also available to Chime Interns