Job Description
AI Evaluations & Red Team Lead
Location: Pune, India
Seniority: Senior (7-12 years)
Purpose: Lead FNZ's unified AI evaluations and red teaming capability, ensuring all AI agent solutions meet rigorous safety, performance, and compliance standards before production deployment.
Key Responsibilities:
Define and evolve AI evaluation strategy aligned to FNZ's six-pillar framework (Task Performance, Safety & Compliance, Efficiency, Groundedness & Reasoning, Robustness, Suitability)
Establish evaluation standards, methodologies, and scoring criteria integrated into FNZ's SDLC as mandatory release gates
Represent evaluations function in AI Governance Committee, providing risk assessments and release recommendations
Build, mentor, and manage team of 4 evaluation specialists across generalist and specialist roles
Design and execute complex evalu...