Senior QA Automation Engineer (AI & LLM Testing)

Nagarro

📍 Hungary, Hungary, Hungary

Full-time Computer Occupations Posted June 24, 2026

Job Description

Job Description

We are looking for a Senior QA Automation Engineer with strong hands-on experience in test automation, API testing, and AI/LLM quality evaluation.

  • Design, develop, and maintain automated test suites for web applications, APIs, and AI-powered features.
  • Evaluate LLM outputs for response accuracy, relevance, consistency, and hallucination risks.
  • Create prompt tests and quality checks using Promptfoo or similar LLM evaluation frameworks.
  • Build end-to-end UI automation using Playwright, following page object models and maintainable test architecture principles.
  • Develop reusable Python scripts for API testing, automation utilities, and LLM quality evaluation.
  • Create and maintain Postman collections for REST API testing, including environments, assertions, and CI/CD integration.
  • Use GitHub Copilot or similar AI coding assistants to improve test coverage, speed up automation development, and identify...