Job Description
About Us
MyRemoteTeam, Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent. We empower businesses by providing world-class software engineers, operations support, and infrastructure to help them grow faster and better.
Job Title: AI Engineer
Experience: 5+ Years
Working Model: Remote
Responsibilities
Build Python-based pipelines for automated quality testing of AI responses.
Integrate LLMs into automated evaluation frameworks (e.g., using GPT-based evaluators, embeddings, or custom scoring).
Automate regression and stress testing for conversational AI flows.
Define evaluation metrics (relevance, factuality, coherence, safety, empathy).
Implement both rule-based and AI-driven quality checks.
Monitor model drift, bias, and hallucinations using automated workflows.
Work with APIs, SDKs, and CI/CD pipelines to embed automated AI evaluation in production.
Develop monitoring dashboards to visualize conv...
MyRemoteTeam, Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent. We empower businesses by providing world-class software engineers, operations support, and infrastructure to help them grow faster and better.
Job Title: AI Engineer
Experience: 5+ Years
Working Model: Remote
Responsibilities
Build Python-based pipelines for automated quality testing of AI responses.
Integrate LLMs into automated evaluation frameworks (e.g., using GPT-based evaluators, embeddings, or custom scoring).
Automate regression and stress testing for conversational AI flows.
Define evaluation metrics (relevance, factuality, coherence, safety, empathy).
Implement both rule-based and AI-driven quality checks.
Monitor model drift, bias, and hallucinations using automated workflows.
Work with APIs, SDKs, and CI/CD pipelines to embed automated AI evaluation in production.
Develop monitoring dashboards to visualize conv...