AI/ML Testing Engineer

About the job

AI Quality Engineer.

We are looking for an AI Quality Engineer who will be responsible for validating cutting-edge artificial intelligence solutions across various domains, including generative AI, conversational AI, and predictive AI.

Must-Have Skills:

• Experience in testing and validating AI/ML solutions including Generative AI, Conversational AI, and Predictive models
• Strong understanding of how ML models are built end-to-end (data preparation, feature engineering, training, validation, tuning)
• Knowledge of core ML algorithms and model types (regression, classification, clustering, tree-based models, neural networks, transformers)
• Proficiency in Python for AI test automation, data analysis, and model output validation
• Hands-on experience with pandas, NumPy, and scikit-learn for data and model validation
• Experience in data quality analysis, profiling, and feature validation
• Understanding of model evaluation metrics and validation of performance results
• Ability to interpret model behavior and explainability outputs
• Experience testing AI APIs and services built using FastAPI or Flask
• Familiarity with cloud-based AI deployments, preferably AWS SageMaker
• Understanding of production ML lifecycle, including deployment validation and monitoring
• Strong analytical, problem-solving, and communication skills for cross-functional collaboration

Desirable:

• Hands-on experience testing Generative AI prompts, hallucinations, and response quality
• Familiarity with Responsible AI, bias, fairness, and safety validation
• Knowledge of MLOps pipelines and CI/CD validation for ML systems Experience with performance, latency, and scalability testing for AI services
• Exposure to adversarial testing and edge-case validation for AI models
• Experience testing AI systems in regulated or high-risk domains

Requirements added by the job poster

• 4+ years of experience in Information Technology

• 4+ years of work experience with Artificial Intelligence (AI)

• 2+ years of work experience with Retrieval-Augmented Generation (RAG)