Senior QA Engineer

Job Title: Senior QA Engineer – Web, Mobile & Conversational AI Testing
Location: Remote
Experience Level: 4+ Years
Employment Type: Contract

Job Overview:

We are seeking a Senior QA Engineer with proven expertise in testing modern web, mobile, and conversational AI applications. In this role, you will be responsible for shaping and executing end-to-end quality assurance strategies, driving both manual and automated testing efforts. Your primary mission will be to ensure robust functionality, seamless user experiences, and high-quality conversational outputs—especially in LLM-based (Large Language Model) AI systems.

This is an exciting opportunity to work on cutting-edge technologies and play a pivotal role in defining the quality benchmarks of AI-driven user experiences.

Key Responsibilities:

✅ Test Planning & Strategy

• Develop comprehensive test plans, test cases, and testing strategies across web, mobile (iOS/Android), and conversational AI platforms.
• Define QA frameworks for functional, UI/UX, performance, regression, and exploratory testing.
• Identify edge cases and potential failure points in user journeys and conversational flows.

✅ Execution & Automation

• Perform both manual and automated testing using tools like Selenium, Appium, Pytest, and Playwright.
• Design, maintain, and scale automated test suites for continuous integration and delivery (CI/CD).
• Execute real-world testing scenarios for mobile apps, including multi-device and cross-platform validations.

✅ Conversational AI & LLM Testing

• Evaluate response quality, relevance, tone, and coherence of LLM outputs (e.g., GPT-like systems).
• Test AI-driven features such as dialogue flows, fallback logic, hallucination detection, and safety.
• Validate integrations with RAG systems, vector databases, and external APIs.

✅ Cross-Functional Collaboration

• Work closely with developers, product managers, and AI/ML engineers to define testable requirements.
• Participate in design and sprint planning meetings to advocate for quality early in the development cycle.
• Communicate bugs and feedback clearly using tools like JIRA, Linear, and QA dashboards.

✅ Monitoring & Reporting

• Track testing coverage and test case execution metrics.
• Report bugs with detailed reproduction steps and prioritize based on business impact.
• Build quality reports and insights to drive informed decisions during releases.

Required Qualifications:

• 4+ years of experience in QA testing for web and mobile applications.
• Strong experience with both manual and automated testing frameworks.
• Hands-on experience with automation tools such as Selenium, Appium, and Pytest.
• Familiarity with LLM and NLP application testing (e.g., ChatGPT, Rasa).
• Experience testing REST APIs using Postman and writing SQL queries.
• Working knowledge of version control systems like Git.
• Comfortable working in Agile/Scrum teams and managing work via JIRA or Linear.

Preferred Skills:

• Exposure to prompt engineering or LLM evaluation frameworks.
• Experience testing AI-driven personalization or intent recognition.
• Familiarity with tools like Langfuse, Phoenix, or vector databases.
• Experience building QA dashboards and automated reporting tools.
• Understanding of CI/CD pipelines and DevOps workflows.

Educational Requirements:

• Bachelor's or Master’s Degree in Computer Science, Information Technology, or a related field.

Why Join Us?

• Work on cutting-edge LLM and AI technologies.
• Collaborate with a world-class team of engineers and product leaders.
• Make a real impact on how users interact with intelligent systems.
• Company Name: Wadhwani Foundations
• Average Salary Offered (for Current Role):10-20 LPA
• Must-have skills:
• 4+ years of experience in QA for web and mobile applications
• Hands-on experience with manual and automation testing tools and frameworks
• Strong knowledge of QA methodologies, testing types, and software development lifecycle
• Automation testing using Pytests
• Experience testing LLM-based or NLP-powered applications (ChatGPT, Rasa or custom LLMs)
• Proficient in writing SQL queries and working with Postman for API testing
• Familiarity with vector databases and conversational testing tools (Langfuse, Phoenix, etc.) is a plus
• How to prepare for the Interview?
• Brush up on your knowledge of QA methodologies, testing types, and the software development lifecycle
• Practice writing test cases and test plans for web, mobile, and conversational AI applications
• Familiarize yourself with automation testing tools and frameworks like Selenium, Appium, and Playwright
• Gain hands-on experience with testing LLM-based or NLP-powered applications, if possible
• Understand vector databases and conversational testing tools like Langfuse and Phoenix
• Be prepared to discuss your experience with prompt engineering and evaluation frameworks for LLM outputs
• Review your knowledge of SQL queries and API testing with Postman
• How to prepare the resume for this job description?
• Highlight your experience in QA for web and mobile applications, emphasizing the number of years
• Showcase your expertise in manual and automation testing tools and frameworks
• Demonstrate your strong knowledge of QA methodologies, testing types, and the software development lifecycle
• Mention your experience with automation testing using Pytests
• Highlight any experience you have with testing LLM-based or NLP-powered applications
• Showcase your proficiency in writing SQL queries and working with Postman for API testing
• If applicable, mention your familiarity with vector databases and conversational testing tools
• Links to prepare for the interview:
• QA Methodologies and Testing Types: https://www.guru99.com/software-testing-methodology.html
• Selenium Automation Testing: https://www.selenium.dev/documentation/en/
• Appium Mobile Automation Testing: http://appium.io/docs/en/about-appium/intro/
• Playwright Automation Testing: https://playwright.dev/docs/intro
• LLM and Conversational AI Testing: https://www.langfuse.com/blog/testing-large-language-models/
• SQL and API Testing with Postman: https://learning.postman.com/docs/writing-scripts/script-references/sql-integration/