AI Software Development Engineer in Test II (AI SDET II) job opportunity at CodeNinja.

Home Jobs In Pakistan Tag: Job AI Software Development Engineer in Test II (AI SDET II) position at CodeNinja


Date2026-01-14T12:29:25.546Z bot
CodeNinja AI Software Development Engineer in Test II (AI SDET II)
Experience: 3-years
Pattern: full-time
Country:
apply Apply Now
Salary:
Status:

Job

Copy Link Report
degreeHigh School (S.S.C.E)
loacation Lahore, Pakistan
loacation Lahore....Pakistan

About the Company CodeNinja is a global AI and engineering services company helping enterprises build, scale, and operate intelligent systems. With 350+ engineers across four continents and 400+ successful deployments, CodeNinja enables organizations to harness artificial intelligence through Global Capability Centers, Work AI, Physical AI, and AI Labs. Recognized among Pakistan’s fastest-growing AI firms and a multi-award recipient on Clutch, CodeNinja empowers over 250 clients worldwide to innovate, automate, and compete in the intelligence economy. About the role We are hiring an AI Software Development Engineer in Test II (AI SDET II) — an experienced quality engineer who owns feature-level test strategy and automation for AI-enabled products . In this role, you will design and maintain automated tests and AI evaluation suites , integrate quality signals into CI/CD pipelines , and partner with Scrum teams to prevent defects and deliver reliable AI capabilities. Key Responsibilities Own test strategy and execution for assigned AI features across unit , API , UI , and end-to-end layers, aligned to Definition of Done and release criteria . Create and maintain automated AI evaluation suites (including golden datasets , scoring , and regression baselines ) covering key user journeys and failure modes. Validate prompt , retrieval , and orchestration changes using repeatable test harnesses and defined acceptance thresholds . Build and extend automation frameworks , reusable utilities, and improve test stability , speed, and signal-to-noise . Integrate automated tests and AI evaluations into CI/CD , triage pipeline failures, and reduce flaky tests . Partner with engineers to improve testability (contracts, logging, observability) and drive shift-left quality practices. Analyze defects and AI output issues, perform root cause analysis , and implement regression coverage to prevent recurrence. Provide guidance to junior SDETs on automation , test design , and AI validation patterns; communicate quality risks to stakeholders. Support production readiness by validating guardrails , monitoring signals, and performance requirements for AI features. Bachelor’s degree in Computer Science/Engineering (or equivalent practical experience). 3+ years of experience in SDET / QA automation / quality engineering with strong hands-on automation and test design skills. Proficiency in at least one programming language (e.g., Python , C# , Java ) and SQL , with the ability to debug services and tests. Experience building and maintaining automated tests (API/UI and/or data/ETL ) and working in CI/CD pipelines . Experience testing AI-enabled systems or demonstrated applied AI fluency (prompt/RAG changes, evaluation basics, common failure modes). Experience working in Agile/Scrum teams and collaborating effectively with Engineering and Product. Preferred: Experience with LLM evaluation techniques (golden sets, rubric-based grading, automated scoring, review workflows). Preferred: Familiarity with AI safety and guardrail testing (grounding, PII handling, injection risks). Preferred: Experience with cloud tooling , containers , observability , and release automation . Why Join Us Build quality systems for real-world AI-enabled products , including LLM evaluations and reliability guardrails. Own feature-level quality end-to-end: from test strategy to CI/CD quality gates to production readiness . Work with strong cross-functional teams across Engineering , Product , and AI stakeholders in an Agile environment. Grow in modern quality engineering practices: observability , automation frameworks , shift-left testing , and AI evaluation patterns. Disclaimer: CodeNinja is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, & performance. Female and minorities are strongly encouraged and preferred to apply for the role

Other Ai Matches

CloudOps Lead Applicants are expected to have a solid experience in handling Job related tasks
Senior SQA Automation Engineer Applicants are expected to have a solid experience in handling Job related tasks
Associate AI Engineer Applicants are expected to have a solid experience in handling Job related tasks
Middle East - Talent Community - Future Roles Applicants are expected to have a solid experience in handling Job related tasks
AI Software Development Engineer in Test II (AI SDET II) Applicants are expected to have a solid experience in handling Job related tasks
Technical Lead (Backend / API & Cloud) Applicants are expected to have a solid experience in handling Job related tasks
QA Engineer (Manual, Automation, SDET) - Talent Community Applicants are expected to have a solid experience in handling Automation, SDET) - Talent Community related tasks
QA Engineer (Manual, Automation, SDET) - Talent Community Applicants are expected to have a solid experience in handling Automation, SDET) - Talent Community related tasks
Technical AI Scrum Master Applicants are expected to have a solid experience in handling Job related tasks
Middle East - Talent Community - Future Roles Applicants are expected to have a solid experience in handling Job related tasks
AI Talent Community - Future Roles Applicants are expected to have a solid experience in handling Job related tasks
Business Analyst - Pre Sales Applicants are expected to have a solid experience in handling Job related tasks
Senior SQA Automation Engineer Applicants are expected to have a solid experience in handling Job related tasks
AI Solution Architect Applicants are expected to have a solid experience in handling Job related tasks
Senior Database Administrator (DBA) Applicants are expected to have a solid experience in handling Job related tasks
Strategy Consultant / Researcher Applicants are expected to have a solid experience in handling Job related tasks
UI/UX - Customer Experience (CX) Specialist Applicants are expected to have a solid experience in handling Job related tasks
Senior AI Engineer Applicants are expected to have a solid experience in handling Job related tasks
Senior AI Success Manager Applicants are expected to have a solid experience in handling Job related tasks
Lead Strategy Consultant - Digital Innovation Applicants are expected to have a solid experience in handling Job related tasks
Senior Business Architect Applicants are expected to have a solid experience in handling Job related tasks
Fintech Integration Specialist Applicants are expected to have a solid experience in handling Job related tasks
Middle East - Talent Community - Future Roles Applicants are expected to have a solid experience in handling Job related tasks