AI Quality Assurance Lead

Salary: £43,500 - 74,500 per year

Requirements

• We require active SC Clearance.
• We need proven experience designing and executing evaluation frameworks for AI, specifically M365 Copilot agents.
• We require strong Python programming skills.
• We need hands-on practical experience using PyTest to build automated testing framework components.
• We require a strong understanding of AI metrics and failure modes, including hallucinations, bias, drift, security vulnerabilities, and model explainability.
• We need a solid background in data science project delivery.

Responsibilities

• We will lead the end-to-end design, development, planning, and execution of test and evaluation frameworks tailored for AI solutions.
• We will work on the Cyber RIIO tech stack and M365 Copilot agents.
• We will build scalable automated test suites and validation pipelines using PyTest.
• We will rigorously test and evaluate AI-specific criteria including accuracy, relevance, faithfulness, hallucination rates, robustness, bias, fairness, security, privacy, and explainability.
• We will apply data science project delivery experience to ensure testing standards align with broader system architecture and delivery goals.

Technologies

• AI
• Copilot
• Python
• REST
• Security
• pytest

More

We are offering a contract role for an AI Test & Evaluation Engineer on an inside IR35 day rate of up to £600-£650. This is a hybrid position with one day per week in the office and the rest remote, with office locations available in London, Cardiff, or Glasgow. The role focuses on building evaluation frameworks and automated validation pipelines for AI platforms, including M365 Copilot agents, within complex technical environments.

last updated 32 week of 2026