[Remote] Quality Assurance Engineer (Remote)

Note: The job is a remote job and is open to candidates in USA. Medical Review Institute of America is a clinically-driven, tech-enabled utilization management company offering expert clinical reviews and actionable insights to healthcare organizations.

The Quality Assurance Engineer will be responsible for ensuring the quality and reliability of software products, particularly in testing applications and validating AI/ML features.

Responsibilities Primary role is testing software applications created or updated by the Application Development teamConduct structured peer and code reviews of test artifacts to ensure adherence to standards and best practicesDesign, create, and maintain realistic and compliant test data sets, including synthetic data for healthcare (PHI-safe) scenariosDesign and execute validation strategies for AI/ML and Retrieval-Augmented Generation (RAG) features, including model quality, grounding, and safety evaluationParticipate in the test management lifecycle, including test planning, execution tracking, defect management and reportingProvide input on release readiness criteria to Engineering and Product stakeholdersMaintain the regression suite after every releaseDevelop and maintain automated test suites in Python (e.g., pytest) and JavaScript/Node.js, integrated into CI/CD pipelinesWrite and review complex SQL queries, joins, and aggregations to validate data accuracy, completeness, and reconciliation across systemsValidate data flows across source systems, transformation layers, and reporting/consumption layersEnsure test data supports both functional automation and AI/ML / RAG evaluation needsDesign and execute test strategies for AI/ML and Retrieval-Augmented Generation (RAG) pipelines and AI-assisted features, including retrieval accuracy, grounding, hallucination, output quality, bias/safety, and drift (regression) checksBuild and maintain automated evaluation harnesses and datasets for AI/ML features, including prompt/response scoring and metric trackingValidate applications, APIs, and test environments deployed on on-prem data center, Microsoft Azure, including Azure-hosted services, data stores, and CI/CD pipelinesRead and interpret React / Node.js / Python / ColdFusion source code, pull requests, and technical designs to identify testable conditions, edge cases, and regression riskWrite detailed test cases and test plans that trace directly to code paths, business rules, and acceptance criteriaPartner with developers during code review to flag insufficient test coverage, untested branches, and high-risk changes before mergePerform post-deployment validation to confirm application, API, and AI feature stability in productionInvestigate production defects and data issues, perform root cause analysis, and drive resolution with Engineering Skills 5+ years of experience in automated software testing5+ years of experience in manual software testing5+ years of experience in component-level testing5+ years of experience in end-to-end testingAt least 5 years of experience in SQLAt least 3 years of experience in ReactAt least 3 years of experience in Node.jsAt least 3 years of experience with Python source code in GitLabAt least 2 years of experience testing AI/MLAt least 2 years of experience evaluating AI/MLAt least 2 years of experience testing Retrieval-Augmented Generation (RAG)At least 2 years of experience testing generative AI featuresAt least 2 years of experience testing workloads on Microsoft AzureHigh School Diploma requiredBachelor's degree (Advantage) Benefits Benefits include healthcare, vision, and dental insuranceA generous 401(k) matchPaid vacation, PTO, and holidaysGrowth and training opportunitiesAn award-winning remote work environment Company Overview Medical Review Institute of America, LLC (MRIoA) is the top medical review company in the United States.

It was founded in 1983, and is headquartered in Salt Lake City, Utah, USA, with a workforce of 501-1000 employees. Its website is https://www.mrioa.com/. Company H1B Sponsorship Medical Review Institute of America has a track record of offering H1B sponsorships, with 1 in 2022.

Please note that this does not guarantee sponsorship for this specific role.