AI Quality Engineer | 6+ Years of Experience | Hybrid | Charlotte, NC | Contract

Walker Healthforce is seeking an experienced AI Quality Engineer to support an innovative enterprise AI initiative within a highly regulated industry. This is an exciting opportunity to help shape the quality, reliability, and performance of next-generation AI solutions, including Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), intelligent assistants, and AI-driven business workflows.

As an AI Quality Engineer, you will partner with cross-functional teams to develop testing strategies, validate AI-powered applications, and establish quality engineering best practices that ensure secure, scalable, and business-ready AI solutions.

START DATE: ASAP
HOURS/SCHEDULE: Monday-Friday, 40 hours
WORKER TYPE: W2 Contractor

CORE REQUIREMENTS:

• Bachelor's degree in Computer Science, Engineering, Information Systems, Data Science, or a related technical field.
• 6+ years of experience in Quality Engineering, Software Testing, QA Automation, or related technical roles.
• Experience testing enterprise applications, APIs, workflow-based systems, and data pipelines.
• Experience validating AI or Machine Learning solutions, including Large Language Models (LLMs), prompt behavior, model outputs, or AI-driven workflows.
• Strong understanding of software testing methodologies, defect management, root cause analysis, and quality metrics.
• Experience with test automation frameworks, test management tools, and CI/CD-driven quality practices.
• Excellent analytical, troubleshooting, and communication skills with the ability to collaborate across technical and business teams.

ADDITIONAL REQUIREMENTS:

• Design and execute functional, integration, regression, usability, performance, and non-functional testing strategies for AI-powered applications.
• Develop evaluation datasets, quality benchmarks, and test cases for prompts, retrieval pipelines, orchestration workflows, and end-user interactions.
• Evaluate AI solutions for accuracy, consistency, relevance, latency, safety, and task completion across realistic business scenarios.
• Identify model failure modes, hallucination risks, workflow breakdowns, and edge cases while recommending corrective actions.
• Support automation of AI testing, monitoring, quality reporting, and release readiness.
• Validate APIs, integrations, data pipelines, and supporting enterprise systems.
• Document defects, testing results, risks, and quality decisions throughout the software development lifecycle.
• Monitor production feedback and operational metrics to drive continuous improvement.
• Help establish enterprise AI quality engineering standards, governance, and best practices.

WE CONSIDER IT A BONUS IF YOU ALSO HAVE:

• Experience with LLM evaluation, prompt engineering, prompt testing, or RAG validation.
• Experience with AI observability, model evaluation, or responsible AI frameworks.
• Proficiency with Python, SQL, REST APIs, or scripting for automation and data validation.
• Experience developing synthetic datasets, golden test sets, or AI evaluation frameworks.
• Knowledge of AI governance, model risk management, privacy, or security controls.
• Experience within financial services, wealth management, healthcare, consulting, or other regulated industries.

WHAT WE OFFER:

• Competitive compensation package.
• Weekly pay via Direct Deposit.
• Medical, Dental & Vision benefits available.
• 401(k) options.

WHY WORK WITH WALKER?

• Access to our top-ranked team of recruiting and placement specialists.
• Continuous, one-on-one support from a dedicated engagement manager.
• Professional resume development and interview preparation.
• Opportunities to work on cutting-edge AI and enterprise technology initiatives.

Contact our Recruiting Team today to experience the Walker Healthforce difference!
 

ABOUT US:

Walker Healthforce is known as the dominant force of performance, precision, expertise, and integrity in the healthcare consulting community. As a certified WMBE, we provide end-to-end healthcare IT and clinical solutions to hospitals, health systems, and payer organizations, including Fortune 100 firms nationwide. We are healthcare experts, we're custom not commodity, and we've been exceeding expectations for nearly 20 years. Join forces with us to experience unparalleled results today!

We are an Equal Opportunity Employer committed to a culturally diverse workforce. All qualified applicants will receive consideration for employment without regard to race, religion, color, age, sex, national origin, sexual orientation, gender identity, disability status or protected veteran status.