• Develop and support AI applications using Python, Generative AI, and Large Language Models (LLMs).

• Design, execute, and automate comprehensive AI test strategies across the AI application lifecycle.

• Validate AI models, LLMs, AI agents, and agentic workflows for accuracy, consistency, reliability, and safety.

• Perform functional, integration, regression, performance, security, and user acceptance testing for AI solutions.

• Conduct LLM evaluation, prompt testing, hallucination testing, bias and fairness assessments, and guardrail validation.

• Develop automated frameworks for testing AI agents, chatbots, copilots, and autonomous workflows.

• Define and track AI quality metrics, benchmarks, and acceptance criteria.

• Identify, analyze, and resolve defects related to AI outputs, workflows, and model behavior.

• Collaborate with business stakeholders and development teams to ensure production-ready AI solutions for airline operations.

Desired Candidate Profile

• Strong programming expertise in Python.

• Hands-on experience with LLMs, Generative AI, RAG architectures, Vector Databases, and AI Models.

• Experience with AI frameworks such as LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, or Semantic Kernel.

• Experience developing and testing AI Agents, Agentic AI solutions, Chatbots, and Conversational AI platforms.

• Strong expertise in AI Testing, including:

• LLM Evaluation & Validation

• AI Agent Testing

• Prompt Testing & Optimization

• Hallucination Testing

• Bias & Fairness Testing

• Functional Testing

• Integration Testing

• Regression Testing

• Performance & Load Testing

• Security & Responsible AI Testing

• Test Automation for AI Applications

• Experience creating AI quality frameworks, test cases, evaluation datasets, and benchmarking methodologies.

• Familiarity with Azure OpenAI, AWS AI, Google AI services, or equivalent platforms.

Domain Experience

• Experience in the Airline, Aviation, Travel, or Transportation domain is required.

• Knowledge of airline operations, reservations, customer service, crew management, airport operations, or passenger experience solutions is an advantage.

Preferred Qualifications

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Data Science, Engineering, or related field.

• Certifications in AI, Machine Learning, Software Testing, or Cloud Technologies are preferred.