AI Test and Evaluation Engineer
6314 Remote/Teleworker US
Looking for the opportunity to work on problems that matter, with colleagues that share your interest and expertise in AI and Machine Learning?
Leidos is looking for a Test and Evaluation Engineer to apply their expertise to help Leidos research, develop, and commercialize test and evaluation techniques for AI systems. This includes leading the test and evaluation planning, process definition, tools and testing pipeline research, as well as verification and validation planning.
We are looking for someone who is intellectually adaptive, likes to collaborate ad is inquisitive. You will work alongside junior and senior research/data scientists and data engineers with expertise in machine learning and artificial intelligence, who lead advanced research in AI.
You will work with a team of data scientists and data engineers to operationalize your research and make resulting tools and technologies deployable into operational environments. You will contribute to new approaches for test and evaluation of AI models that produce measurable advances over state of the art solutions.
Job Description
Looking for the opportunity to work on problems that matter, with colleagues that share your interest and expertise in AI and Machine Learning?
Leidos is looking for a Test and Evaluation Engineer to apply their expertise to help Leidos research, develop, and commercialize test and evaluation techniques for AI systems. This includes leading the test and evaluation planning, process definition, tools and testing pipeline research, as well as verification and validation planning.
We are looking for someone who is intellectually adaptive, likes to collaborate ad is inquisitive. You will work alongside junior and senior research/data scientists and data engineers with expertise in machine learning and artificial intelligence, who lead advanced research in AI.
You will work with a team of data scientists and data engineers to operationalize your research and make resulting tools and technologies deployable into operational environments. You will contribute to new approaches for test and evaluation of AI models that produce measurable advances over state of the art solutions.
Primary Responsibilities
- Develops Test and Evaluation CONOPS
- Creates scalable Test and Evaluation plans for AI systems including process, techniques and tools.
- Works with data scientists and developers to understand the AI system under test to develop test procedures
- Works with data scientists and developers to identify test data for positive and negative testing
- Works with MLOps engineers to integrate testing tools and procedures with the CI/CD pipeline
- Collects metrics to analyzes processes for potential improvements
- Collaborates with AI Governance team to maintain visibility and explainability through testing
- Implements testing process in the AI system design, development and deployment life cycle
- Identifies the risk in testing of projects, particularly for assessing the limitations of planned tests on complex AI systems
- Works within teams of AI/ML researchers and engineers using Agile development processes
Basic Qualifications
- Bachelor's degree in Computer Science, Data Science or related field and 8+ years of relevant experience, Masters with 6+ years experience, or PhD with 4+ years experience.
- Strong Python programming fundamentals
- Experience with system and subsystem level test process and automation
- Experience with creating user acceptance test scenarios
- Experience with SecDevOps tooling and MLOps pipeline development
- Experience with AI Performance and vulnerability assessment
- AI model assurance evaluation
- Experience applying and automating AI interpretability & explainability tools and methods
- Experience with developing CONOPS and presentations
- Good understanding of machine learning algorithms, tools and platforms
- Self-starter with high intellectual curiosity
- Great communication skills, able to explain model and test results to a non-technical audience
- Proficient in data exploration techniques and tools
Preferred Qualifications
- Experience with data visualization libraries such as Plotly, Streamlit, and matplotlib.
- Experience with AI/ML tools, such as common Python packages (e.g., scikit-learn, NumPy, Pandas) and Jupyter notebooks
- Experience with database administration and data repositories
- Experience in data exploration techniques and tools
- Experience with building LLM and other Generative AI applications.
- Willing to learn new skills and platforms to support data analytics.
- Ability and willingness to obtain a Top Secret security clearance
Original Posting Date:
2024-10-28While subject to change based on business needs, Leidos reasonably anticipates that this job requisition will remain open for at least 3 days with an anticipated close date of no earlier than 3 days after the original posting date as listed above.
Pay Range:
Pay Range $101,400.00 - $183,300.00The Leidos pay range for this job level is a general guideline only and not a guarantee of compensation or salary. Additional factors considered in extending an offer include (but are not limited to) responsibilities of the job, education, experience, knowledge, skills, and abilities, as well as internal equity, alignment with market data, applicable bargaining agreement (if any), or other law.
ApplyJob Profile
Remote/Teleworker US
Tasks- Analyze processes for improvements
- Collaborate with data scientists
- Design
- Develop
- Develop test and evaluation plans
- Implement testing processes
- Integrate testing tools with CI/CD
- Planning
- Research
- Test
- Testing
Administration Agile Agile Development AI AI Explainability AI Interpretability AI/ML Analytics Artificial Intelligence Assessment Automation CI/CD Communication Compensation Computer Computer Science Data Database Administration Data Exploration Data Science Data Visualization Deployment Design Education Evaluation Generative AI Governance Jupyter Jupyter notebooks Machine Learning Matplotlib ML MLOps Numpy Pandas Planning Plotly Programming Python Research Scikit-learn SecDevOps Security Security Clearance Self-Starter Streamlit Support System design Teams Technical Test and Evaluation Testing Validation Verification Verification and validation
Experience4-8 years
EducationArtificial Intelligence AS Bachelor's Bachelor's degree Bachelor's degree in Computer Science Business Computer Science Data Science Degree Master's Ph.D. Related Field Science Security Technical
CertificationsSecret Security Clearance Security Clearance
TimezonesAmerica/Anchorage America/Chicago America/Denver America/Los_Angeles America/New_York Pacific/Honolulu UTC-10 UTC-5 UTC-6 UTC-7 UTC-8 UTC-9