Duration
3 days
Target Audience
Anybody who has to test or manage testing of systems that involve AI.
Pre-requisites
In general, there are no prerequisites for attending an AiU course or taking the exam. We do recommend you hold the ISTQB CTFL certificate and have first experience in the field of AI & ML.
Objectives
- Understand applications of Artificial Intelligence (AI) using Machine Learning (ML)
- Compare different implemented ML algorithms to help choose the most suitable one
- Define a test strategy for testing of AI systems
- Design and Execute test cases for AI-systems
- Use various methods for bringing transparency into model workings
- Understand where AI can be used in manual testing and test automation
- Use AI based tools for test automation and test execution
- Evaluate models for both-Supervised and Unsupervised Learning
Chapter 1 - Introduction to Artificial Intelligence
- Artificial Intelligence (AI) and Machine Learning (ML)
- Defining Artificial Intelligence (AI)
- Types of AI
- Types of ML
- Supervised learning - Classification and Regression
- Unsupervised learning - Clustering and Association
- Reinforced Learning
- Deep Learning (DL) and types of Neural Networks (RNN, DNN, CNN)
- Stages of ML Process
- Stages of ML Process - CRISP-DM process
- Steps for Identification of ML problem type
Chapter 2 - Overview of Testing AI Systems
2.1.1 Offline and Online testing of AI Systems
2.2.1 AI testing Vs traditional testing
- Quality characteristics for evaluating AI systems
- Extended Quality characteristics specific to AI
Chapter 3 - Offline Testing of AI Systems
- Data preparation and preprocessing
- Steps of data preparation and preprocessing
- Data manipulation and filtering
- Processing of Unstructured data (images)
- Processing of Unstructured data (text)
- Dimensionality Reduction
- Data visualization
- Anomaly/Outliers detection
31.8 Outliers Detection Techniques
3.1.9 Data imputation
- Metrics
- Role of metrics
- Metrics for supervised and unsupervised learning
- Inertia and adjusted rand score
- Support, confidence and lift
- Confusion matrix
- Accuracy, precision, recall, specificity and F1-score
- RMSE and R-Square
- Training, validation and testing datasets
- Underfitting and Overfitting
- Cross-validation
3.4.1 Analytics
Chapter 4 - Online Testing of AI Systems
- Architecture of an AI application
- Components of an intelligent app and their testing needs
- Interaction of AI and non-AI parts
- Linguistic analysis method
- Linguistic analysis based test design
- Testing AI systems
- Test a chatbot
- Testing an image-classifier
Chapter 5 - Explainable AI
- Explainable AI and its need
- LIME
- CAM for Neural Networks
- Counterfactual examples
Chapter 6 - Risks and Test Strategy for AI Systems
- Risks in testing AI
- Challenges of testing AI systems
- Risk of using pre-trained models
- Risk of Concept Drift (CD)
- Challenges of AI test environment
- Test Strategy
- Test Strategy for Testing AI applications
Chapter 7 - AI in Testing
- AI for STLC
- AI for Software Testing Lifecycle methods
- AI for smart reporting and dashboards
- AI based automation tools
- Tools
Programme scope, delivery arrangements and suitability are confirmed during enquiry.