CT-AI ISTQB Certified Tester AI Testing Exam Free Practice Exam Questions (2026 Updated)

Automation bias is defined in Section4.4 – Human Factors in AI Testingof the ISTQB CT-AI syllabus. It refers to the human tendency to overly trust, rely on, or defer to automated system outputs. The syllabus explains that this bias arises especially indecision-support systems, where humans may accept AI judgments without adequate verification. This aligns directly with Option B.

Option A is incorrect: automation biasdoesinfluence testing, especially when testers rely excessively on AI outputs. The syllabus cautions about testers adopting the same cognitive biases as end users. Option C is incorrect because autonomous systems are not the primary context; rather,systems supporting human decisionsare most impacted. Option D is incorrect because the quality of human inputmatters significantly, and poorly designed user studies can mask or distort automation bias.

Thus,Option Bis the syllabus-accurate description of automation bias.

The syllabus states that while pairwise testing is effective at finding defects by reducing the number of test cases needed, the resulting test suite can still be extensive and require automation:

"Even the use of pairwise testing can result in extensive test suites… automation and virtual test environments often become necessary to allow the required tests to be run."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.2, Page 67 of 99)

Recognizing digits from a scan of handwritten numbers using machine learning is an example of classification. Here’s a breakdown:

Classification: This type of machine learning involves categorizing input data into predefined classes. In this scenario, the input data (handwritten digits) are classified into one of the 10 digit classes (0-9).

Why Not Other Options:

Reinforcement Learning: This involves learning by interacting with an environment to achieve a goal, which does not fit the problem of recognizing digits.

Regression: This is used for predicting continuous values, not discrete categories like digit recognition.

Clustering: This involves grouping similar data points together without predefined classes, which is not the case here.

The question asks which test is least likely to be performed during the ML model testing phase. Let's consider each option:

Testing the accuracy of the classification model (A): Accuracy testing is a fundamental part of the ML model testing phase. It ensures that the model correctly classifies the data as intended and meets the required performance metrics.

Testing the API of the service powered by the ML model (B): Testing the API is crucial, especially if the ML model is deployed as part of a service. This ensures that the service integrates well with other systems and that the API performs as expected.

Testing the speed of the training of the model (C): This is least likely to be part of the ML model testing phase. The speed of training is more relevant during the development phase when optimizing and tuning the model. During testing, the focus is more on the model's performance and behavior rather than how quickly it was trained.

Testing the speed of the prediction by the model (D): Testing the speed of prediction is important to ensure that the model meets performance requirements in a production environment, especially for real-time applications.

The ISTQB CT-AI syllabus distinguishes between several types of bias relevant in AI testing, includingsample bias,algorithmic bias, andinappropriate bias. In Section3.3 – Bias in AI-Based Systems, the syllabus stresses the importance of identifying biases that originate fromtraining data,model development, anddecision logic. Sample bias occurs when the training data does not adequately represent the population; algorithmic bias arises when the model produces systematically skewed results due to learned patterns; inappropriate bias involves ethically or socially problematic distortions in the outcomes. All three of these bias types directly affect theoutputs of the AI modeland are therefore highly relevant when testing an industrial inspection system intended to reliably detect defects. These biases can lead to defective items being missed or false alarms being raised, which impacts quality assurance significantly .

Automation bias, however, is fundamentally different. It refers to ahuman cognitive bias, where users (e.g., inspectors) overly trust or rely on the AI system’s output. While important in user-interaction testing, it isnota biaswithin the ML model itself. Since the question asks which bias isleast important when testing the model, automation bias can be legitimately deprioritized duringmodel-level testing. Therefore, OptionBis correct.

Section1.6.3 – Reinforcement Learningof the ISTQB CT-AI syllabus states that reinforcement learning (RL) is based on anagent interacting with an environment, performing actions, and receivingrewards or penalties. The core concept is thereward function, which guides the agent’s learning process. The syllabus emphasizes that training in RL isdriven by rewards, and the agent aims to maximize cumulative reward over time. Therefore, OptionCdirectly reflects the correct description: the agent learns by being rewarded for successful actions .

Option A is incorrect because RL doesnotuse labeled data; that applies to supervised learning. Option B contradicts the syllabus definition: RL fundamentallyrequiresinteraction with the environment. Option D is incorrect because the reward function isdefined by humans, not learned by the agent; the agent learns apolicy, not the reward function itself.

Thus, OptionCis the only statement consistent with RL as defined in the syllabus.

The syllabus describes that data pre-processing includes cleaning (e.g., fixing or removing invalid data) and transforming data (e.g., changing data types such as numbers stored as strings).

"Transformation: The format of the given data is changed… converting categorical data into numerical data, changing image formats…"

(Reference: ISTQB CT-AI Syllabus v1.0, Section 4.1.1, Page 34 of 99)

The ISTQB CT-AI syllabus explains supervised learning under Section1.6 – Machine Learning Approaches. It definesclassificationas predictingcategorical labels, whereasregressionpredictscontinuous numerical values. OptionB—deciding whether an object is a bicycle or a motorcycle—fits the definition of classification precisely because the model chooses between discrete categories. The syllabus also uses similar examples to illustrate classification tasks, reinforcing that this is the correct interpretation .

Option A is incorrect because image recognition of a dog is aclassificationtask, not regression. Option C is incorrect because predicting a 10% price rise involves forecasting anumerical value, which is aregressionproblem. Option D is incorrect because classification can involveany number of classes, not only two. Multiclass classification is explicitly mentioned in the syllabus.

Therefore, OptionBis the only answer aligned with the syllabus’ definitions.

Section1.7 – Neural Networksof the ISTQB CT-AI syllabus explains that neural networks consist of neurons connected by weighted links. During training,learning occurs by adjusting the weights on these connections. This is the essence of gradient descent and backpropagation. Option B correctly states this behavior: only theweightsare modified, not the activation functions, neuron counts, or architectural structure.

Option A is incorrect because a neuron’sbiasis not determined by previous activations; it is an independent trainable parameter added to the weighted input sum. Option C is incorrect because the syllabus states that asingle-layer perceptron is a valid type of neural network, although limited to linearly separable problems. Option D is incorrect because no rule requires the number of input neurons to exceed or equal the number of output neurons. Instead, input neurons correspond to thenumber of features, while output neurons correspond totasks or classes.

Therefore,Option Bprecisely reflects the syllabus definition of what changes during neural network training.

The syllabus explains:

"Where the internal structure of an AI-based system is too complex for humans to understand, the system can only be tested as a black box. Even when the internal structure is visible, this provides no additional useful information to help with testing."

This confirms that black-box testing is chosen because the tester does not need to understand the system’s internal structure.

(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.5, page 61 of 99)

The syllabus explains that ML models can be used to analyze reported defects and suggest which developers are best suited to fix them based on historical data about defect assignment and resolution speed:

"Assignment: ML models can suggest which developers are best suited to fix particular defects, based on the defect content and previous developer assignments."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 11.2, page 78 of 99)

The ISTQB CT-AI syllabus explains inSection 4.5 – Testing AI-Specific Risksthat adversarial testing is a structured test activity in which testers applyadversarial attacks—crafted or perturbed inputs—to intentionally expose weaknesses in the ML model. The purpose is to identify vulnerabilities that could be exploited throughdata poisoning,evasion attacks, orinput manipulation. OptionCcorrectly reflects this syllabus definition: adversarial testing is aboutusing attacks to locate weaknesses so they can be removed or mitigated.

Option A is incorrect because regression testing does not verify data sourcing policies; it verifies unchanged functionality after modifications. Option B is incorrect because adversarial examples are oftenaddedto training datasets to improverobustness(a practice called adversarial training), not excluded. Option D is incorrect because AIB testing is not described as superior to exploratory data analysis in outlier detection; both have different purposes, and EDA remains essential for data quality assessment.

Thus,Option Cis consistent with syllabus-defined adversarial testing.

 Testing Autonomy:Testing for human handover when it should not be relinquishing control is the least appropriate because it contradicts the very definition of autonomous systems. The other tests are relevant to ensuring smooth operation and transitions between human and AI control.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Sections on Testing Autonomous AI-Based Systems and Testing for Human-AI Interaction​​​​. ​

A. The difficulty of defining criteria for improvement before the model can be accepted.

Defining criteria for improvement is a challenge in the acceptance of AI models, but it is not directly related to the performance drop in real-world scenarios. It relates more to the evaluation and deployment phase rather than affecting the model's real-time performance post-deployment.

B. The fast pace of change did not allow sufficient time for testing.

This can significantly affect the model's performance. If the system is self-learning, it needs to adapt quickly, and insufficient testing time can lead to incomplete learning and poor performance.

C. The unknown nature and insufficient specification of the operating environment might have caused the poor performance.

This is highly likely to affect performance. Self-learning AI systems require detailed specifications of the operating environment to adapt and learn effectively. If the environment is insufficiently specified, the model may fail to perform accurately in real-world scenarios.

D. There was an algorithmic bias in the AI system.

Algorithmic bias can significantly impact the performance of AI systems. If the model has biases, it will not perform well across different scenarios and data distributions.

Given the context of the self-learning nature and the need for real-time adaptability, optionAis least likely to describe the fall in performance because it deals with acceptance criteria rather than real-time performance issues​​.

The syllabus mentions that experience-based testing and EDA are effective for detecting biases:

“Experience-based testing can be used to verify that the training dataset is operationally relevant and identify potential sources of bias. EDA is also useful for exploring the data and understanding any relationships that might lead to bias in the model.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.3, page 58 of 99)