CT-GenAI iSQI ISTQB Certified Tester Testing with Generative AI (CT-GenAI) v1.0 Free Practice Exam Questions (2026 Updated)

Hallucinations—where an LLM generates factually incorrect or nonsensical information—occur primarily when the model lacks sufficient specific information and "fills in the gaps" using probabilistic patterns from its training data. The most effective mitigation strategy is "grounding," which involves providing the model with detailed, project-specific context. By including technical specifications, existing API schemas, business rules, and identified constraints within the prompt, the tester restricts the model’s operational space to the "project realities." This ensures the model does not have to guess or improvise details about the System Under Test (SUT). In contrast, randomizing prompts (Option B) or relying on generic examples (Option C) increases the likelihood of inconsistent and inaccurate outputs. Furthermore, using "longer" or higher temperature settings (Option D) actually encourages creativity and randomness, which is the opposite of the precision required for testing and significantly increases the risk of hallucinations. Therefore, rich contextual grounding is the technical foundation for reliable AI-assisted test analysis.

In the context of structured Generative AI capability building, a prompt pattern is a formalized method of interaction that ensures repeatability and reliability. Much like software design patterns, prompt patterns provide a reusable and structured template designed to guide Large Language Models (LLMs) toward producing specific, high-quality, and consistent outputs. Without these patterns, testers often rely on "zero-shot" or ad hoc prompting, which frequently leads to non-deterministic results that are difficult to validate in a professional testing lifecycle. By adopting prompt patterns, organizations can standardize how requirements are translated into test cases or how code is analyzed for defects. This standardization is critical for scaling GenAI across a team, as it allows for the creation of a "prompt library" where successful structures—such as Persona-based, Few-shot, or Chain-of-Thought patterns—are documented and reused. This approach moves the use of GenAI from a trial-and-error activity to a disciplined engineering practice, ensuring that the model understands the specific context, constraints, and expected output formats required for rigorous software testing tasks.

As Generative AI becomes integrated into the software testing lifecycle, the role of the tester shifts from manual authoring to the "orchestration" of AI models. Mastering prompt engineering is the primary competency required to effectively steer LLMs. Prompt engineering involves the deliberate design of inputs—incorporating roles, context, instructions, and constraints—to elicit the most accurate and "on-policy" outputs from the model. In a testing context, "on-policy" refers to testware that adheres to organizational standards, security protocols, and specific project requirements. While technical skills like network configuration or low-level programming (Options B, C, and D) are valuable in specific engineering domains, they do not directly influence the communicative interface between the human and the AI. A tester proficient in prompt engineering can utilize techniques like "Chain-of-Thought" or "Few-shot prompting" to ensure the LLM understands the nuances of a test plan, thereby reducing hallucinations and ensuring the generated test cases are actionable, relevant, and compliant with the project's quality gates.

Shadow AIrefers to the use of artificial intelligence tools and services within an organization without explicit approval or oversight from the IT or Security departments. In a software testing environment, this often occurs when testers use public, consumer-grade LLMs to analyze proprietary code or sensitive requirement documents to speed up their work. The primary compliance concern is theviolation of established data handling and regulatory compliance standards(such as GDPR, HIPAA, or SOC2). When sensitive test data is fed into a "shadow" AI tool, that data may be stored on external servers or used to train future iterations of the model, leading to massive data leaks and legal exposure. This bypasses the organization's security controls, such as data masking and role-based access. Unlike "authorized" AI which undergoes a rigorous vendor risk assessment, Shadow AI creates an invisible attack surface. For a test organization, mitigating this risk involves providing approved, secure AI alternatives and implementing strict policies and monitoring to ensure that internal intellectual property is never processed by unvetted external services.

Generative AI is exceptionally capable of creating structured and unstructured data, but its role is limited to "generation" and "transformation," not infrastructure management or direct database administration. Creating production database backups (Option A) is a physical data management task involving the copying of actual stateful data from a server to storage; this is handled by database management systems (DBMS) and DevOps pipelines, not LLMs. Conversely, LLMs excel at the logic-based tasks listed in the other options. They can analyze requirements to identify and set boundary values (Option B) for input validation. They are also highly effective at creating combinatorial data (Option C), such as pairwise or all-combinations tables, by understanding the relationships between variables. Finally, one of the most powerful uses of GenAI in testing is generating synthetic datasets (Option D)—creating "fake" but realistically structured data that mimics production patterns without exposing Sensitive Personally Identifiable Information (SPII), thereby supporting privacy-compliant testing.

In the context of LLM inference,Temperatureis a hyperparameter that controls the randomness or "creativity" of the model's output. When the temperature is set high, the model's probability distribution is "flattened," meaning it is more likely to select less-probable tokens, leading to more diverse and sometimes unpredictable text. For software testing, where precision and repeatability are paramount,lowering the temperature(Option C) is the standard practice. A temperature of 0.0 makes the model "deterministic," meaning it will consistently choose the token with the highest probability. This narrows the sampling distribution and significantly reduces variability between runs. While a larger context window (Option D) allows the model to process more information, it does not directly control the randomness of token selection. Similarly, the "learning rate" (Option B) is a parameter used during thetrainingorfine-tuningphase, not during inference. For generating test cases or scripts that must follow strict logic, a lower temperature ensures that the model remains focused and produces consistent results.

Understanding the hierarchy of LLM types is vital for selecting the right tool for specific testing tasks.Foundation LLMsare trained on massive datasets to predict the next token; they excel at broad, creative "ideation" (Statement i) but often struggle with following specific instructions or constraints (making Statement iv incorrect).Instruction-tuned LLMshave undergone additional training (Fine-tuning) to follow explicit commands and templates. They are highly effective at structured tasks like converting requirements into Gherkin feature files (Statement ii).Reasoning LLMs(or those utilizing specialized prompting like Chain-of-Thought) are designed to handle complex, multi-stage logic. This makes them the superior choice for diagnostic tasks like root-cause analysis, where the model must synthesize information across logs and requirements to find a defect's origin (Statement iii). Statement v is incorrect because while instruction-tuned models are capable, complex "stepwise reasoning" usually requires specific prompting techniques or the inherent logic of specialized reasoning models. Therefore, the combination of i, ii, and iii represents the correct alignment of model capability to testing functionality.

A hallucination refers to a phenomenon where a Large Language Model generates text that is grammatically correct and seemingly plausible but is factually incorrect or unsupported by the provided context or real-world data. In the context of software testing, this is a critical limitation. For example, an LLM might generate a test case for a software feature that does not exist or cite a non-existent API parameter. These errors occur because LLMs are probabilistic engines designed to predict the "most likely" next token rather than "reasoning" from a set of verified facts. They do not have a built-in "truth" mechanism. While a logical mistake (Option B) is a failure in reasoning and a systematic preference (Option D) describes bias, a hallucination is specifically about the fabrication of information. Testers must be particularly vigilant regarding hallucinations, as they can lead to "false confidence" in test coverage or the creation of invalid bug reports. Mitigations include grounding the model with Retrieval-Augmented Generation (RAG) and implementing rigorous "human-in-the-loop" verification of all AI-generated test artifacts.

A Retrieval-Augmented Generation (RAG) workflow is designed to "ground" an LLM's output in specific, verifiable data. The logical flow begins with an initial input or "focused query" (Step iv) that defines the tester's goal—in this case, generating cases for a new payments rule. The system then uses this query to perform a semantic search in avector database(Step i) to find the most relevant "chunks" of the new API specification. Following this, the system retrieves complementary data from therelational database(Step iii), such as historical test cases that might provide structural patterns or regression context. Finally, all the retrieved information—the new specs and the historical context—is bundled together and "fed" into the LLM as part of an augmented prompt (Step ii). This ensures the LLM doesn't hallucinate rules but instead synthesizes the new requirements with established organizational testing standards. Following the order in Option B ensures that the model is provided with the most relevant and logically organized context prior to generating the final testware.

As Generative AI is integrated into the testing lifecycle, the role of the human tester undergoes a significant shift from "author" to "orchestrator and reviewer." In traditional testing, a significant portion of a tester's time is spent manually drafting test cases, scripts, and documentation. With GenAI, these artifacts can be generated in seconds. Consequently, the tester’s responsibility shifts towardreviewing, refining, and validatingthe AI-generated testware to ensure accuracy, relevance, and compliance with project goals. This "Human-in-the-Loop" (HITL) approach is critical because LLMs are prone to hallucinations and may lack the deep domain context of a human expert. Testers must apply their critical thinking to verify that the AI-generated scripts actually cover the necessary edge cases and do not contain logical errors. This evolution does not mean the end of human oversight (Option B) or a move exclusively to white-box testing (Option C). Instead, it elevates the tester to a higher-level analytical role, focusing on quality strategy and the final verification of AI outputs rather than the repetitive task of initial content creation.

Classical Machine Learning(which includes algorithms like Random Forests, Support Vector Machines, and Linear Regression) is characterized by its reliance onFeature Engineering. This is the process where human experts manually select, extract, and transform raw data into a set of "features" or variables that the algorithm can process. For instance, in a classical ML model predicting software defects, a tester might have to manually define features like "lines of code changed" or "number of previous bugs." In contrast,Deep Learningand its subset,Generative AI(Options B and D), utilize "Representation Learning." This means the multi-layered neural networks automatically identify and extract the relevant features from raw, often unstructured data (like text or images) without explicit human instruction.Symbolic AI(Option A) is based on hard-coded logical rules rather than data-driven learning. Understanding this distinction is fundamental for testers, as it determines the level of data preparation required: Classical ML requires high human effort in data structuring, while GenAI requires high effort in prompt engineering and grounding.