NCP-AII NVIDIA AI Infrastructure Free Practice Exam Questions (2026 Updated)

" GPU-Host latency " issues in NVIDIA DGX or HGX systems are frequently caused by incorrect PCIe affinity or sub-optimal NUMA (Non-Uniform Memory Access) mapping. If a GPU is forced to communicate with a CPU core or an HCA that is not on its local PCIe switch/root complex, latency increases significantly as data must cross the QPI/UPI inter-processor links. The command nvidia-smi topo -m provides a detailed matrix of the system ' s internal topology, showing how GPUs, CPUs, and NICs are connected. It identifies whether the connection is via a single PCIe switch (PIX), multiple switches (PXB), or across the CPU (SYS). By inspecting this map, an administrator can identify if a software process is pinned to the wrong NUMA node or if a hardware path is unexpectedly degraded. While DCGM (Option C) is good for checking component health, it doesn ' t map the logical-to-physical affinity paths that cause specific latency " threshold " warnings.

The output provided is a result of the NVIDIA System Management (NVSM) tool, specifically the nvsm show healthinfo command. NVSM is an essential diagnostic framework for NVIDIA DGX systems that monitors hardware health, identifies faults, and helps ensure the system remains within its validated operational state.

In this specific diagnostic trace, the system reports that the " Verify installed GPU ' s " check has returned a status of Unhealthy. To provide a root cause, NVSM cross-references the live hardware enumeration from the lspci command against the system ' s known " Golden Configuration " (the hardware manifest defined in the firmware). The explicit error message, " Missing GPU at PCI address ' 07:00.0 ' " , indicates that the system expects a GPU module to be present at that specific PCIe bus address, but the hardware is not responding or visible to the bus.

This insight allows a system administrator to conclude that a GPU is missing from the logical perspective of the system. This is a critical hardware fault rather than a software or driver issue. In a DGX H100 or A100 system, this could be caused by a physical module failure, a power delivery issue to that specific segment of the GPU baseboard, or a failure in the PCIe switch fabric. Because the DGX relies on a full set of 8 GPUs for high-speed collective communications (NCCL), a single missing GPU will prevent the node from participating in large-scale training jobs, requiring physical inspection or a GPU tray replacement (RMA).

The NVIDIA System Management (NVSM) tool is the definitive CLI utility for monitoring the health of DGX platforms. While replacing a PSU (Power Supply Unit) is a common maintenance task, verifying that the new component is correctly integrated into the system’s health model is mandatory. While nvsm show power would provide specific data regarding wattage and voltage for the PSU, the most comprehensive way to ensure the replacement hasn ' t caused secondary issues or that the system hasn ' t remained in a " Degraded " state is to run nvsm show health. This command performs a global check across all subsystems: GPUs, NVLink switches, storage, fans, and power. If the PSU replacement was successful and the system is back to full redundancy, nvsm show health will return a " Healthy " status. In an AI factory setting, where DGX H100 nodes pull significant power, ensuring that all 6 PSUs (in an N+N or N+1 configuration) are not only physically green but logically acknowledged by the Baseboard Management Controller (BMC) is critical for preventing unexpected shutdowns during high-load training iterations.

In an NVIDIA DGX BasePOD or SuperPOD environment, " Cluster Health " is a binary state: either the entire fabric and all compute resources are ready, or the cluster is considered degraded. Using the Bright Cluster Manager (BCM) shell (cmsh), administrators can aggregate telemetry from every node in the cluster. For a system to be considered " Production Ready, " every single GPU across the multi-node deployment must report a status of Health = OK. This verification ensures that the hardware is communicating correctly over the PCIe bus, the NVLink fabric is initialized, and no ECC (Error Correction Code) memory errors are present. If even a single GPU in a 32-node cluster is unhealthy, collective communication libraries like NCCL may hang or experience significant performance penalties during " All-Reduce " operations, as the entire job typically scales to the speed of the slowest/unhealthiest component. Therefore, seeing Status_Health = OK for every device is the mandatory exit criterion for the bring-up phase.

GPUDirect Storage is the most effective solution because it reduces unnecessary CPU involvement in the data path between storage and GPU memory. In a traditional AI data pipeline, checkpoint data may move from storage into system memory, be processed by the CPU, and then be copied into GPU memory. This creates CPU overhead and can leave GPUs idle while waiting for data. GPUDirect Storage allows compatible storage systems, file systems, drivers, and NVIDIA GPUs to transfer data more directly into GPU memory. This improves checkpoint reload performance, reduces CPU copy overhead, and helps keep GPUs saturated during training. Increasing batch size may reduce the frequency of storage access, but it does not solve the checkpoint transfer bottleneck. SATA SSDs in RAID 0 may improve local sequential throughput, but they are not ideal for distributed training or shared cluster data access. Adding more GPUs can worsen the issue if the storage path is already the bottleneck. In NVIDIA AI infrastructure, storage-to-GPU optimization is critical for large model training, checkpointing, recovery, and maintaining high accelerator utilization.

The nvidia-ctk runtime configure command is a crucial step that modifies the Docker daemon configuration file (/etc/docker/daemon.json) to register the nvidia runtime. However, the Docker daemon only reads this configuration file during its initialization phase. Even though the toolkit is installed and the configuration file is updated, Docker will not be able to spawn GPU-accelerated containers until the service is refreshed. Executing sudo systemctl restart docker (or the equivalent for your container engine) is the mandatory final step. This forces Docker to reload its settings and recognize the NVIDIA Container Runtime as a valid option. Without this restart, attempting to run a container with the --gpus all flag will result in an error stating that the " nvidia " runtime is not found or is unconfigured. This is a common point of failure in automated AI infrastructure deployments where the configuration script finishes, but the service state remains stale.

For the " North-South " and " Management/Storage " Ethernet fabrics in an NVIDIA AI Factory, high availability is paramount. Unlike the InfiniBand compute fabric, which uses its own routing logic, the Ethernet side relies on standard data center protocols. To provide true hardware redundancy and double the available bandwidth (Load Balancing), NVIDIA recommends MLAG (Multi-Chassis Link Aggregation). MLAG allows two physical switches to appear as a single logical unit to the DGX nodes. The DGX can then bond its two Ethernet NICs (e.g., in an 802.3ad LACP bond) and connect one cable to each switch. This configuration provides several benefits: if one switch fails, the traffic seamlessly stays on the other link without the slow convergence times associated with Spanning Tree Protocol (Option A). Furthermore, it allows the cluster to utilize the combined bandwidth of both links for heavy storage traffic (like NFS or S3 ingestion). Using a single switch (Option C) or unmanaged hardware (Option D) creates single points of failure and lacks the traffic isolation (VLANs) required for secure AI infrastructure.

With the advent of the NVIDIA Container Toolkit and modern Docker versions (19.03+), the --gpus flag is the official, verified method for resource allocation. To restrict a container to specific hardware IDs, the syntax requires a specific string format: --gpus ' " device=0,2 " ' . This tells the NVIDIA Container Runtime to map only those specific physical GPU devices into the container ' s namespace. While environment variables like NVIDIA_VISIBLE_DEVICES (Option B) were used in older " nvidia-docker2 " setups, they are now considered legacy and can be overridden by the more modern --gpus flag. Option D is incorrect because simply mapping the device nodes (/dev/nvidiaX) is insufficient; the container also needs the appropriate volume mounts for the NVIDIA drivers and libraries, which the --gpus flag handles automatically. This precise isolation is critical in multi-tenant AI environments to ensure that a single developer or job doesn ' t accidentally utilize the entire 8-GPU tray of a DGX H100.

In a Bright Cluster Manager HA setup, the database (MySQL/MariaDB) must remain perfectly synchronized between the active and standby head nodes to allow for a seamless transition. This synchronization typically occurs over a dedicated management or heartbeat network. If cmsh status shows the database service as [FAIL] on the secondary node, it almost always points to a communication breakdown. Without a stable network path, the secondary node cannot receive the binary logs from the primary node to keep its local copy up to date. While licensing (Option A) is important, a license failure usually disables management capabilities entirely rather than just the MySQL sync. Furthermore, head nodes are management servers and do not require GPU drivers (Option C) for their primary function. Ensuring low-latency, reliable connectivity between the two head nodes is the primary troubleshooting step for resolving " MySQL FAIL " states in BCM.

The correct option is export HPL_OOC_SAFE_SIZE=4.0. NVIDIA HPL out-of-core mode allows matrix data that exceeds GPU memory capacity to be placed in host memory, but the GPU still needs reserved memory for drivers and runtime overhead. NVIDIA documents HPL_OOC_SAFE_SIZE as the amount of GPU memory, in GiB, reserved for the driver and not used by HPL out-of-core mode; increasing it is recommended when GPU out-of-memory errors occur. HPL_OOC_MODE=0 disables out-of-core mode, which would not help run a larger 40B matrix. HPL_OOC_NUM_STREAMS=8 changes the number of CUDA streams used for out-of-core operations, but it does not reserve driver memory. HPL_OOC_MAX_GPU_MEM=90 limits total GPU memory use, but the specific variable intended to leave safe driver space is HPL_OOC_SAFE_SIZE. During cluster burn-in, this setting helps preserve test validity while avoiding false failures caused by memory reservation issues rather than actual hardware instability.

The correct answer is ngc config set --org org-name --ace ace-name. NGC CLI uses configuration context to determine which organization, team, or entitlement scope the user is operating in. When users belong to multiple organizations, the CLI must be configured with the intended organization context so that registry access, model downloads, private containers, and enterprise software resources resolve correctly. NVIDIA documentation shows ngc config set as the command used to configure or switch NGC context, including organization and team values, and other NVIDIA installation workflows show NGC configuration prompting for organization, team, and ACE fields. Manually editing JSON configuration files is not the recommended operational method because it increases the risk of syntax errors and inconsistent authentication state. ngc registry login is not the normal command for setting persistent NGC CLI organization context. Exporting an environment variable may influence a session, but it is not the best persistent configuration method. In AI infrastructure operations, correct NGC configuration is important for pulling validated containers, GPU drivers, vGPU software, models, and enterprise assets from the correct account scope.

When evaluating NVIDIA Collective Communications Library (NCCL) performance, it is vital to distinguish between Algorithm Bandwidth and Bus Bandwidth. For an all_reduce operation, the Bus Bandwidth represents the effective data transfer rate across the hardware links, which includes the overhead of the ring or tree collective algorithm. In an NDR (400G) InfiniBand fabric, the theoretical peak per link is 50 GB/s (unidirectional). In a 64-GPU cluster (8 nodes of 8 GPUs), achieving a bus bandwidth of 656 GB/s indicates that the fabric is efficiently utilizing the multiple 400G rails available on the DGX H100. This result is considered optimal as it reflects near-line-rate performance when accounting for network headers and synchronization overhead. Algorithm bandwidth is naturally lower because it represents the " useful " data moved from the application ' s perspective. If the bus bandwidth were significantly lower, it would suggest congestion, cable faults, or sub-optimal routing.

While there are many ways to stress a system, the verified method to check for errors under load in an NVIDIA DGX environment is to monitor the system using NVSM (NVIDIA System Management). Running nvsm show health is the standard command to verify that all hardware components—including GPUs, memory, and storage—are operating within their defined specifications. To truly ensure no errors occur " under load, " an administrator will typically run a workload (like HPL or NCCL tests) and concurrently run nvsm show health or nvsm monitor to check for real-time telemetry such as thermal throttling, PCIe errors, or power fluctuations. nvsm show health aggregates data from the BMC and the OS to provide a " Red/Green " status of the entire system. There is no standard command named nvsm stress-test (Option D) or stress-test --usage (Option B) in the official NVIDIA DGX software stack.

The correct command is df -h | grep ' /var ' because df reports filesystem capacity, used space, available space, and mount utilization. In DGX systems, monitoring disk space is important because system logs, container runtime data, package caches, telemetry, temporary files, and service data can consume space under operational filesystems such as /var. If /var fills up, services such as Docker, containerd, logging, monitoring agents, NVIDIA management tools, or package management can fail or behave unpredictably. nvidia-smi --query-disk-space is incorrect because nvidia-smi reports GPU telemetry, not filesystem capacity. du -sh /home/* summarizes directory usage under /home, but it does not directly show remaining filesystem space or a threshold-ready view of the mounted filesystem. lsof +L1 is useful for finding deleted files still held open by processes, but it is not the primary command for checking available disk capacity. In NVIDIA AI infrastructure, routine OS-level checks like filesystem capacity are part of reliable server operations and help prevent avoidable workload failures.

Deep learning training involves iterating over a dataset many times (epochs). If a 32-node cluster pulls the same dataset from a central NFS storage server for every epoch, the network and storage fabric quickly become a bottleneck due to " Incast " traffic. By using the high-speed NVMe drives internal to a DGX system (configured in RAID-0 for maximum performance, not redundancy), the system can implement a local cache. During the first epoch, data is pulled from the remote storage and simultaneously written to the local SSDs. For all subsequent epochs, the training framework reads the data directly from the local RAID-0 array. This significantly reduces NFS traffic and network congestion, allowing the training to proceed at the full speed of the local NVMe storage ($25\text{ GB/s}+$ on modern DGX systems). Option C is incorrect because RAID-0 provides no redundancy; if a drive fails, the cache is lost, but since it is just a cache, the data still exists on the primary storage. Option B refers to GPUDirect Storage, which is a separate technology from local RAID-0 caching.