iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
NVLink setup starts with hardware, not a software switch. First identify the exact Tesla V100 form factor and server topology, then confirm that the GPUs have a supported physical NVLink path. On the installed system, use NVIDIA System Management Interface (nvidia-smi) to inspect topology and peer-to-peer capability. If the GPUs are assigned through vGPU, validate the vGPU profile, hypervisor, and guest operating system as one supported combination. Finally, configure multi-GPU execution in your inference framework and benchmark the real model. A topology report alone does not prove that every link is active or predict application performance.
This guide covers bare-metal, PCIe-passthrough, and vGPU deployments without assuming a particular server, driver, CUDA release, or inference framework. Those details determine which physical connections and software settings are available.
What NVLink changes in a V100 inference system
NVLink is a high-speed hardware interconnect between compatible NVIDIA GPUs. In a multi-GPU inference workload, it can provide a direct peer path for data transfers instead of forcing all traffic through a host-side path. That can matter when a model is split across GPUs, when activations or parameters must move between devices, or when a framework exchanges data during distributed execution.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
NVLink does not automatically combine two V100s into one larger GPU. It does not select devices for an inference server, enable model parallelism, or guarantee lower latency. The framework still has to place the model and tensors, establish peer access where supported, and schedule work across the devices. A system can report an NVLink-capable topology while an application uses a different path or leaves the link idle.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
There is no universal V100 inference-speedup percentage that applies to every model. The result depends on model partitioning, batch size, precision, transfer volume, GPU placement, software versions, and the particular host. Measure the workload you actually intend to run.
Confirm the V100 hardware and deployment mode
Before installing software or changing framework settings, record what is physically installed and how it is exposed to the operating system. The same V100 name covers different board and platform arrangements, and those arrangements do not share an identical NVLink topology.
Form factor matters
NVIDIA’s cited vGPU support documentation specifically lists Tesla V100 SXM2 and Tesla V100 SXM2 32GB configurations for NVLink peer-to-peer. That statement is not a promise that every V100 PCIe card, SXM board, or server supports the same links. A V100’s memory capacity or product name by itself is not enough to determine connectivity.
Use the board label, the server’s bill of materials, and the system vendor’s platform documentation to establish the exact model. Do not buy a generic V100 and assume that placing it in any server adds NVLink. The chassis, baseboard, bridge or connector arrangement, cooling, power delivery, firmware, and supported GPU population all have to match the vendor’s design.
Classify the host path
| Deployment | What must be verified | Important limitation |
|---|---|---|
| Bare metal | Physical GPU form factor, server topology, bridge or direct-link arrangement, driver, CUDA stack, and framework support | CUDA and the framework still need to use peer access; an NVLink path does not configure inference by itself |
| PCIe passthrough | Physical host connections, IOMMU and passthrough design, guest driver support, and the GPU assignment | The guest view may differ from the host’s complete topology; verify what the inference process can actually access |
| vGPU | Physical V100 type, vGPU profile, hypervisor release, guest OS, NVIDIA software versions, and supported peer path | The documented V100 vGPU NVLink path is narrower than a general bare-metal assumption and is version dependent |
Keep these categories separate when troubleshooting. A statement about a validated vGPU configuration should not be generalized to all bare-metal CUDA installations, and a physical link visible on the host does not prove that a virtual machine can use it.
Prerequisites checklist
- Exact V100 board and form factor, including whether it is SXM2, SXM2 32GB, or another configuration.
- Server model and vendor topology documentation, including the intended GPU population and physical interconnect arrangement.
- Deployment type: bare metal, PCIe passthrough, or vGPU.
- Installed NVIDIA driver and CUDA versions, plus the inference framework and its multi-GPU feature set.
- For vGPU, the vGPU profile, hypervisor release, guest operating system, and host software versions.
- Administrative access to run
nvidia-smion the host or guest where inference will run. - A representative model, fixed test inputs, and a way to collect latency, throughput, errors, and GPU utilization.
If any item is unknown, resolve it before interpreting a topology result. A command can be syntactically correct while the underlying deployment is unsupported.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
Step 1: inventory the installed GPUs
Start on the machine that will run inference. Record the GPUs visible to that operating system and the identifiers used by your framework. A minimal inventory command is:
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →nvidia-smiSave the output with the date and software versions. Confirm that every intended GPU appears, that the driver is loaded, and that no device is in an error or unavailable state. In a virtual machine, this output describes the devices assigned to that guest, not necessarily every physical GPU in the server.
Then document the mapping between physical slots, GPU indices, UUIDs, and any framework-specific device names. Stable UUID-based selection is often safer than assuming that index 0 is always the same physical board, especially after hardware changes or virtualization reconfiguration.
Step 2: inspect the GPU topology
NVIDIA documents the following command for viewing relationships among GPUs and other system components:
nvidia-smi topo -mRun it on the host for a bare-metal deployment. For a vGPU deployment, run it in the guest as well as on the host when you have access to both views. Read the GPU-to-GPU entries and note whether the reported path is a direct NVLink relationship, a host-side path, or another topology label. The exact symbols and labels can vary with driver and platform, so interpret them using the output’s legend and the documentation for the installed release.
The matrix answers a topology question: what relationship does the NVIDIA software report between these devices? It is not a bandwidth test, does not show how much traffic an inference job is generating, and does not by itself prove that every physical link is currently active.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
Step 3: query peer-to-peer capability
Use NVIDIA’s peer-to-peer query to check whether the visible GPU pairs report NVLink P2P capability:
nvidia-smi topo -p2p nCompare the result with the GPU pairs you intend to use. A positive capability result is an important prerequisite, but it is not equivalent to application-level peer access. The framework, process permissions, device placement, and runtime configuration still determine whether transfers use that path.
Keep the command output with your deployment record. If the topology matrix and P2P query disagree with the server documentation, stop before tuning inference. Recheck which machine you queried, which GPUs are assigned, and whether the driver is exposing the intended devices.
Step 4: apply the correct support rules for vGPU
Virtualization adds a support matrix that cannot be inferred from the physical cards alone. Verify the GPU model, vGPU profile, hypervisor release, and guest operating system together. NVIDIA’s validated guidance for the cited V100 vGPU NVLink P2P path names V100 SXM2 configurations, Linux guest support, and direct connections. It also states limitations for that path, including no NVSwitch and no PCIe P2P.
Those conditions apply to the documented vGPU scenario; they are not a blanket rule for every V100 deployment. If your design uses another guest operating system, a different V100 form factor, an indirect connection, NVSwitch, or PCIe peer-to-peer, consult the support matrix for the exact release instead of assuming equivalence.
Host and guest checks
- On the host, confirm that the physical GPUs and their interconnects match the platform design.
- Confirm that the selected vGPU profile is one of the profiles supported for the intended peer path.
- Verify the hypervisor release and host NVIDIA software against the same support documentation.
- Verify the guest operating system and guest driver or vGPU software version.
- Inside the guest, run
nvidia-smi,nvidia-smi topo -m, andnvidia-smi topo -p2p n. - Use only the GPUs and peer relationships visible and supported in that guest; do not infer guest capability from a host-only matrix.
NVIDIA’s Virtual GPU Software User Guide directs administrators to determine NVLink topology between physical GPUs or vGPUs by running nvidia-smi topo -m. Treat that as the topology inspection step, not as a guarantee that a guest application will achieve a particular transfer rate.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
Step 5: configure multi-GPU inference separately from NVLink
After hardware and topology validation, configure the inference software. NVLink does not select GPUs or turn on a framework’s distributed mode. The exact commands depend on the framework and serving architecture, which are not specified here, so use the framework’s supported mechanism for one of these patterns:
Recommended Free Tools
Model parallelism
Partition one model across multiple GPUs when the model or its working memory does not fit efficiently on one device, or when the framework supports splitting layers and tensors. Define the device map explicitly, confirm that the intended neighboring stages are on the GPUs with the direct path, and check for unintended CPU transfers. A topology-aware placement can reduce unnecessary movement, but the framework must expose and honor it.
Data parallelism or replicated inference
Load a copy of the model on each GPU and distribute independent requests or batches. This can improve aggregate throughput when requests are sufficiently independent, but it may not reduce single-request latency. Ensure that requests are partitioned consistently and that synchronization or result gathering does not become the bottleneck.
Pipeline or distributed execution
Some serving systems use separate processes or stages. In that design, verify which process owns each GPU, how inter-process communication is configured, and whether the runtime can use CUDA peer access. A process-level configuration that lists two device IDs is not proof that transfers will use NVLink.
Keep placement reproducible
Record the GPU UUIDs or explicit device order, model revision, precision mode, batch size, sequence lengths, concurrency, and software versions. Run the same configuration on every comparison. Changing placement, precision, or batching at the same time as enabling multiple GPUs makes the result impossible to attribute to NVLink.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Step 6: verify behavior with a representative benchmark
Benchmark the actual inference workload rather than relying on topology output or a theoretical link specification. Establish a single-GPU baseline first, then compare the same model and inputs with the intended multi-GPU configuration.
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
- Warm up the model until initialization, compilation, and memory allocation are no longer included in the measured steady state.
- Use the same model files, input shapes, precision, batch size, concurrency, and request mix for each run.
- Record end-to-end latency and throughput, and distinguish first-request latency from steady-state latency.
- Record GPU utilization, memory use, errors, and any host-to-device or device-to-host transfer behavior exposed by your monitoring tools.
- Repeat enough times to see run-to-run variation, then retain the raw logs with the topology and software inventory.
- Compare the result with a configuration that deliberately uses one GPU or a non-NVLink path when feasible, so you can identify whether the multi-GPU arrangement helps this workload.
Do not publish or plan around an assumed percentage improvement. The available official guidance establishes how to inspect topology and capability, not a universal V100 inference result. A model with little cross-GPU communication may see little benefit, while a partitioned model with frequent transfers may be more sensitive to placement and link availability.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to interpret common command results
| Observation | What it establishes | What it does not establish | Next action |
|---|---|---|---|
GPU pair shows an NVLink relationship in topo -m |
The driver reports that relationship in the queried environment | That every link is active, that the application uses it, or that performance will improve | Check peer capability, link status, and a real inference benchmark |
topo -p2p n reports peer capability |
The queried pair is reported as capable for the requested P2P category | That your framework has enabled peer access or placed tensors on that pair | Verify framework placement and monitor the workload |
| Only a host-side path is reported | The visible topology does not show the expected direct NVLink relationship | Whether the physical link is absent, unsupported in the current software view, or hidden by virtualization | Check form factor, cabling or bridge arrangement, host documentation, and guest assignment |
| Host shows a link but guest does not | The guest is not being presented with the same topology | That the guest can use the host link | Review vGPU profile, hypervisor support, guest OS, and the documented direct-connection requirements |
| Topology looks correct but inference is not faster | Topology alone is insufficient to explain application performance | That NVLink is faulty | Profile transfers, placement, synchronization, batching, and link activity before changing hardware |
Check link state and diagnostics separately
Topology documentation describes known relationships; it is not a live bandwidth measurement or proof that each link is up. For unexpected results, use the link-status and diagnostic facilities appropriate to the installed NVIDIA software and platform. Compare those results with the host’s physical documentation and the GPU error state.
When diagnostics are clean but the workload behaves unexpectedly, inspect the application path. A framework may copy through host memory, place communicating layers on nonadjacent GPUs, synchronize too often, or spend most of its time in computation rather than transfer. Those are application and placement issues, not evidence that an NVLink switch is missing.
Troubleshooting by symptom
The second V100 is not visible
- Confirm the card is installed in a supported slot and appears in the host inventory.
- Check power, cooling, firmware, and the server’s supported GPU population.
- Verify that the driver sees both devices and that passthrough or vGPU assignment includes the intended GPU.
- Re-run topology commands on the machine where inference actually runs.
No direct NVLink relationship appears
- Reconfirm the exact form factor; do not assume a PCIe V100 has the same connectivity as an SXM2 configuration.
- Check the server vendor’s required bridge or direct-link arrangement and supported slot combinations.
- Confirm that the host is not reporting a virtualized or restricted view.
- For vGPU, validate the profile, hypervisor, guest OS, and direct-connection requirement as one combination.
P2P capability is absent
- Check that you ran the command on the intended host or guest and that both GPUs are visible there.
- Compare the result with the supported matrix for the exact V100 form factor and software release.
- Review driver state and any virtualization restrictions.
- Do not force a framework configuration that assumes peer access until the platform reports the required capability.
Topology is correct but throughput is poor
- Compare against a controlled single-GPU baseline.
- Check whether the model is replicated or partitioned as intended.
- Measure synchronization, queueing, host transfers, and request batching.
- Verify that communicating stages use the GPUs with the direct path.
- Check link activity and diagnostics independently; do not treat the matrix as a performance result.
Results differ after reboot or redeployment
- Compare GPU UUID-to-index mapping rather than relying on index order.
- Check that the same vGPU profile, hypervisor placement, guest assignment, driver, and framework configuration were restored.
- Save a fresh topology matrix and software inventory before comparing benchmark logs.
Bare metal versus vGPU: choosing the validation path
| Decision axis | Bare metal or passthrough | vGPU |
|---|---|---|
| Primary dependency | Physical board and host topology plus CUDA/framework support | Physical topology plus vGPU profile, hypervisor, guest OS, and release-specific support |
| Topology to inspect | Host operating system where inference runs | Host and guest views, with the guest view governing the application |
| Peer-path assumption | Must be confirmed for the exact GPU and server | Must be confirmed for the exact documented virtualized combination |
| Known V100 vGPU limitation in the cited guidance | Not applicable as a vGPU rule | Named path is for supported V100 SXM2 configurations, Linux guests, and direct connections; it excludes PCIe P2P and NVSwitch for that path |
| Performance conclusion | Requires a workload benchmark | Requires a workload benchmark inside the guest, not just host topology |
Choose the path that matches your operational constraints, then validate it end to end. There is no software-only conversion from an unsupported physical or virtual topology into an NVLink-capable one.
A repeatable validation record
For each machine or VM, keep a short record containing:
- Server model and slot population.
- V100 model and form factor for every GPU.
- Deployment mode and, where relevant, hypervisor and vGPU profile.
- Guest operating system and host and guest NVIDIA software versions.
- Output from
nvidia-smi,nvidia-smi topo -m, andnvidia-smi topo -p2p n. - Link-status or diagnostic results collected separately from topology.
- Inference framework, model version, precision, batch size, concurrency, GPU placement, and benchmark logs.
- Any differences between host and guest topology views.
This record makes a hardware, virtualization, driver, or application regression easier to isolate. It also prevents a topology screenshot from being mistaken for a complete performance qualification.
Final setup checklist
- Identify the exact Tesla V100 form factor and server topology.
- Classify the installation as bare metal, passthrough, or vGPU.
- Confirm that the physical arrangement and, for vGPU, the profile, hypervisor, and guest OS are supported together.
- Run
nvidia-smiand verify every intended GPU is visible and healthy. - Run
nvidia-smi topo -mand inspect the GPU-to-GPU paths. - Run
nvidia-smi topo -p2p nfor the pairs your model will use. - Check link status and diagnostics separately when the result is unexpected.
- Configure model parallelism, data parallelism, or the framework’s supported distribution mode independently of NVLink detection.
- Benchmark the real model with controlled placement, precision, batch size, and concurrency.
- Keep the topology, software inventory, diagnostics, and benchmark logs together for future troubleshooting.
The Bottom Line
For Tesla V100 multi-GPU inference, NVLink is established by a supported physical and platform topology, then verified with nvidia-smi topo -m and nvidia-smi topo -p2p n. Configure the inference framework separately, validate vGPU conditions when applicable, and trust measured workload results—not topology output alone—to determine whether the arrangement helps.
Free tools Windows power users keep installed
One-click scans. No signup required.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
