iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Use a dedicated GPU node pool when you need a particular accelerator, exclusive device access, predictable performance, or an independent scaling boundary. Use GPU sharing when several compatible workloads can safely use one device and improving utilization matters more than strict isolation. In many production clusters, the best design combines both: a pool selects the right machines, while MIG, MPS, or time-slicing determines how those machines are divided.
These are different decisions, not competing features. A node pool answers “which GPU-equipped machines may run this workload?” Sharing answers “how may multiple workloads use a GPU on one machine?” Keeping those layers separate makes capacity planning, scheduling, security reviews, and troubleshooting much clearer.
The short decision: pool first, sharing second
Start by classifying the workload, then choose the smallest architecture that satisfies its isolation and performance requirements.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minute| Workload condition | Recommended starting point | Reason |
|---|---|---|
| Large training or inference job that needs the whole accelerator | Dedicated whole-GPU allocation in a suitable GPU node pool | Exclusive access is easier to reason about and gives the most predictable performance. |
| Several trusted, bursty jobs with modest utilization | GPU sharing inside a GPU node pool | Sharing can expose otherwise idle capacity, provided interference is acceptable. |
| Strong workload-to-workload isolation on supported hardware | MIG partitions in a GPU node pool | MIG provides hardware-level instances rather than merely interleaving processes. |
| Concurrent processes that need configured compute and memory limits | MPS, where the NVIDIA software path supports it | The NVIDIA device plugin documents MPS as a mode with per-workload resource controls. |
| Older or non-MIG-capable GPUs and tolerant workloads | Time-slicing | It can share access by interleaving work, but does not provide MIG-like memory or fault isolation. |
| Different GPU models, zones, drivers, or scaling policies | Separate pools, optionally with sharing within each pool | Pool boundaries keep hardware and lifecycle choices explicit while sharing improves utilization inside a boundary. |
Do not treat a pool as a security wall. Separate-node scheduling creates placement separation, but taints, labels, and pools alone do not prevent every privilege-escalation path. Google Cloud explicitly warns that dedicated pools can affect cost efficiency, autoscaling, and utilization, and that additional controls are needed for isolation. See the GKE dedicated-node guidance.
#1 Best Overall
- NVIDIA Volta GV100 Architecture — 4,608 CUDA Cores, 640 1st-Gen Tensor Cores delivering 14 TFLOPS FP32 and 112 TFLOPS deep learning performance for AI training, inference, HPC, and scientific computing workloads
- 32GB HBM2 ECC Memory — 900 GB/s Bandwidth — High-bandwidth memory on a 4096-bit bus with ECC error correction provides the memory capacity and throughput required for the largest AI models, simulations, and datasets
- PCIe 3.0 x16 Interface — 250W TDP — Standard PCIe Gen3 connectivity with passive cooling designed for enterprise rack server deployment in HPE ProLiant, Dell PowerEdge, and Supermicro platforms with adequate chassis airflow
- NVLink — Scale to 96GB Unified Memory — Connect two V100 GPUs via NVLink at 300 GB/s bi-directional bandwidth to scale GPU memory from 32GB to 96GB for larger AI training and HPC workloads
- Multi-Precision Computing — Supports FP64 (7 TFLOPS), FP32 (14 TFLOPS), FP16 (112 TFLOPS) and INT8 precision modes for flexible deployment across training, inference, and scientific simulation workloads
What Kubernetes is actually scheduling
Kubernetes does not understand a vendor GPU by itself. A vendor device plugin discovers devices on each node and advertises an extended resource, commonly nvidia.com/gpu. Pods request that resource, and the scheduler places them only on nodes advertising enough capacity. The device-plugin mechanism is described in the Kubernetes device plugins documentation.
GPU requests work together with placement constraints. A pod can request a GPU resource while also requiring a node label for an accelerator type, a zone, or another operational characteristic. A taint can keep ordinary pods off GPU nodes; a matching toleration allows the intended GPU workload to enter. The request alone does not guarantee a specific model, zone, driver version, or performance level.
Kubernetes GPU scheduling has been stable since v1.26, according to the Kubernetes GPU scheduling documentation. The cluster still needs compatible drivers and a functioning device plugin. If those components are absent or unhealthy, the scheduler cannot expose or allocate the GPU resource, regardless of the node’s physical hardware.
What a GPU node pool controls
A GPU node pool is a capacity and placement boundary. It groups machines with a chosen accelerator configuration so that they can be provisioned, labeled, tainted, upgraded, and scaled as a unit. In provider implementations such as Google Kubernetes Engine (GKE), separate pools can be used for different accelerator types and can have their own autoscaling behavior. That is provider guidance, not a universal requirement imposed by Kubernetes.
Use a separate pool for hardware differences
Different GPU models can have materially different memory capacity, supported partitioning modes, driver requirements, and availability by zone. A workload that needs one model should target a pool labeled for that model instead of relying on a generic “GPU present” label. If a model is available only in selected zones, restrict the pool to those zones; otherwise the scheduler may see an apparently suitable request that cannot be provisioned.
Use taints and tolerations for placement hygiene
Tainting a GPU pool prevents unrelated pods from consuming expensive nodes. Add a matching toleration only to workloads intended for that pool, then use node labels or affinity to select the required accelerator class. This protects capacity from accidental placement, but it is an operational control rather than a complete security boundary.
Plan autoscaling around startup time
GPU nodes are expensive to keep idle and can take longer to provision than CPU nodes. A pool autoscaler can add capacity for queued work, but your queueing policy must tolerate node startup and driver initialization time. Separate pools let a burst of one accelerator type scale without forcing every GPU class to scale together.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Account for driver lifecycle
Drivers, the device plugin, and the container runtime must remain compatible with the control-plane and node software. GKE’s current documentation describes version-dependent NVIDIA driver installation behavior, so verify the instructions for the control-plane version and accelerator type you actually run rather than copying a procedure from another release. The current provider guidance is at GKE GPU node pools.
What GPU sharing changes
Sharing changes allocation on an underlying device or hardware partition. It does not select the machine, zone, or GPU model; those remain node-pool and placement decisions. Sharing can increase access to a device that would otherwise sit idle, but no universal utilization percentage or cost break-even point is established. Results depend on concurrency, memory footprints, GPU model, provider pricing, and interference between jobs.
Time-slicing
Time-slicing interleaves GPU work from multiple replicas or processes. It can support GPUs that do not offer MIG, making it useful when compatible workloads need opportunistic access to older hardware. The trade-off is that replicas share the same memory and fault domain. NVIDIA’s GPU Operator documentation states: “Unlike Multi-Instance GPU (MIG), there is no memory or fault-isolation between replicas, but for some workloads this is better than not being able to share at all.” See the versioned NVIDIA GPU sharing documentation.
Do not interpret a request for multiple shared replicas as a fractional-GPU performance guarantee. More replicas create scheduling entitlements, not a promise of proportionally more compute. A memory-heavy process can interfere with another process, and a fault or runaway allocation can affect the shared device. Use time-slicing only when workloads are compatible, trusted, and tolerant of variable latency.
MIG (Multi-Instance GPU)
MIG divides a supported accelerator into hardware-level instances. Those instances provide stronger separation of compute and memory resources than time-slicing, making MIG the sharing choice when workload isolation and predictable partition capacity matter. MIG profiles are predefined by the hardware; you select an available profile rather than requesting an arbitrary percentage of a GPU.
NVIDIA’s DRA guide identifies Ampere-or-newer NVIDIA data-center GPUs as MIG-capable, but exact support also depends on the model, operating mode, drivers, and software stack. Confirm the GPU and mode before designing scheduling or capacity plans around MIG. The hardware-specific guidance is in the NVIDIA DRA MIG guide.
MPS
MPS is a separate NVIDIA sharing mode for concurrent processes. The NVIDIA device plugin documents MPS as managing access while permitting compute and memory limits per workload. This can be appropriate when processes need controlled concurrency but the available hardware and software stack do not fit a MIG design.
Rank #3
- Colour: brown
- Brand: Nvidia
- Packed with features
- Best product in its class
MPS and time-slicing are mutually exclusive configuration choices in the NVIDIA device plugin. Check the device-plugin or GPU Operator version, CUDA compatibility, and supported GPU behavior before selecting MPS. Limits described by configuration are not a substitute for testing real interference: measure latency, throughput, memory pressure, and failure behavior with representative processes. The plugin’s current documented behavior is in its NVIDIA device-plugin README.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Choose by workload behavior
Large training jobs
Use dedicated whole-GPU allocation when a training job needs the device’s full memory or sustained throughput. Sharing adds contention and makes run time less predictable. If a training job can be decomposed into independent, smaller workers, evaluate MIG profiles individually, but ensure each profile has enough memory for the model, framework overhead, and batch size. A partition that looks large enough on paper can still fail when runtime allocations and data loading are included.
Latency-sensitive inference
Start with dedicated GPUs when tail latency and consistent throughput are contractual requirements. Time-slicing can introduce queueing and jitter, while a memory or fault event in one replica can affect others. MIG may be suitable for independent inference services when a supported profile meets the model’s memory and compute needs. Benchmark p95 and p99 latency under concurrent load instead of inferring performance from the number of replicas.
Development, notebooks, and batch jobs
Time-slicing is often a practical fit for trusted development or batch workloads that can tolerate variable completion time. Taints and tolerations keep general cluster traffic away from the pool, while sharing lets more users access a limited number of devices. Apply quotas and admission policies appropriate to your platform; sharing alone does not prevent a user from submitting too many requests.
Mutually untrusted tenants
Do not rely on time-slicing or a node-pool label as a tenant-security mechanism. Shared memory and fault domains increase the consequence of unsafe or buggy code. Consider hardware-isolated MIG instances where supported, and apply Kubernetes security controls, identity boundaries, runtime hardening, and network policies separately. A dedicated pool can reduce accidental co-location, but it does not by itself close every privilege-escalation path.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Software that assumes exclusive access
Some applications initialize all visible memory, depend on exclusive device mode, or behave poorly when another process competes for context time. Test these applications on the intended sharing mode; if they fail or become unpredictable, assign a whole GPU in a suitable pool.
How to combine pools and sharing
Combining the mechanisms is normal. For example, create one pool for a particular accelerator generation and zone set, then expose MIG instances to several inference services. Create another pool for older GPUs and use time-slicing for development jobs. Keep the pools separate when their drivers, autoscaling policies, maintenance windows, or trust levels differ.
Rank #4
- Bulk Pack without retail box
- Inventory the workload. Record peak and steady-state memory, compute demand, latency targets, concurrency, failure tolerance, and whether jobs are mutually trusted.
- Inventory the hardware. Confirm the exact GPU model, zones where it is available, supported driver versions, and whether MIG is supported in the required mode.
- Define pool boundaries. Group nodes only when they share accelerator, zone, lifecycle, and operational requirements. Plan independent autoscaling where demand differs.
- Install and verify the device path. Install the vendor driver and device plugin, then verify that nodes advertise the expected extended resource. Follow the provider’s version-specific installation behavior.
- Apply placement controls. Label accelerator types, taint GPU nodes, add tolerations to intended pods, and use affinity or selectors so a workload cannot land on an unsuitable model or zone.
- Select one sharing mode per device-plugin configuration. Choose MIG, MPS, or time-slicing according to isolation and control requirements. Do not configure MPS and time-slicing together when using the NVIDIA device plugin.
- Benchmark representative contention. Run the real framework, model, batch size, and concurrency. Capture throughput, memory errors, tail latency, completion time, and behavior when one workload crashes.
- Set operational limits. Establish quotas, queue policies, autoscaler limits, maintenance procedures, and alerts for pending GPU pods, allocation failures, memory pressure, and device-plugin health.
A pod should request the resource exposed by the plugin rather than assuming that a generic CPU or memory request reserves a GPU. The resource request, node-placement rules, toleration, and sharing configuration must agree; a mismatch commonly leaves pods pending or places them on an unsuitable class of node.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Isolation, reliability, and security trade-offs
| Choice | Isolation characteristics | Operational implication |
|---|---|---|
| Dedicated whole GPU | Strongest device exclusivity and clearest performance expectations | Idle capacity is possible, and the node remains an expensive unit to autoscale. |
| Dedicated node pool | Placement and lifecycle separation, not a complete security boundary | Requires labels, taints, tolerations, autoscaling, and pool maintenance. |
| MIG | Hardware-level partitioning on supported GPUs | Capacity is constrained by predefined profiles and model support. |
| MPS | Plugin-documented compute and memory controls for concurrent processes | Requires compatible NVIDIA software and validation of actual interference. |
| Time-slicing | No MIG-equivalent memory or fault isolation between replicas | Simple access to shared capacity, but variable performance and shared failure impact must be accepted. |
Isolation also affects incident response. With time-slicing, a device-level fault or memory problem can involve every replica on that GPU. With MIG, an instance boundary narrows the blast radius, but the node, driver, and control-plane components are still shared. Dedicated nodes reduce co-location but do not replace workload identity, container security, or least-privilege configuration.
Common failure modes and corrections
GPU pods remain Pending
- Check that the driver and device plugin are running on the node.
- Confirm the pod requests the exact extended resource advertised by the plugin.
- Inspect taints, tolerations, labels, affinity, and zone constraints for contradictions.
- Verify that the selected GPU model is available in the pool’s zones and that autoscaling is allowed to create a node.
A shared workload is slower than expected
Check whether another replica is consuming memory or context time. With time-slicing, multiple replicas do not guarantee proportional compute. Reproduce the workload at the intended concurrency, then move it to a less contended profile, MPS configuration, MIG instance, or whole GPU as appropriate.
MIG configuration is unavailable
Confirm the exact GPU model and whether it is an Ampere-or-newer NVIDIA data-center GPU, then verify operating mode, driver, operator, and plugin support. A generic “NVIDIA GPU” label does not prove MIG capability.
Security expectations are too high
Review whether the design treats a pool, taint, or sharing replica as a tenant boundary. Add Kubernetes security controls and stronger hardware isolation where the threat model requires it. Dedicated-node guidance from Google Cloud specifically cautions that separate pools affect efficiency and do not provide every isolation guarantee.
Configuration behaves differently after an upgrade
Pin and document the device-plugin or GPU Operator version, driver version, GPU model, and sharing mode. NVIDIA behavior is version- and hardware-dependent; consult the matching versioned documentation rather than assuming that a configuration from another release has identical semantics.
Recommended Free Tools
Cost and utilization without false precision
A whole-GPU pool is easier to budget per job but can leave expensive nodes idle between bursts. Sharing can raise utilization when jobs are small or intermittent, yet contention can increase completion time and operational complexity. There is no universal utilization improvement or break-even price. Compare the actual provider price for the exact GPU, zone, node shape, attached storage, and autoscaling behavior against measured workload demand.
Use observed data rather than replica counts: GPU memory occupancy, compute utilization, queue wait, job duration, tail latency, eviction or retry rate, and node idle time. A design that appears cheaper per pod can cost more if interference forces retries or requires a larger pool to meet latency targets.
A practical selection checklist
- Do you need a specific accelerator model, zone, driver, or lifecycle? Create or target a matching node pool.
- Does the workload require the entire device or exclusive access? Allocate a whole GPU.
- Can trusted workloads tolerate variable latency and a shared memory/fault domain? Consider time-slicing.
- Does the exact GPU support MIG, and does a predefined profile fit the workload’s memory and compute needs? Consider MIG.
- Do concurrent processes need configured compute and memory limits, and does your NVIDIA software stack support MPS? Test MPS.
- Are tenants mutually untrusted? Treat node pools and time-slicing as insufficient by themselves; evaluate stronger isolation and Kubernetes security controls.
- Can the platform provision the GPU in the required zones, with compatible drivers and a healthy device plugin?
- Have you benchmarked representative jobs at realistic concurrency and captured tail behavior?
Bottom line
Choose a GPU node pool to control machine type, placement, lifecycle, and scaling. Choose GPU sharing to control how workloads divide a device after they reach that machine. Dedicated whole-GPU allocation is the safest default for full-device, sensitive, or latency-critical jobs. MIG is the strongest sharing option when supported hardware and predefined profiles fit. MPS is for controlled concurrent processes on a compatible NVIDIA stack. Time-slicing is the pragmatic option for trusted workloads that value access and utilization more than strict isolation or proportional performance. In a mature cluster, these choices coexist: specialized pools provide the right hardware, and an appropriate sharing mode fills otherwise idle capacity without pretending that all GPU fractions behave alike.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Free tools Windows power users keep installed
One-click scans. No signup required.
