Three reasons to add another device
Independent experiments can run on separate devices with little interaction. Data-parallel training distributes work while exchanging updates. Model-parallel execution splits a model and can require frequent communication. Those cases impose different network and software requirements.
Write down which case you have before requesting more devices. An application that launches four copies of a model is not performing the same work as a model that needs four devices to fit.
Measure scaling efficiency
If one GPU finishes a job in eight hours and four GPUs finish it in three, the elapsed-time speedup is about 2.67. The four-GPU run consumes twelve GPU-hours instead of eight. At equal per-GPU rates, it costs more even though it finishes sooner.
These are illustrative numbers. They make the purchasing question explicit: is finishing five hours earlier worth the extra allocation? A deadline may justify it; a flexible experiment may not.
| Configuration | Elapsed time | GPU-hours |
|---|---|---|
| 1 GPU | 8 hours | 8 |
| 4 GPUs | 3 hours | 12 |
Specify the actual placement
Ask whether devices share a physical host and what connects them. Modal documents that its multi-GPU function allocations are on one physical machine. Other products and cluster arrangements need their own topology checks.
For multi-node work, benchmark the communication path and shared storage with the intended software. A test on a different network can create a false expectation of scaling.
References: Modal GPU configuration
Try the cheaper bottleneck fix first
Profile input loading, CPU preprocessing and checkpoint writes. If the GPU is waiting on those stages, adding more GPUs can amplify the bottleneck. Improve the pipeline and repeat the single-device measurement.
When memory is the constraint, compare a larger-memory device with a distributed configuration. The simpler deployment can be preferable even when its hourly rate is higher, if it reduces engineering and failure-recovery work.
Questions readers ask
Do four GPUs provide four times the performance?
Only in some well-parallelized workloads. Communication, input processing and serial work limit scaling.
Should I optimize for elapsed time or GPU-hours?
Track both. The deadline determines how much extra cost is justified for an earlier result.
Sources and editorial notes
Primary documentation reviewed October 9, 2026. Provider descriptions reflect published documentation. Recommendations are our analysis; worked figures are illustrative unless explicitly stated otherwise. No independent provider benchmark or uptime ranking is claimed.
Read the editorial policy. NeuraMesh is a prototype, not live verified GPU inventory.