Tensor split
Also known as: layer split · multi-GPU split
Tensor split (layer/weight split) divides a model’s layers across multiple GPUs — or between GPU and CPU — so a model that’s too big for one device can still run.
- Split 70B across two 24 GB cards.
- Or offload a fraction of layers to CPU.
- The per-token speed reflects the slowest device in the chain.
The "make it fit" tool: when one GPU isn't enough, tensor split is how you stretch a big model across what you have — at the cost of the slowest link.