Dense model
Also known as: dense LLM
A dense model is the classic LLM design: every token passes through all the model’s parameters.
- Active parameters = total parameters.
- Slower per token as size grows.
- The baseline that MoE architectures improve on.
The contrast: dense = "everything, every time"; MoE = "only the relevant experts". Same total size, very different speed.