Weight initialization

Weight initialization is how a model’s weights are set before any training. It’s not random chaos — it’s a deliberate scheme (e.g., scaled small values) chosen so that signals neither vanish nor explode as they pass through layers.

Good initialization is a quiet prerequisite for training to work at all.

• Too large → signals explode.
• Too small → signals vanish.