Babies are tiny.
They have to be tiny for a few reasons.
First, their heads have to fit through the birth canal, and second it is less taxing on the mother to have the baby do most of its developing outside of her body. Having two sets of teeth like we do (the technical term is diphyodonty) allows us to have no teeth at birth and lets our jaw grow immediately after birth while we feed on milk.
Then we get baby teeth around the age where we need them to process food. These teeth fit together just right (precise occlusion) so that the cutting and grinding functions work correctly. As we grow, can accommodate larger teeth and if we kept the baby teeth they wouldn’t continue to fit together correctly. So in order to keep our teeth fitting together right, we need a new set that fits our adult jaw.
For more on this, look into the evolution of lactation, followed by diphyodonty, followed by precise occlusion.