Jesse Marczyk in Psychology Today:
Sticking to mammals just for the sake of discussion, males of most species endure less obligate parenting costs than females. What this means is that if a copulation between a male and female results in conception, the female bears the brunt of the biological costs of reproduction. Many males will only provide some of the gametes required for reproduction, while the females must provide the egg, gestate the fetus, birth it, and nurse/care for it for some time. Because the required female investment is substantially larger, females tend to be more selective about which males they're willing to mate with. That said, even though the male's typical investment is far lower than the female's, it's still a metabolically-costly investment: the males need to generate the sperm and seminal fluid required for conception. Testicles need to be grown, resources need to be invested into sperm/semen production, and that fluid needs to be rationed out on a per-ejaculation basis (a drop may be too little, while a cup may be too much). Put simply, males cannot afford to just produce gallons of semen for fun; it should only be produced to the extent that the benefits outweigh the costs.
For this reason, you tend to see that male testicle size varies between species, contingent on the degree of sperm competition typically encountered. For those not familiar, sperm competition refers to the probability that a female will have sperm from more than one male in her reproductive tract at a time when she might conceive. In a concrete sense, this translates into a fertile female mating with two or more males during her fertile window. This creates a context that favors the evolution of greater male investment into sperm production mechanisms, as the more of your sperm are in the fertilization race, the greater your probability of beating the competition and reproducing. When sperm competition is rare (or absent), however, males need not invest as many resources into mechanisms for producing testes and they are, accordingly, smaller.
More here.