Cadmium (Cd) is a ubiquitous environmental contaminant, a known carcinogen, and understudied as a developmental toxicant. In the present study, we examined the relationships between Cd levels during pregnancy and infant birth outcomes in a prospective pregnancy cohort in Durham, North Carolina. The study participants (n = 1027) had a mean Cd level of 0.46 µg/L with a range of <0.08 to 2.52 µg/L. Multivariable models were used to establish relationships between blood Cd tertiles and fetal growth parameters, namely birth weight, low birth weight, birth weight percentile by gestational age, small for gestational age, pre-term birth, length, and head circumference. In multivariable models, high maternal blood Cd levels (≥0.50 µg/L) during pregnancy were inversely associated with birth weight percentile by gestational age (p = 0.007) and associated with increased odds of infants being born small for gestational age (p<0.001). These observed effects were independent of cotinine-defined smoking status. The results from this study provide further evidence of health risks associated with early life exposure to Cd among a large pregnancy cohort.