Objective The aim of this study was to evaluate the rates of fetal mortality in a Peruvian hospital between 2001 and 2020 and to investigate the association of indicators of social inequality (such as access to prenatal care and education) with fetal mortality. Methodology We conducted a retrospective cohort study, including all pregnant women who attended a Peruvian hospital between 2001 and 2020. We collected data from the hospital’s perinatal computer system. We used Poisson regression models with robust variance to assess the associations of interest, estimating adjusted relative risks (aRR) and their 95% confidence intervals (95% CI). Results We analyzed data from 67,908 pregnant women (median age: 26, range: 21 to 31 years). Of these, 58.3% had one or more comorbidities; the most frequent comorbidities were anemia (33.3%) and urinary tract infection (26.3%). The fetal mortality ratio during the study period was 0.96%, with the highest rate in 2003 (13.7 per 1,000 births) and the lowest in 2016 (6.1 per 1,000 births), without showing a marked trend. Having less than six (aRR: 4.87; 95% CI: 3.99–5.93) or no (aRR: 7.79; 6.31–9.61) prenatal care was associated with higher fetal mortality compared to having six or more check-ups. On the other hand, higher levels of education, such as secondary education (aRR: 0.73; 0.59–0.91), technical college (aRR: 0.63; 0.46–0.85), or university education (aRR: 0.38; 0.25–0.57) were associated with a lower risk of fetal death compared to having primary education or no education. In addition, a more recent year of delivery was associated with lower fetal mortality. Conclusion Our study presents findings of fetal mortality rates that are comparable to those observed in Peru in 2015, but higher than the estimated rates for other Latin American countries. A more recent year of delivery was associated with lower fetal mortality, probably due to reduced illiteracy and increased access to health care between 2000 and 2015. The findings suggest a significant association between indicators of social inequality (such as access to prenatal care and education) with fetal mortality. These results emphasize the critical need to address the social and structural determinants of health, as well as to mitigate health inequities, to effectively reduce fetal mortality.