Trying to conceive is, in a lot of ways, all about the little things. From timing the baby dance to counting days in between your periods, getting pregnant can be full of details. Taking the right vitamin or herbal supplements and eating fertility friendly foods are just some of the components to successfully becoming pregnant.

On the other side of things, though, there are foods to avoid. These foods aren’t necessarily harmful or unhealthy, the just can delay you from getting pregnant or even keep you from getting pregnant altogether.

Here are five of the most unfriendly foods for fertility:

1. Peas. There is a natural chemical in peas known as m-xylohydroquinone that may keep you from getting pregnant. Research on peas goes back as far as the 1950s. There is even research ongoing about how peas could be used as a natural contraceptive.

2. Soy. Soybeans contain the plant version of estrogen, known as phytoestrogens. In fact, the extra estrogen provided by soy products can be enough to create a hormone imbalance. This can affect the production of eggs, as well as ovulation. Watch out for soy in unexpected places, such as flour, oils, and milk.

3. Processed meats. Most processed meats, such as hot dogs and lunch meats, contain nitrates. These nitrates act as a preservative and can interfere with fertility. Nitrates have a controversial history and have been thought to cause a variety of health problems. While many nitrate-related problems have been dispelled as myths and most processed meats now contain much smaller amounts of nitrates than in the past, there is still enough to interfere with fertility.

4. Calcium-depleting foods. Good examples of these kinds of food include spinach and peanut butter. These foods sap your body of calcium. Your body needs that calcium to help your baby develop.

5. High-sugar foods. Pop, candy or even high-sugar fruit drinks all can contribute to low levels of blood sugar. This can severely interfere with your reproductive hormones. Chocolate and pop both typically contain the added element of containing caffeine, which reduces the risk of conception, as well.