Downing double martinis when you’re pregnant will most likely earn you some dirty looks, if not wagging fingers—this isn’t your grandmother’s generation. But what about before you know you’re pregnant? Or even when you’re just starting to try to be?

Messages from the medical community regarding alcohol consumption while pregnant have been mixed over the years, and it’s the same with that murky in-between period. So, if you had a couple glasses of wine with dinner and then found out you were expecting, how scared should you be that your baby will be affected? The answer, experts say, is complicated, but ultimately no amount of alcohol can be considered totally safe.

The idea that pregnant women’s drinking might be putting their babies at risk started to take hold in the 1970s when researchers first recognized fetal alcohol syndrome. In the decades that followed, doctors and education campaigns warned women about the long-term effects their drinking could have, but there were still different lines of thinking about what was and wasn’t okay. Some doctors continued to tell patients that an occasional glass of wine or bottle of beer was fine.

“Some people in the media have interpreted that it’s safe for mothers to consume some alcohol in small amounts, but no study has ever proven that.” —Steve Rad, OB/GYN

“This whole situation has obviously stirred up a lot of confusion and controversy,” Los Angeles-based OB/GYN and maternal fetal medicine specialist Steve Rad says. “Some people in the media have interpreted that it’s safe for mothers to consume some alcohol in small amounts, but no study has ever proven that. So, a [universal] safe level of alcohol consumption hasn’t been established. Basically, the metabolism of alcohol varies depending on various factors, so because you can’t establish a safe amount of drinking, even a small amount in some women might affect the baby, even if in some other women, it might not.”

Those factors could include a mother’s nutrition, her age, her size, or her genetic background. As Dr. Rad indicates, because the exact elements have historically been hard for researchers to identify, public recommendations have leaned more and more toward “better safe than sorry.” But when the CDC put out new guidelines in 2016 encouraging all sexually active women of childbearing years who are not on birth control to completely cut out alcohol just in case, many viewed the advice as patronizing overkill.

If you’re actively trying to get pregnant, however, Dr. Rad says this really is the way to go. Most women don’t know they’re pregnant until about four to six weeks into the pregnancy, the CDC says, and drinking during that window could potentially be harmful.

“Alcohol freely crosses the placenta, so when the mother drinks, so does the baby.”

“Alcohol freely crosses the placenta, so when the mother drinks, so does the baby,” he says. And the effects, which can range from miscarriage or stillbirth to structural defects in the baby’s face, bones, or organ systems to developmental delays, can sometimes be hard to detect until much later. A recent study published in the Journal of the American Medical Association estimates that the number of  American children with neurological damage from fetal alcohol syndrome and other alcohol-related disorders could be as much 1.1 to 5 percent of the population, about five times previous estimates.

Of course, stuff happens. “Certainly, if you have one glass of wine at a party and then find out that you’re pregnant and you’re freaking out, that probably isn’t going to cause a lot of harm,” says Melanie Manning, MD, a clinical associate professor of pathology and pediatrics and fetal alcohol researcher at Stanford Children’s Health. “But we just don’t know what level is safe. So what’s recommended is women, if they know that they’re wanting to get pregnant, abstain from alcohol. Or as soon as they find out they’re pregnant, they abstain for the rest of the pregnancy.”

One 2016 Danish study found that women who drank less than 14 servings of alcohol a week were no less likely to conceive than women who didn’t drink at all.

Whether drinking while trying to get pregnant can actually affect your fertility is something else researchers are still looking into. While one 2016 Danish study found that women who drank less than 14 servings of alcohol a week were no less likely to conceive than women who didn’t drink at all, Dr. Rad says other evidence suggests there could be a link but more research is needed. Another study from 2014 indicated that men who drank more than five drinks a week produced lower-quality sperm, with the effects getting worse the more they drank.

So, while the sometimes stressful process of trying to make a baby might understandably make you crave something to take the edge off, until more precise information is available about what that beer could do to you or a just-forming baby, the experts say it’s best to stick to mocktails.

After cutting out alcohol, you should avoid eating these healthy foods while pregnant, too. And here are 10 things that happen to your body while pregnant you might not have heard about.