There are some things you know are going to make your anxiety worse: WebMDing your stomach ache, foregoing sleep to list all the ways your work presentation could go wrong, calling your friend who freaks out about everything…But treating yourself to a scoop of Rocky Road or a cupcake from your favorite bakery, that’s going to make you feel better right?

Sometimes, 100 percent yes. But other times, that sweet treat can backfire, sneakily causing all sorts of changes that can lead to the opposite of feeling good: anxiety. Here, health experts break down the relationship between sugar and anxiety, and what you can do to combat it.

How are sugar and anxiety related?

The problem with sugar, says hormone health educator Candace Burch, is that it causes blood sugar spikes and drops, which directly affects mood. “The rush of sugar leads to sugar highs, giving a lot of energy, but then the lows lead to feeling sluggish and down.”

“Sugar can exacerbate your feelings of anxiety because of the way our bodies respond to digesting them,” adds Brigitte Zeitlin, RD, owner of BZ Nutrition, a New York-based nutrition practice. “[Sugary foods] cause your blood sugar to spike and then drop faster than they would after eating non-high-sugar-foods. This quick spike and drop causes you to feel uneasy and can even at times mimic a panic attack.” Having low blood sugar levels can actually put the body into a stress response, which, as Zeitlin mentions, can increase anxiety.

Our bodies obviously don’t like being stressed or anxious, says Zeitlin. People combat that in various ways, including reaching for sugary foods. “Foods high in sugar trigger the release of serotonin, which is a feel good hormone,” Zeitlin says. “We are trained to eat sugar and feel good, which makes it understandable why people stress eat, because they just want to feel better when they are feeling stressed and anxious.”

However, when your body is stressed or anxious, you also have higher levels of cortisol (a.k.a. the “stress hormone”). Zeitlin says when this happens, your body suppresses the release of insulin, the hormone that takes up glucose to use for energy. You now have a one-two punch of spiked blood sugar levels (since you’re eating more sugar to combat your stress) and storing excess sugar as fat since you’re not turning it into energy. “So, eating more sugar when you are stressed or anxious just amplifies the amount of sugar your body would naturally have already flowing, and contributes to more severe drops in blood sugar and more drastic drops in your mood,” she says. Enter a cycle of turning to something sweet every time they need another energy and mood boost, and a subsequent rollercoaster of ups and downs which can also contribute to feelings of anxiety.

This sugar-and-anxiety cycle isn’t just relegated to the daytime hours. “High-sugar foods can keep you up because of their energy that prevents your natural stress-booster of sleep from kicking in,” Zeitlin says. “When we don’t get enough sleep we feel even more anxious and stressed because our body missed an opportunity to process it properly.” You’re now going into the next day with less sleep, and thus lower energy levels and higher stress levels. And what do many people reach for to combat stress and anxiety? You guessed it: sugar.

And “high-sugar foods” doesn’t just mean candy, cookies, and cake. “Studies have found that women who eat more refined carbohydrates (baked goods, candy, white breads/rice/cereals, bagels, etc) were more likely to suffer from depression and mood swings because of the drastic peaks and deeps in blood sugar levels,” Zeitlin says.

How to keep sugar from contributing to anxiety

Of course, this isn’t just to freak you out and make you throw out all of the dairy-free ice cream in the fridge. Lots of other things can contribute to anxiety, including stress, coffee, work, and even family relationships—so cutting out Oreos isn’t the magic bullet for reducing anxiety. But the impact of sugar on anxiety levels can affect anyone, says Zeitlin—and if you have an existing anxiety disorder, sugary foods will likely exacerbate your condition, she adds.

One way to combat this is certainly to reduce your sugar intake, including processed foods and breads. It’s also a good idea to load up on foods low in sugar and high in fiber (think veggies, fruit like berries, and whole grains)—Zeitlin says they have a much more even effect on your blood sugar, which can help cut back on feelings of “increased anxiety.” She also recommends stopping eating about two hours before you go to sleep. “This gives your body time to properly digest and process the food—sugary or not—and let that energy subside in time for you to actually fall asleep and stay asleep.”

When you are eating foods higher in sugar, Burch suggests pairing it with foods higher in fiber and healthy fats. “This slows the absorption of sugar, preventing it from spiking blood sugar as much,” she says, and thus making it less likely to put you in an anxiety spiral.

But all this comes with a big caveat: Changing one’s diet shouldn’t be the only thing a person does to fight back against anxiety. “Changing your diet to limit high-sugar foods will not treat or cure your anxiety disorder, but it will help manage it better and optimize the times you are feeling good and less anxious,” says Zeitlin. Psychologist Gail Saltz, PhD, says some ways to reduce anxiety not related to food include deep breathing, working out, and (to bring it all full circle) getting enough sleep. If none of these lifestyle changes are helping, it’s essential to see a professional to help you come up with a treatment plan.

“Sugary foods contribute to mood swings and anxiety. Period,” Zeitlin says. And now that you understand the connection, it’ll be easier to be more mindful when you are consuming foods with sugar in them.

Additional reporting by Jessie Van Amburg.

If you want to cut out sugar completely but don’t know how, here’s some tips. And if your anxiety is worst in the morning, this could be why.