The healthy food world loves a good debate. (Is nut milk still milk? Is wine good for you? Is coffee?) But at this point, no one in the health community is really sitting around wondering: Is sugar bad for you?

Science has shown over and over again that consuming too much sugar is linked to diabetes, obesity, inflammation, and mood disorders. And it’s become the part of the nutrition label health-conscious consumers pay attention to the most.

However, there’s a line between sugar-conscious and sugar-obsessed. “Quite a lot of people are getting low levels of orthorexia, where they’re starting to be very fearful or obsessive about health,” says holistic health coach Nikki Sharp. “People are becoming terrified of eating a piece of fruit!”

So where’s the line? When is avoiding sugar healthy, and when does it reach the point where it gets in the way of reaping the rewards of other benefits, like the fiber and antioxidants in fruit?

Check your intention

Sharp says the first thing to consider when wondering whether your focus on sugar is healthy is mindset. “When I talk to my clients and friends about why they’re scared of sugar, it almost always comes back to a fear of gaining weight,” she says. And that can be connected to feelings of confidence and self-worth.

Sharp says this fear of sugar can also be connected to a bigger fear of losing control—as was the case with her. “I used to be so scared to have a piece of cake or donut, because I was worried I wouldn’t be able to stop myself from binging,” she says. “And I know that’s the case for a lot of people. But then people end up trying to nutrition bars or other things—also often full of sugar—and it doesn’t work. So many people are stuck in this restrict-binge cycle.”

So before you zero in on just how much sugar intake is “okay,” Sharp says, you need to figure out what’s really driving your sugar avoidance. It’s great if you’re looking to healthify your eating habits. But if the root cause is emotional, it’s important to deal with it before it takes over your life—as well as your diet. “Usually when someone is avoiding something super intensely, it has to do with a deeper issue that has nothing to do with the food,” she says.

Understand the nuance

Another issue: Sugar has been so demonized that people have become confused about how they should actually be eating, says Sarah Wilson, journalist and author of I Quit Sugar. “So often, people turn to a wellness solution to be the answer to all their stresses,” she says. The result can be counting grams of sugar so intensely (or carbs, or calories) that mealtime becomes stressful instead of nourishing. “Some perspective has been lost,” she says.

The most important thing to know about sugar is that it’s not all the same, says Amy Shapiro, RD. It might seem obvious, but she stresses that sugar in, say, a cookie, affects the body differently than sugar in fruit.

“Fructose, the sugar in fruit, is processed through the liver and it’s low on the glycemic index, meaning it doesn’t spike blood sugar the way glucose—which is in carbohydrates and table sugar—does,” she says. “So you’re not going to get the same mood swings or hormonal disruption.”

Shapiro also says that sugar in processed food is nutritionally void, while sugar in fruit comes with other benefits, like antioxidants and fiber. “So if you avoid fruit because of the sugar, you’re also missing out on these benefits,” she says. Just stick with the whole fruit as much as possible, since things like juices and smoothies are often loaded with more sugar than you should be consuming at once (despite mostly coming from fruit).

Don’t overthink it

In general, Shapiro recommends women cap their added sugar intake (which includes table sugar, honey, and maple syrup) at six teaspoons a day. As for fruit, she says two to three pieces daily is okay in her book.

But if tracking your intake leads you down an obsessive road, Wilson suggests ditching the teaspoons and focusing on a whole foods-based diet that you prepare yourself. “My biggest health tip for people is to start cooking. Because when you prepare your own food, you aren’t going to put half a cup of sugar into something so readily because you see it,” she says. She adds that eating more like our great-grandparents (three homemade meals a day, no packaged or processed foods) naturally lends itself to eating less sugar.

Sharp agrees. “When you eat real foods, like making your own pizza with good quality ingredients, and not fake foods with processed ingredients, your body is going to digest it better and you’ll actually feel satisfied.” And less likely to feel like you “need” a sugary snack, no less.

Of course, that doesn’t mean that treats are completely off the table forever. Sharp also recommends listening to what your body craves to take out any fear or stigma you might feel about eating sugar. “If you’re craving a piece of cake, eat a delicious piece of cake,” Sharp says. “Go out and eat it with your friends and make it an experience. Don’t make it this whole secret thing where you then feel ashamed.” Hear, hear—because there’s nothing to be ashamed about.

Here’s more advice on how to eat with intention, and what mindful eating really means.