Some people swear they can drink coffee all day—even after dinner—and still drift off to sleep minutes after their head hits the pillow. Others won’t touch caffeine at all after early afternoon, saying if they did, they’d be “up all night.” My colleague’s doctor advised her to should stop drinking coffee by 10 a.m. in order to avoid disrupting her sleep cycle. But is there a specific time of day when we should cut ourselves off from caffeine?

To find out when exactly when to put down the mug, I turned CEO and co-founder of FitnessGenes Dan Reardon, MD, who has studied how coffee affects different people. “The speed at which you metabolize caffeine creates a natural cut off point through the day,” he says. “Whereas a fast metabolizer might clear 100-200 milligrams—what’s in a typical cup of coffee—in just a few hours, it might take a slow metabolizer 12-hours plus, which could impact sleep.”

If you’re a fast coffee metabolizer, Dr. Reardon recommends having your last cup by 5 p.m. (Sorry, coffee isn’t recommended for anyone much later.) “Let’s not forget that even for a fast metabolizer of caffeine, there could still be a cumulative effect of drinking lots of coffee early in the day, which could slow down the rate at which they break down their caffeine later,” he says. If you are a fast metabolizer who enjoys multiple cups a day, he says, cut yourself off by lunchtime so your body has time to process it all. Otherwise, you could have trouble sleeping.

If you’re a slow coffee metabolizer, Dr. Reardon says your cut-off time should be earlier, between 12 p.m. and 2 p.m., as it can often take the body a good eight hours to completely metabolize the caffeine in a cup of coffee. “I would also recommend that people pay attention to there consumption of other caffeine containing products such as energy drinks, pre-workout drinks, and fat burners,” says Dr. Reardon.

Your coffee habit isn’t the only factor that plays into your sleep habits—not by a long shot. But keep the cut-off rule in mind and you’re a lot more likely to wake up refreshed and, well, ready for your morning cup.

Here’s the verdict on if coffee is actually good for you. Plus, how seven different caffeine sources affect the body.