It basically works like this:
Eat high grade good -> Max benefits -> eat lesser grade food -> benefits lowered or eliminated -> eat high grade food again -> retain effects of lower grade food until the duration of that effect wears off, at which point you'll have whatever time you waited in between consuming the lower grade and the higher grade to enjoy the max benefits.
Eat lower grade food -> nothing short of divine intervention is going to save you.
This isn't exactly a glaring problem, but when you go out of your way to obtain certain food for the express purpose of its benefits, and then want to consume something socially, and get slogged, it's really annoying.
Am I the only one bothered by this?