Colors aren’t the names that people give them. They are relative to each other. If you put a gray near a warm color, that “gray” is going to look blue.
It’s because if you look at it through a color circle rather than a color wheel, while the gray isn’t shifting in hue towards blue, what’s happening is that it’s coming closer to blue by desaturating. A good rule of thumb is that when you desaturate a hue, you’re making it go in the direction of the opposite hue on the color wheel.
Grays are more able to be harmonious with other grays and even other saturated colors. Saturated colors next to other saturated colors crash against each other in an unpleasant way.
When colors are less saturated, it’s easier for them to be in harmony. Not all of them have to be desaturated though. One or two important colors may stand out by being more saturated.