I really don't see them doing it, but if they did, here's what I think:
Super Mario Galaxy 3 would have to revise the whole formula. Come at it from a completely different angle.
If you ask me, Super Mario Galaxy 2 was more like an add-on for the original. It added new levels, fixed some things that needed fixing, made some things worse (*cough*faceship*cough*), we got new power-ups, etc. But it was basically more content in the same style as the original.
Nintendo, Miyamoto and friends; they are forever the innovators. If Super Mario Galaxy 3 ever happens, or if they ever decide to bring similar mechanics back to the series (and I hope they do because Super Mario Galaxy is one of my favourite games ever), it won't be anything like the original 2 games. I don't think Odyssey is quite it. Maybe it'll be like Super Mario 3D Land. Maybe it'll be like Super Mario Sunshine. Maybe it'll be like nothing we've ever seen before. Maybe, someday, our prayers will be answered.