(French here) It’s not really about words distinguished by gender, but misgendering a word will sound very weird to French people. So much so you’ll be labeled as foreign immediately. As crazy as it may seem, but misgendering objects sounds worse than misgendering people (in a purely linguistic way).
I mean, of course, but sounding obviously foreign isn't so bad if your meaning can be understood and you haven't accidentally said something with a wildly different meaning than intended.
People really overestimate the importance of this. If you go to le or la gare doesn't matter much. But if you don't know "gare" in the first place you could have a harder time.
Also "embarrassing yourself" doesn't happen because everybody loves it when n-th (for any n > 1) language speakers attempt to speak their language.
