1. Will men's wristwatches return to smaller sizes as was found from the 1920s to 1980s? No, because their size was in large part a reaction to the size of pocketwatches. That's no longer a factor. Maybe smartwatches will exert some effect on mechanical wristwatch size if the former miniaturizes, but that has yet to be seen.
2. Can you wear a smaller watch? Will men titter into their lacy hankerchiefs as you prance along the boulevard? No. While people Do notice watches, even if you don't notice their noticing, this is more about colored gold than size. It is larger sizes that are more eye-catching, not smaller ones. Maybe someone will notice the diamonds, but that person is likely to be female.
In short, no one would notice or care about you wearing a 35mm watch. Have confidence in yourself (it is a man's watch because you are a man and wearing it), and enjoy.
I have a 7.25-7.5"ish wrist and I wear a few 35.5mm Breguets. They look great.