I hardly ever change watches during the day (e.g., at the end of day), and I never wear a suit or dress up. So, I basically always wear a Lange while dressed informally. You SHOULD NOT feel like you need to dress up to wear a Lange -- no worries whatsoever about that!
One of the many things I like about Lange is that most of their watches really transcend context -- formal, informal, etc. -- except, as you note, Lange hasn't produced a sports watch (and I'm in the camp that hopes they never do!).
[That being said, I do trade off between Langes and some other brands for days at a time. For example, I've been wearing the Lange Zeitwerk Striking Time for about 2 weeks -- a very long time for me -- but tomorrow I'm switching to an Omega Ploprof (new version, not vintage). I guess I feel the need to stick with big watches for now!
]