One of my professors was asked the same question and let me paraphrase his response:

You give up on intuition when it gives up on you.

One of the many reasons why most of physics is deeply mathematical is because our intuition alone is unable to explain all the results that we observe in nature and when that happens, we rely on mathematical theories to shed light on the nature of reality