Why do I sometimes have anxiety about my orientation and sometimes I’m fine?

I guess the key thing is to find a way to not let the phases worry you!
I feel like anxiety is always waiting to find something to latch onto and when it does, you feel like that thing is the problem - i.e I used to avoid things that "made me anxious" but I eventually realised it was not the THINGS (situations, environments, people, jobs etc.) but that anxiety was latching onto ANYTHING it could! If I quit a job that was making me anxious, the next (totally different) job would eventually become stressful too...
And any sort of uncertainty about the future or any situation that COULD go in lots of different ways (giving lots of different scenarios to be anxious about!) is fertil soil for anxiety!
But if you can feel certain that you are bi and that fluctuations of attraction will happen and are normal - anxiety will have to move on to some other topic! (Like the telephone in my case - even a phone ringing on TV makes my heart race and my stomach tighten!)