Absolutely! We each know best what our bodies can handle.
I do think, however, that some patients don't realize there are ways to reduce the physical demands of travel. You can taxi to the airport (wearing headphones and dark glasses), and be wheelchaired from curb to plane and plane to curb (again with headphones and dark glasses). Taxi to the hotel (the wheelchair attendant will take you to the taxi). Go straight to bed as soon as you get to the hotel. Order room service if your diet allows. Don't go anywhere except to the specialist's office for your appointment and stay in bed the rest of the time. Plan rest days at the hotel before and after the appointment if you need them.
You don't have to walk, you don't have to pick up or carry luggage. You don't have to drive. Hotels will give you a room near a door or the lobby if you tell them you have health issues that require it. If you act like the very sick person you are, most people are helpful and will treat you well.
If you have a travel companion, it's even better because your companion can even wheelchair you in and out of the hotel and doctor's office. You can act like luggage -- just sit in the chair the whole trip and shut out everything. I've done that a number of times, particularly when I was in the stage where I PEMed walking across a room.
I'm not saying this will work for everyone. I'm just saying that too many PWME want to act "normal" when they travel and then feel they can't travel because they can't do it like a healthy person does. The good news is: we don't have to! We're very sick people and we can (and should) travel like very sick people.
I'm also not saying it's 100% easy to travel by air to see a top specialist. When I was bedbound, I had to plan far in advance, rest even more before and after the trip, and sometimes I still got PEM. But it was worth feeling worse for a couple weeks a few times to get the treatment that moved me from bedbound to working full-time (after a fashion). After a few twice-a-year trips to the specialist, the treatment improved me enough that I don't PEM from the travel (as long as I'm smart about how I travel).
Long-distance travel to see a top specialist is definitely not for everyone. We all need to make that kind of decision based on our understanding of our individual condition. But I suspect too many people are depriving themselves of a very real opportunity for significant quality of life improvements because of incorrect perceptions about the difficulties of traveling as a very sick person.