It doesn't mean you don't have XMRV. There are no standard physical tests yet that definitely prove if someone has CFS/XMRV, unfortunately. This is a curse, as even after about thirty years, there's still discussion if someone is just depressed or has CFS!
It's true that a MRI does seem to show some weird lesions in some CFS patients, but certainly not all.
We don't know yet how this virus works and where it likes to hang out in the body. We do know from other viruses that they can cause different symptoms in different people. There are also many examples of people infected with a virus, who don't develop symptoms.
Take HIV for example: there are people who can control the virus, because their immune system is somehow able to supress the virus. Also one person infected with HIV can get AIDS within a year, while another might get AIDS only after seven years. Factors like the immune system, genetics, our diet, coninfections, etc play a very big role.
Another example is Epstein Barr virus. About 95% of the adult population is infected with this virus, but only a small percentage has had symptoms.
As a general rule of thumb:
If a virus has infected a larger population group, you will be seeing more people with symptoms.
If a virus only causes disease in 1% of the population, it makes a big difference if 5% of the population is infected or 50%.