To me the biggest problem with EBM is it has no underlying conceptual or mathematical framework to talk specify what a piece of evidence is or how to combine pieces of evidence. Hence we get gross over generalisations, meta analysis comparing different things and individual pieces of evidence dismissed as anacdote (which is particularly significant from a safety perspective).
But there never can be. Such things are generally too difficult to reduce to numbers like that. In fact I think the mathematical issues are a lot less important than the other things. Statistics tells you how repeatable something is, but it does not tell you if the entire something is nonsense or not. EBM will get more rigorous over time, but will this be enough? I do not think so.
Lets presume, which is not the case but just for argument, that the PACE trial had overwhelming success instead of dismal failure. This would show (hypothetically) that CBT/GET worked ... IF the results were based on objective evidence, if it were replicable, and if the study design were sound. However it can NEVER show the underlying hypothesis is sound. That wasn't tested. Such cognitive hypotheses cannot ever be tested or proven ... and it matters naught if you improve the mathematics, or better specify the conceptual framework. Its a null question. Tackling with such issues is very hard to do in a formal conceptual framework, though there are simple tests that can be performed that help. Is the underlying hypothesis tested? Clearly NO. Therefore its not a sound study with respect to the underlying hypothesis. This is easy enough to say, but in many cases this is not going to be so easy to demonstrate, a point I am hoping to look into in the next few years.
My point is mainly that there are issues in medicine, particularly psychiatry, that cannot ever be formally specified or tested with any methodology to date. They still need to be dealt with ... usually by rejecting them I would hope. If you do this though most of psychiatry would disappear in a puff of logic, precipitating a global medical crisis.
Let me give a further example. Suppose they had a drug that could be shown to moderately treat conversion disorder in a phase 3 clinical trial. The trial was large, and the statistical significance good. Is this a valid test? How do you assess a therapy if the underlying condition itself is hypothetical, untested and objectively unvalidated? Do you even know what you are treating?
Some imprecision is always going to be required I think, but that does not mean it is immune to reason. Intense rational investigation is still going to be required. Further we still have to be able to assess the old evidence for many things, and sometimes that evidence is low grade, sporadic or suspect. How do we quantify that? We cannot reject half of medicine because its unproven, which is a common estimate of how much is suspect.
I have only just begun investigating EBM. I will be saying a lot as I investigate further.
Of course there is the thorny issue that almost nothing can be proved with science, only disproved. That is a whole other problem.