Psychoquackery on BBC Radio 4

[Sasha]

That's closer to a complex art than science. Self-report of subjective phenomena is not true science as it is not objectively measurable. Sometimes it's the best we have, so we use it, but the simple fact that it requires opinion-based interpretation by the user minimizes its scientific validity.

This isn't to say the psychology, properly done, is not valuable. Many lives have been saved by good psychology. Medicine is willing to admit that it is, largely, an art with scientific underpinnings. Psychology is even more of an art, and not at all an easy art to perform well. It takes a special kind of person to be a good psychologist -- intelligent, empathetic, and wise. But that does not make is a science... or at least not more than any other social science, where "science" is used in only the broadest sense of "a study".

Depends what branch of psychology you're talking about, I think. Not all psychology studies involve self-report: perceptual and cognitive psychology ones often don't, for example. Psychological studies of babies don't - they measure behaviour such as length of time gazing at different stimuli.

I think most people are more familiar with clinical than experimental psychology and that tends to give a bit of a skewed idea of what it's all about.

Psychology is the scientific study of complex, open systems. It's not easy, but that doesn't make it art rather than science!

 

[MeSci]

Re: cognitive psychology--brain washing is tightly focussed on objective measures too. And neuropsychology depends on actual scientific instruments for measuring.

Before psychology, the study of the human mind individually and socially was the purview of philosophy.
I think it belongs there. Because science is/can be/should be so precise other domains of study have hoped to cash in on that certainty. While there may be some well designed studies that stand up to scrutiny at the end of the day they can only generalise. People are not protons they don't follow scientific laws. Any psychological studies will have all manner of exceptions, outliers, variations.

What would you do though, when 'psychological' problems are found to correlate with physical brain abnormalities? Neuroscientists have to span the objective physical manifestations of the illnesses, which may or may not be causal, and the observed behaviour/ability of the patients.

@alex3619 has some interesting views on psychology.

 

[shahida]

i heard that programme too. Drove me mad. The BBC ought to at least present both sides of the story- they should present the science on fibro' and aren't there people doing science on IBS too?

 

[sarah darwins]

i heard that programme too. Drove me mad. The BBC ought to at least present both sides of the story- they should present the science on fibro' and aren't there people doing science on IBS too?

Absolutely, on all counts. I was thinking of contacting the programme makers but I felt I should have a suggestion for someone they should interview to get the other side of things. As Rona MM didn't mention me/cfs this time (though she definitely works on it) it should probably be someone working on conditions she did mention on this occasion. I wouldn't know who to suggest.

 

[Kyla]

Absolutely, on all counts. I was thinking of contacting the programme makers but I felt I should have a suggestion for someone they should interview to get the other side of things. As Rona MM didn't mention me/cfs this time (though she definitely works on it) it should probably be someone working on conditions she did mention on this occasion. I wouldn't know who to suggest.

How about George Davey Smith?

http://forums.phoenixrising.me/inde...eorge-davey-smith-summary-with-comments.1805/

I have no idea if he does this sort of thing (radio appearances), but he definitely provides the polar opposite viewpoint on behavioural approaches to these sorts of illnesses, and is a very well respected scientist.

 

[shahida]

Maybe if there's a scientific paper/ write up on fibro which she did mention or a proper IBS one, attach a ink for them to see. And maybe best not to mention if you have me either or i suspect the default position will be 'there go those crazy wound up ME-ers again'. I was going to but am too ill.
The thing i couldn't believe is when she said if someone's got IBS and constant diarrhea you have to look at the social aspects. CONSTANT DIARRHEA!!!!!!!!

 

[SOC]

Depends what branch of psychology you're talking about, I think.

Fair enough.

Not all psychology studies involve self-report: perceptual and cognitive psychology ones often don't, for example. Psychological studies of babies don't - they measure behaviour such as length of time gazing at different stimuli.

True, but the problem is that it is not left at the reporting of observation and statistical analysis of the results. Guesses, assumptions, and just plain opinions about the reasons for the observed phenomena are presented as factual with no evidence to back them up. In the case of length of gaze time of infants, assumptions are made about why infants look longer at some things than others and those assumptions are reported as fact. They don't say, "Babies look longer at animals that appear to talk than at ones that appear to make animal noises"; they say, "Because babies look longer at talking animals, they realize that animals should not be talking". That's not a scientific conclusion. It falls more under the category of natural philosophy.

Don't get me wrong. I think that kind of study can be valuable. In fact, I took my infant to participate in many of that kind of study and learned many interesting things. The conclusions drawn were interesting and thought-provoking; they simply weren't scientific conclusions.

Psychology is the scientific study of complex, open systems. It's not easy, but that doesn't make it art rather than science!

I suppose for this discussion to continue productively, we have to define science. Social science fields such as psychology, economics, politics, sociology, anthropology, and even sometimes history consider themselves sciences, but the origin of "science" in that context is rooted in the meaning of science as "a field of study". That is entirely different from the definition of science used by physical and natural scientists who demand rigorous, objective, verifiable, and repeatable evidence to support any conclusion. So a social scientist will naturally consider their field to be a science, but a natural or physical scientist sees very little in it that they recognize as science. Research, yes; science, no.

 

[Sasha]

True, but the problem is that it is not left at the reporting of observation and statistical analysis of the results. Guesses, assumptions, and just plain opinions about the reasons for the observed phenomena are presented as factual with no evidence to back them up. In the case of length of gaze time of infants, assumptions are made about why infants look longer at some things than others and those assumptions are reported as fact. They don't say, "Babies look longer at animals that appear to talk than at ones that appear to make animal noises"; they say, "Because babies look longer at talking animals, they realize that animals should not be talking". That's not a scientific conclusion. It falls more under the category of natural philosophy.

Again, it depends on the study and the competence of the individual scientist. The original baby-gazing studies were (I think) done to test the hypothesis that babies could distinguish between certain classes of stimuli (a fair deduction because if they couldn't, there'd have been no difference in gaze time between classes).

I suppose for this discussion to continue productively, we have to define science. Social science fields such as psychology, economics, politics, sociology, anthropology, and even sometimes history consider themselves sciences, but the origin of "science" in that context is rooted in the meaning of science as "a field of study". That is entirely different from the definition of science used by physical and natural scientists who demand rigorous, objective, verifiable, and repeatable evidence to support any conclusion. So a social scientist will naturally consider their field to be a science, but a natural or physical scientist sees very little in it that they recognize as science. Research, yes; science, no.

No, that's not how science is defined: it's a particular kind of organised curiosity (there's a whole field of philosophy of science and it's not just 'a field of study'). Scientists form theories, generate hypotheses from those theories, and test those hypotheses against reality. That's the point at which you find out if you have repeatable evidence, but not before. You can study something scientifically and consistently fail to find evidence in support of your theory. That means your theory is wrong or you're studying a difficult system. It doesn't mean that you're not doing science.

The sciences are on a continuum of difficulty according to the type of system (closed/open, simple/complex) that they're studying. The more open and complex the system, the harder it is to study but it doesn't mean that those studying them aren't doing science. If physicists look at (properly done) psychological experiments and don't recognise them as science, it may be because they haven't studied philosophy of science.

But we're still digressing! We can surely all agree that bad science is bad science. But if we call psychology 'not a science', we risk sounding as though we don't know what science is.

 

[sarah darwins]

How about George Davey Smith?

http://forums.phoenixrising.me/inde...eorge-davey-smith-summary-with-comments.1805/

I have no idea if he does this sort of thing (radio appearances), but he definitely provides the polar opposite viewpoint on behavioural approaches to these sorts of illnesses, and is a very well respected scientist.

Looks like a great suggestion, Kyla. A bit of googling shows he has contributed to BBC broadcast before - this one from 2012: http://www.bbc.co.uk/programmes/b01mw15s

Too tired to do anything now but will try to pen something to them in the morning.

 

[SOC]

Scientists form theories, generate hypotheses from those theories, and test those hypotheses against reality. That's the point at which you find out if you have repeatable evidence, but not before. You can study something scientifically and consistently fail to find evidence in support of your theory. That means your theory is wrong or you're studying a difficult system. It doesn't mean that you're not doing science.

Certainly!

I didn't say science always finds evidence in support of a hypothesis, or that all hypotheses result in repeatable results. I said

[...the definition of science used by physical and natural scientists who demand rigorous, objective, verifiable, and repeatable evidence to support any conclusion.

My problem (and that of most natural and physical scientists) with much social science research is that conclusions are drawn (even begun with ) that are not supported with objective, verifiable, repeatable evidence. There are many valid research studies that show that the hypothesis was invalid, or whose results don't turn out to be repeatable. In those cases, it is determined that the conclusion is not supported by scientific evidence. The conclusion is not considered "truth", unlike in social science where conclusions not supported by scientific evidence are accepted as truth. That can be accepted by society, but it's not science.

Study all you want. Hypothesize all you want. But you don't have a scientific conclusion without objectivity, verifiability, and repeatability. Observation followed by theorizing about the reasons for those observations is natural philosophy, not science. It has it's place. It's the necessary groundwork for developing hard scientific studies that verify those hypotheses. Most social sciences do the observation and hypothesizing, but not the hard science needed to actually scientifically prove those hypotheses.

The original baby-gazing studies were (I think) done to test the hypothesis that babies could distinguish between certain classes of stimuli (a fair deduction because if they couldn't, there'd have been no difference in gaze time between classes).

That's a good example. The only verifiable, repeatable scientific conclusion that can be drawn from that data is that babies can distinguish (in some fashion) between stimuli, that is, babies behave differently faced with different stimuli. Fine. That's not exactly ground-breaking news. Where the problem comes in is when the researcher concludes the reason the babies look longer at some things than others. Where's the hard evidence to support that conclusion? It doesn't exist. It's the researcher's philosophical speculation about the observation.

In one of the studies my baby participated in, the researchers recorded gaze time of the babies when faced with situations that were physically normal (a can continuing to roll some time after rolling down a slope) and situations that were physically abnormal (the can stopping immediately at the bottom of the slope). The conclusion was not that babies looked longer at one situation than the other. That would be the objective and repeatable conclusion.

They concluded that babies understand physics (at some level) as early as six months old. Where is the evidence that that is the reason the babies gazed longer? And where is the evidence that they understand at six months but not three? It might be that younger babies understand just as well, but can't hold their gaze as long. Or that younger babies show their scientific understanding differently (more blinks per minute?) It might be that the babies understood the physics. It might be that it was interesting for some other reason entirely. Maybe babies recognize "what I usually see" from "what I don't usually see", regardless of it's physical truth. You can't verify or repeat that babies understand physics, yet that is the conclusion they drew. That's not a scientific conclusion, it's a philosophical construct. It's interesting, and well worth speculating about, but it's not verifiable or repeatable. All we can repeat is that babies look longer at some things than others. Why they do so is sheer speculation.

Good research in the area would gather massive amounts of information about the different things babies look longer at, and try to classify them into groups. We could then, perhaps, improve the quality of our speculations based on a much larger collection of data. But in the end, the "why" is still speculation, a hypothesis that is as yet unproven. That doesn't stop social scientists from treating their speculative conclusion as scientific fact. And that's where we start running into trouble with psychological research.

It's not that social studies are necessarily not constructed with a hypothesis and testability. It's that the conclusions are not supported by objective, verifiable, repeatable evidence. The "conclusions" themselves are new hypotheses, or opinions of the researcher on the reason for the results, not scientific conclusions themselves.

It seems to me that we (as with all the other social/physical/natural scientists in the world) are going to have to agree to disagree about what is classified as science. It's a big question with a lot of egos invested in it and we are not going to resolve the issue here and now.

 

[Sasha]

We agree on some things and not on others, here, @SOC - and we're still digressing! I'm going to bow out of this now (sorry to do that at the point where you've just written a great long thing but I think we're off-topic and not getting any closer to agreeing what science is.)

But we can agree that there are plenty of instances where some psychologists go beyond their data. Which brings us back to the thread, I think!

 

[SOC]

We agree on some things and not on others, here, @SOC - and we're still digressing! I'm going to bow out of this now (sorry to do that at the point where you've just written a great long thing but I think we're off-topic and not getting any closer to agreeing what science is.)

But we can agree that there are plenty of instances where some psychologists go beyond their data. Which brings us back to the thread, I think!

I can't argue with any of that. If we agreed on everything, I'd be a little worried, cuz that's just not normal. We do agree, I think, that psychology can be a fascinating and valuable field, and it's unfortunate for good practitioners that there are some that grossly misuse it.

 

[JamBob]

Certainly!

I didn't say science always finds evidence in support of a hypothesis, or that all hypotheses result in repeatable results. I said

My problem (and that of most natural and physical scientists) with much social science research is that conclusions are drawn (even begun with ) that are not supported with objective, verifiable, repeatable evidence. There are many valid research studies that show that the hypothesis was invalid, or whose results don't turn out to be repeatable. In those cases, it is determined that the conclusion is not supported by scientific evidence. The conclusion is not considered "truth", unlike in social science where conclusions not supported by scientific evidence are accepted as truth. That can be accepted by society, but it's not science.

Study all you want. Hypothesize all you want. But you don't have a scientific conclusion without objectivity, verifiability, and repeatability. Observation followed by theorizing about the reasons for those observations is natural philosophy, not science. It has it's place. It's the necessary groundwork for developing hard scientific studies that verify those hypotheses. Most social sciences do the observation and hypothesizing, but not the hard science needed to actually scientifically prove those hypotheses.


That's a good example. The only verifiable, repeatable scientific conclusion that can be drawn from that data is that babies can distinguish (in some fashion) between stimuli, that is, babies behave differently faced with different stimuli. Fine. That's not exactly ground-breaking news. Where the problem comes in is when the researcher concludes the reason the babies look longer at some things than others. Where's the hard evidence to support that conclusion? It doesn't exist. It's the researcher's philosophical speculation about the observation.

In one of the studies my baby participated in, the researchers recorded gaze time of the babies when faced with situations that were physically normal (a can continuing to roll some time after rolling down a slope) and situations that were physically abnormal (the can stopping immediately at the bottom of the slope). The conclusion was not that babies looked longer at one situation than the other. That would be the objective and repeatable conclusion.

They concluded that babies understand physics (at some level) as early as six months old. Where is the evidence that that is the reason the babies gazed longer? And where is the evidence that they understand at six months but not three? It might be that younger babies understand just as well, but can't hold their gaze as long. Or that younger babies show their scientific understanding differently (more blinks per minute?) It might be that the babies understood the physics. It might be that it was interesting for some other reason entirely. Maybe babies recognize "what I usually see" from "what I don't usually see", regardless of it's physical truth. You can't verify or repeat that babies understand physics, yet that is the conclusion they drew. That's not a scientific conclusion, it's a philosophical construct. It's interesting, and well worth speculating about, but it's not verifiable or repeatable. All we can repeat is that babies look longer at some things than others. Why they do so is sheer speculation.

Good research in the area would gather massive amounts of information about the different things babies look longer at, and try to classify them into groups. We could then, perhaps, improve the quality of our speculations based on a much larger collection of data. But in the end, the "why" is still speculation, a hypothesis that is as yet unproven. That doesn't stop social scientists from treating their speculative conclusion as scientific fact. And that's where we start running into trouble with psychological research.

It's not that social studies are necessarily not constructed with a hypothesis and testability. It's that the conclusions are not supported by objective, verifiable, repeatable evidence. The "conclusions" themselves are new hypotheses, or opinions of the researcher on the reason for the results, not scientific conclusions themselves.

It seems to me that we (as with all the other social/physical/natural scientists in the world) are going to have to agree to disagree about what is classified as science. It's a big question with a lot of egos invested in it and we are not going to resolve the issue here and now.

I think it's very important that psychologists don't overstate the findings of their research. And your concerns are certainly evident in a lot of areas of psychology. Psychology is a broad church and does encompass methodologies that aren't scientific at all e.g. qualitative methods. And as you say, psychologists shouldn't draw conclusions that they can't support.

However, falsifiability is an element of experimental design used in a number of psychological experiments. Eg. instead of trying to prove a theory through induction, you find falsifiable theories and attempt to falsify them. In this way you might draw some conclusion.

Popper, deduction, falsifiability, control of variables, repeatability, validity, reliability, the need for replication etc. all formed the core subject matter of the research methodology courses I attended. We also had lectures on qualitative methods which aren't scientific. Also not all experiments are based on self-report and observation. Some experimental psychologists do use tools such as EEG, cortisol assays etc. in their experiments.

 

[Snowdrop]

What would you do though, when 'psychological' problems are found to correlate with physical brain abnormalities? Neuroscientists have to span the objective physical manifestations of the illnesses, which may or may not be causal, and the observed behaviour/ability of the patients.

Depends on how you define or what you mean by psychological problems. If they're hallucinations then neuroscience should continue to evolve tests for the brain abnormalities that underpin them and treat accordingly.
If they are problems of how best to handle and deal with life issues then I would suggest finding some wise person who you admire and respect who you think lives well and I'd suggest talking to them if you could. If that wasn't possible I'd suggest choosing a very lagging behind second choice and maybe talk to a psychologist.

Caveat: having said that it is possible to find the two in one package. I won't go into odds.

I'm not being flip here. I really am suggesting that we overlook what people have depended on for centuries. People who have good life skills and are willing to share insight. They are out there. But the whole business has been co-opted and commodified and turned into science.

 

[alex3619]

I think it's very important that psychologists don't overstate the findings of their research.

This is a critical point everywhere. I think its alarming that so much research passes peer review, particularly in psychiatry, where conclusions are not adequately justified. Its even worse if there is no way to test them effectively. When research devolves to only collecting facts to support a conclusion it cannot be called science. Such practices were stamped out in mainstream research more than half a century ago. Fact collecting for an hypothesis is an important early step, and necessary, but science goes way beyond that.

Science should be about detailed verifiable predictions, and that verification has to be based on objectively measured data. Those predictions need to be put to the test.

PACE for example can be considered an utter failure in a scientific sense, largely through methodological, reasoning and interpretation issues and poor results in the data, leading to failure to support their hypotheses. If its science its about a failed hypothesis. However I do not consider it is a scientific study. Its an academic study. Its large, and full of documentation, but it fails to meet what I consider to be rigorous standards.

Academic non-scientific studies can be conducted rigorously if effort is made to do so. They can also be conducted badly.

In dealing with complex layered real world problems, systems theory has a very old maxim, I think it goes back to the 1930s: the map is not the territory. That is you can build a complex representation of reality, but its not the reality just a model, with all the potential flaws that come with that. Many in the "soft sciences" are aware of that, and take it into account. Sadly many don't.

In psychiatry there are physical disorders that have cognitive issues. Such disorders are often valid diagnostically, at least at our present level of understanding. Most of the rest of psychiatry is made up categories with no proven real world validity. Most psychs know this, they just don't often talk about it. DSM for example is for diagnostic codes. That is, for something to send to insurance companies or for bureaucratic use. There is usually no scientific basis to such categories, no methods for objective diagnosis. Hence they have such a hard time figuring out what is going to work.

We need something like psychiatry. If it did not exist we would have to invent it. However what we do not need is psychiatry as it is right now. Its not only failing patients and society, its visibly failing patients and society. So long as many or most psychiatrists do not take an honest look at their own profession, and demand things improve, psychiatry is going nowhere, and will continue to be both a service and dis-service to society.

 

[alex3619]

But the whole business has been co-opted and commodified and turned into science.

Actually, its been turned into pretend science. Or zombie science, or McScience as its starting to be called. Psychiatry can probably be split into different streams. Some is scientific, some is zombie science, and a lot is nonscientific. Just because something is not scientific does not mean its without value or is not useful. However when we pretend its scientific, and put it on a pedestal, then we have potential problems.

There is a label for unscientific and nonscientific claims that pretend to be scientific, and to use that scientific label as a badge of authority. Its called pseudoscience.

 

[alex3619]

I think psychiatry needs to be split up, and some of it thrown out of medicine. We should be speaking of alternative psychiatry, just as we speak of alternative medicine. Medicine should be focused on the scientific. Many of the issues we see is bacause nonscientific claims are part of psychiattry, which is part of medicine, which many view as scientific. So legal, ethical, social, political and very human problems and issues can arise.

Psychiatrists, and doctors generally, need to lift their game, and hold all this to higher standards.

 

[alex3619]

This radio presentation starts with some fairly obvious statements. Then moves to things that are reasonable sounding but may not be supportable. Then makes reasonable sounding claims that may be completely invalid. Sigh.

For example, to avoid excessive testing for those labeled with a MUS, and for better treatment, she wants clinics or services to have better integrated psych behavioural methods available.

The cost issue is a big one in justifying the argument. All those test! Shocking! Yet those tests do sometimes find something. That is the point of the tests. Further as more and better tests become available it could be expected that more and more MUSes patients will have definitive diagnoses.

This is marketing of economic medicine, and managed medicine - both financially managed and managing of patients. Yet its also about parking patients in the too hard basket.

I think its far more honest, and sustainable, and useful, to simply say "We Don't Know" and to emphasize that until it is figured out treatment is going to be about dealing with symptoms. At no point should it ever be presumed that further tests should never be run.

Patients often have unrealistic expectations that doctors can figure things out. Government has unrealistic expectations that basically leads to them wanting to abandon sick people because of cost, and then presuming that this will not itself lead to more costs. Doctors often have an expectation of themselves that they can supply answers. They don't like admitting failure, or that is the way I see it. For me "I don't know" is not failure, its integrity and honesty.

There is also a huge deception going on about MUSes. As we know from ME there is verifiable and reliable testing for pathophysiology. Sure we do not know what is wrong, not really, and don't have validated diagnostic tests, but we do know about the physiology. I think its very deceptive to pretend that many or most MUSes patients have nothing provably wrong with them. Its just that we don't know all the ins and outs of the problems. EDS, IBS and fibro, and probably many more diseases, are similar to ME in this respect.

 

[MeSci]

Depends on how you define or what you mean by psychological problems. If they're hallucinations then neuroscience should continue to evolve tests for the brain abnormalities that underpin them and treat accordingly.
If they are problems of how best to handle and deal with life issues then I would suggest finding some wise person who you admire and respect who you think lives well and I'd suggest talking to them if you could. If that wasn't possible I'd suggest choosing a very lagging behind second choice and maybe talk to a psychologist.

Caveat: having said that it is possible to find the two in one package. I won't go into odds.

What I had in mind among other things was 'Alzheimer's disease'. This has physiological correlates in the brain, but I was astonished, after studying the issue for a Master's project, to find that 'experts' could not agree, from looking at scans and even post-mortem brains, on who had (had) Alzheimer's and who had/did not, or even who had dementia. Other surprising facts were that the supposedly-typical 'Alzheimer's' pathology correlated poorly with clinical presentation. More recently - and unsurprisingly to me following my study - it was found that removing one of the physiological correlates from the brains with a drug did not improve the patients' condition.

So scientists too are prone to unjustified assumptions - that clinical correlates are causal rather than consequential (on something).

But I guess that is just bad science. Sadly it is very common.

 

[MeSci]

I think psychiatry needs to be split up, and some of it thrown out of medicine. We should be speaking of alternative psychiatry, just as we speak of alternative medicine. Medicine should be focused on the scientific. Many of the issues we see is bacause nonscientific claims are part of psychiattry, which is part of medicine, which many view as scientific. So legal, ethical, social, political and very human problems and issues can arise.

Psychiatrists, and doctors generally, need to lift their game, and hold all this to higher standards.

It's rare for me to disagree with you, Alex, but it's an issue of nomenclature that I feel very strongly about.

The term 'alternative' contains an assumption. All it tends to mean, when applied to medicine, is that it is different from what has become the norm. Norms are created by those with power and influence. But the trouble with the term 'alternative' is that it has pejorative connotations - as though it is not as good as the norm, when in fact very often natural medicine - my preferred term - is superior to pharmaceutical or surgical 'alternatives' (sic) - and they are usually safer too. Many of them are also scientifically proven to work, and some that are regarded as 'alternative' in one country are mainstream in another, so it's really a false dichotomy.

So I would prefer an 'alternative' word if we are going to do the same with psychiatry!