Recently there has been interest on this page in the scientific merits of positive reinforcement methods in training horses. Then this morning I read a post by John Saint Ryan about his reasons why he is involved in the International Society for Equitation Science (ISES), in which he gave good and valid reasons for his interest in the scientific study of horsemanship. So I thought maybe it is time I gave my take on the science of horsemanship, so people can put my views into context.
Firstly, it seems important for some people that before anyone can express a view; they must state their credentials (with the mistaken assumption that credentials validate an opinion).
I have a PhD in Physiology and spent about 15 years as a medical researcher in the field of experimental heart surgery (developing techniques for improved methods of cardiac surgery) and then in fetal endocrinology (the study of hormones in fetal development and birth). I have authored and coauthored dozens of papers in quality journals, held grants, received awards blah, blah, blah…. – you get the idea. So that’s about me.
Now let’s talk about science – what it is and what it isn’t. If you have not worked in science, there are some things you may not know.
The first thing I want to say is that science is a business. It’s a business like business is a business. It is not a group of altruistic people endeavouring to make the world a better place either for people or for horses. It’s a business. People’s careers, reputations, financial rewards, advancements and self-esteem are tied up in the business of science – not the science of science. A person’s success in science is as much tied into making the right connections, working in the correct field, giving positive reviews to certain people’s grants and negative reviews to others, and publishing anything and everything you possibly can irrespective of the merit of your work. Not only is it a business, but also it is often a cut-throat business.
The winners in science are often not the best scientists doing the best science. Too many times they are the best self-promoters doing less than the best work in their field. So don’t be impressed by scientists with high profile reputations.
The second thing about science that you should know, is that publication of a person’s work is not a guarantee that the work is good. Even peer review of a manuscript does not protect against bad science being published.
Probably the biggest problem that impacts on the quality of science is that a person’s career is linked to the number of publications. The quality of the work is less important than that it is published. The more publications, the faster a person rises to the top. The result is that a lot of nonsense gets published. Many scientists know their work is crap and hunt around for whatever journal is likely to publish it.
But the more damaging problem that the need to publish brings is scientific fraud. When a person has 3 kids and a mortgage, and his job depends on getting the next grant funding, the pressure to have top-level results from your work is enormous. Some people get creative with their results. I have personal experience of this when I discovered a colleague had made up some results. My colleague denied it, but couldn’t explain how he achieved the results. When I went to the director of the institute to talk about it, I was told to say nothing because the results were already published in a prestigious journal and a big 5 year grant for $20 million was about to be decided.
I believe scientific fraud is as common as fraud in business. And like in business only a fraction of it is ever discovered.
Now what has all this got to do with horses and horsemanship?
I have noticed in recent years that a lot of the principles of horsemanship have come under the scrutiny of behaviourial scientists. In addition, labeling a principle in horsemanship as “scientifically validated” seems to transform the principle from a theory to a fact, in the minds of many.
This would not be a problem for me, except that when I have reviewed the studies for myself, too much of the research is bad. Mostly this is due to poorly designed studies that do not allow the researcher to make the conclusions they claim. In many cases of poor studies that I have reviewed, it seems that the researchers have designed a study to prove their theory. However, this automatically builds a bias into the results and conclusions. In good scientific methodology, a study is designed to test a theory, not prove one. The difference in the outcome of results can be huge.
My theory (and it is only a theory) and general impression is that many of the workers in the field of equitation science come from departments of psychology and behaviour. It seems from reading their work that many lack the training and understanding of the scientific method and the rigour it requires. Designing good studies is hard, even when the answers can be black and white. But it is far more difficult when looking into behaviours that are affected by a multitude of factors. For this reason, behaviourial science needs to be even more rigorous than some other fields – but it doesn’t seem to be the case.
When the studies are badly designed, they are badly designed and it is no excuse for accepting the results as “scientifically proved”. It reminds me of the state that the field of epidemiology was about 50 years ago. It was poor science, performed by medical doctors who had no understanding of scientific method.
Please don’t misunderstand me. I believe, when done well, science is beautiful, elegant and shines a bright light on the truth. But when done poorly, it is little more than a confidence trick. The trouble is that few people can spot the difference.
I’m guessing that most of you reading this are not trained in scientific method and don’t work in science. The average horse person is put at a significant disadvantage when trying to dissect the truth when they hear “studies have shown” or “scientifically proved”. It’s a real problem that works to the advantage of the behaviourial scientists, and disadvantage of trainers, riders and clinicians whose experience does not agree with the “scientifically proved” facts.
I believe we should continue to use scientific method to find an understanding of how horses operate and how training impacts of them. However, I strongly believe there needs to be better training of the researchers in the basics of scientific method.
In addition, I feel scientists should take the lead from the very best horse people. Researchers should look at what the best horse people are doing and then use the power of scientific investigation to discover how these people achieve their brilliant results. This is more likely to lead to a true understanding of the nature of horses, rather than the present approach of behaviourist using their laboratory-type research to tell horse people how to train horses. They should study the best in the business to find out why those methods work and not study horses to tell the best in the business how they should train.
I love good science and as long-time readers may guess, I have a very analytical mind that questions everything. However, there is too much bad science in the study of horses and horsemanship, and bad science is no more than a biased opinion. If you are not in a position to determine what is good or bad science, I suggest you do what my PhD supervisor once told me, “Assume everything you are told is wrong, until you are satisfied is isn’t.”