Just the facts…..
The story is told of a scientist walking down a country lane with a friend. The friend comments, “Those cows sure are a pretty shade of brown.” To which the scientist replied, “Well, they are brown on this side.” The point is, the scientist could not see the other side of the cows and was unwilling to comment on something for which there was no direct observation.
The core of science and medicine is data. “Evidence based medicine” is the hallmark of the day. This begs the age old question, though, “What is truth?”
Trying to follow the data in a particular field (say, for example, colorectal cancer) is like watching a tennis match. A lot of back and forth before anything is decided.
The fact is, scientists are human. And tests are complicated. And the right kind of statistical analysis can make the numbers look beautiful. Much as we cherish the image of a researcher as a rigorous “I can only see this side of the cow” kind of person, the reality is shaped by a lot of factors.
Here are some of the issues faced by science:
--What question to you ask? Suppose you are eating a nice, juicy apple and I ask you, “Please answer yes or no: Is that thing you are eating round or is it an apple?” What kind of data would I get? Asking the wrong question may result in beautiful data, but the data is likely to be meaningless. What if you combine two chemotherapy drugs, giving full doses of each? Will that make treatment twice as good? Turns out that some combination therapies work better than single therapy. Some work the same. Some are less effective. It is all in the balance and combination. But if you look only at the first question you could conclude that combining therapies doesn’t work.
--How much data is enough? Good scientists never like to stop accumulating data. At some point they must, painfully, simply report their findings. The challenge is being sure you have enough data to make the results meaningful. We all know that if you flip a coin 10 million times, all things being equal you will get “heads” half the time and “tails” half the time. We also know that you can flip a coin three times and get “heads” all three times. Several months ago a report came out that combining some supplements with a particular therapy interfered with that therapy’s ability to work. A few months later another report came out showing that those supplements actually make the therapy work better. The difference? More data.
--Aberrations. The worst thing to happen in research is to have one or two outliers. All your data lines up neatly. Your charts look great, your graphs are nice and orderly. Except for those one or two data points that don’t fit the profile. It is terribly tempting to dismiss those data points. To ignore them and hope they will go away. In all likelihood they are nothing. Unless they aren’t.
So data and truth aren’t necessarily the same thing. I am not saying that scientists are lying. Despite a few high-profile cases of researchers fabricating data, the vast majority of investigators do what they do because they have a passionate, burning desire to find out stuff. I do believe, though, that data in and of itself is a trap. Truth comes from lots of different studies over a long period of time. We can move medicine forward on data, but should not be confused that we truly understand the truth of all that is happening in the human body.
This weekend is the annual meeting of ASCO: The American Society of Clinical Oncologists. This is the meeting where all the new information on cancer research is presented. Thousands of talks and poster sessions and conversations. Over 30,000 cancer docs from all over the world. The news will, no doubt, report on several new findings, on new studies with new data. A lot of it will be exciting. And some of it will even represent the truth.

