I wrote the following post for the now-defunct americanscienceblog.com in November 2014. But I enjoyed writing and rereading it so much that I wanted to preserve it here.
Like a lot of us, I’m applying for jobs. In practical terms, that means I have been firing a lot of PDFs into the cloud, with no reason to believe that anyone will ever read them. Months go by with nary an automatic email of receipt. So when, one recent day, I heard from an institution’s search committee, my surprise was so complete that my mind could only grasp its dimensions through an analogy. Or, more precisely, through an analogy to an analogy.
What immediately popped into my head was a famous description of a rather more important shock in the history of science: Ernest Rutherford’s astonishment that alpha particles—today aka helium nuclei—reflected off a piece of gold foil. Being told of the reflection “‘was quite the most incredible event that has ever happened to me in my life,” Rutherford said in one of his last recorded lectures (he died in 1937). “It was almost as incredible as if you fired a 15-inch shell at a piece of tissue paper and it came back and hit you.”
I wrote the following post for the now-defunct americanscienceblog.com in September 2014. But I enjoyed writing and rereading it so much that I wanted to preserve it here.
One frigid February evening, I arrived home from a long day of dissertation-writing and fellowship-applying at my MIT office, and settled down with some kind of takeout to watch the latest episode of FX’s “Archer.” I thought I was done with the history of metrology for the day. I was so wrong.
Yesterday, along with the other faculty members of UVA CLEAR (the Corruption Lab on Ethics, Accountability, and the Rule of Law), I participated in our lab’s big launch event at the Miller Center. The room was packed—it turns out corruption is a hot topic among a certain crowd these days. Who knew? (It was a fun game trying not to mention the T word, the I word, or the U word.)
My contribution to the conversation was to historicize definitions of corruption at a larger scale, all the way back to the London docks in the 1600s and anti-Reconstruction in the 1870s. But academics weren’t what drew the crowd.
But by far the best thing I found was filed under “Collected Artworks“: this cartoon of an astronaut in an early-60s capsule radioing mission control to say he’d spotted a 10-pound bag of sugar floating in orbit.
Two weeks ago I delivered half of the weekly Medical Center Hour humanities talk at the University of Virginia’s medical center. The title of the March 6th hour was “Sugar!”, and the other half was given by Jennifer Kirby, an endocrinologist at UVA. You can see the talk below (including closed captions).
There’s no abstract, but here is the introduction:
The reproducibility crisis in the age of digital medicine
If anyone doubts the explosive growth of interest in digital medicine, consider a recent conference and workshop in Beijing, jointly organized by the People’s Liberation Army General Hospital and MIT Critical Data to showcase the opportunities and challenges of applying machine learning to the kind of data routinely collected during the provision of care. In person, 500 attendees heard a keynote and panels and participated in a health data hackathon. Online, however, the event was streamed to more than one million unique viewers.
As databases of medical information are growing, the cost of analyzing data is falling, and computer scientists, engineers, and investment are flooding into the field, digital medicine is subject to increasingly hyperbolic claims. Every week brings news of advances: superior algorithms that can predict clinical events and disease trajectory, classify images better than humans, translate clinical texts, and generate sensational discoveries around new risk factors and treatment effects. Yet the excitement about digital medicine—along with the technologies like the ones that enable a million people to watch a major event—poses risks for its robustness. How many of those new findings, in other words, are likely to be reproducible?
Digital medicine must take steps to avoid a reproducibility “crisis” of the kind that has engulfed other areas of biomedicine and human science in the last decade and shaken public confidence in the validity of scientific work. The goal of this paper is to use a historical perspective on reproducibility and its current crisis to suggest how digital medicine can avoid a reproducibility crisis of its own.