Subscribe in a reader

Cambridge MedChem Consulting

Detecting Bad Science with Data

A couple of interesting programs to listen to.

The More or Less team are highlighting an important area. https://www.bbc.co.uk/programmes/p0fpb87t.

For more than a decade there’ve been longstanding concerns about the credibility and reliability of science research. This “bad science” has often stemmed from poor data practice or worse. But statistics can also help us identify and understand some of what’s going wrong, whether that’s selective data-slicing or outright fabrication.

There is also a more detailed investigation on Radio 4 The Truth Police https://www.bbc.co.uk/programmes/m001lqvg.

For years, science has had a dirty secret; research has been dogged by claims and instances of fraud, malpractice and outright incompetence. Suspicious-looking data sets, breakthrough results that can’t be replicated, eyebrow-raising statistical sleights of hand, science has been undergoing something of an existential crisis.

This of course has a potential profound effect on drug discovery.

A recent study, reported in Science 28 August 2015: Vol. 349 no. 6251 DOI looking at psychological science, attempted to replicate published work suggests that 39% of effects replicated the original result. Also Amgen, tried to replicate 53 'landmark' cancer studies and failed to replicate the original studies in all but six occasions, Nature 483, 531–533 (29 March 2012) DOI.

A report by Arrowsmith noted that the success rates for new development projects in Phase II trials have fallen from 28% to 18% in recent years, with insufficient efficacy being the most frequent reason for failure (Phase II failures: 2008–2010. Nature Rev. Drug Discov. 10, 328–329 (2011))1. In a follow up article Nature Reviews Drug Discovery volume 10, page 712 (2011) it was reported that that only in 20–25% of the projects were the relevant published data completely in line with in-house findings. This resulted in extended target validation studies and in most cases project termination.