Science Seminar Report - Case studies

Compiled by Dr Daniel Glaser, University College London, October 2004

Day 1: Everyday scientific practice

Peer review
Professor Alana Advisor has become disillusioned with the peer-review process. Her postdoc, Aaron Associate, switched to neuroscience from psychology, and told her that in his area most reviewers for journals sign their reviews, as is permitted by most journals. Enthused by this spirit of openness from a more human domain, she has started signing her reviews. At the Neuroscience conference this year she had a beer with Alan Autograph, the editor of the journal for which she most frequently reviews. When she asked him about her new strategy, he confessed that he had been systematically deleting her name before sending the reviews out to the authors. Horrified, she asked why, and he said that he was bored of authors ringing him up with personal attacks on the reviewers who waived their right to anonymity.

Swallowing her indignation, she switched the conversation to her PhD student, Anna Author. Anna had signed an online petition the previous year, undertaking only to send her work to open-access non-profit journals, and was giving Alana grief for wanting to submit their new results to a high impact-factor commercial publication. Alan is irritated by this approach. While he concedes that major journals may have been guilty of profiteering, they add value and play an essential role in the peer review process by offering an impartial third voice, elevated from the petty rivalries which would otherwise rule scientific discourse. Alana is reminded of a recent paper in the highest profile of Alan’s journals, where, as far as she heard, all three reviewers argued against publication but the 'sexy' story was published anyway, and wonders whether she is in the wrong business. She comforts herself with the fact that her HR department told her in a recent meeting that anonymous peer review wa s probably not going to last long anyway since the Data Protection Act meant that, as with job references, the author has the right to read anything held on file about her.

Statistical honesty
Classical statistics depends on finding evidence to falsify a null hypothesis. Traditional scientific rhetoric treats significant results as if they were true and a failure to reject the null as if the experiment had failed. A brain imaging study dealing with a basic question in cognitive neuroscience has just been published by a high-profile journal and as is traditional, a p value of 0.05 was employed. The researcher gets a call from a colleague who is a about to run a trial of a controversial new drug which may have a dramatic effect on the reasoning skills of children with learning difficulties. She is suddenly nervous because she feels that she should adjust or at least check her statistics which were appropriate for basic research but she feels might be too lenient if an actual drug is going to be administered based on the results. She remembers that she actually tried five or six different analysis strategies and measures on the data before she came to the one which 'worked' producing significant resul ts which were featured in the publication. She tries to remember what a Bonferroni correction involves, but rather fears that it would result in not being able to publish anything from the expensive and time consuming study. She thinks fondly of the 'talking dog' result (if you produce a talking dog no one would ask you what the sample size was from which it was drawn) and wonders again whether the statistical tests are not just window dressing.

Funding
Author takes swipe at scientific elite

Researchers forget that making errors is path to knowledge

Ian Sample, science correspondent Thursday June 3, 2004, The Guardian

The upper echelons of the scientific community were yesterday accused of 'usually being wrong' and guilty of 'a systematic resistance to discovery', at the Guardian Hay book festival.

The attack came from Nigel Calder, author of Magic Universe: the Oxford guide to modern science, a tome weighing more than the latest Harry Potter book and short listed for the 2003 Aventis science book awards. Calder, whose swipe was a rare example of a science writer biting the hand that feeds him, was among the first journalists to work on New Scientist magazine when it was launched in the mid-1950s and went on to become its editor.

During his talk at the festival, Calder criticised leading scientists for having forgotten that big new scientific discoveries, which remain to be uncovered in many fields, can overturn widely held beliefs and [also criticised] the use of review panels that hold the purse strings of university research, were exclusive and had the effect of hindering rather than encouraging new discoveries.

'It amounts to a systematic resistance to discovery,' he said. Such 'self-appointed clubs that claim to be experts” supported the ... funding of mainstream work, rather than innovative science.'

He said scientists were wilfully resisting pursuing certain lines of inquiry because they could upset the balance of science research. 'The vast number of scientists are not even trying to do research that could lead to a Nobel prize because they don’t want to rock the boat.'

In other words, maverick researchers, by making discoveries that undermine the work being pursued by the scientific elite, could cause ripples many at the top would rather not witness. 'The top people may be toppled from their perches and people may lose their jobs,' Calder said.

Frank Close, the Oxford University astronomer and vice-president of the British Association for the Advancement of Science, said while scientists were far from trying to hinder new discoveries, it was possible review panels might at times be too conservative.

'Are there blue skies research projects that are not getting funded, but should be? Are we being too conservative? There’s always a chance of that, though I’ve not seen any convincing examples,' he said. 'And you have to bear in mind, this is taxpayers' money.'

Day 2: Ethics of public engagement

Who does the engaging?
A university press office is asked to publicise the results of clinical trials of a commercial drug. The trial was thought up, designed and conducted by three of the university’s professors in collaboration with researchers from other universities. The drug company was the main sponsor of the trial, although a government department and a charity also contributed some funding. The results of the trial have been written up in a paper which has been submitted to a medical journal. The journal has not yet accepted the paper.

In the meantime, the results are being presented at a research conference, and the university has been asked by the drug company to issue a press release to publicise the conference announcement. Two days before the conference, the press officer is contacted by a national TV science programme who say that they have heard from the PR firm representing the drug company and are featuring the drug in this week’s episode. They ask if the scientist would be prepared to describe the results in her laboratory. The Dean and the Head of Department recently had a meeting with the press officer in which the need to raise the university’s profile was discussed, and, separately, the charity which part-funded the work has a requirement that its funded work is disseminated to the public by the scientists. [With thanks to Jenny Gimpel, UCL Press Officer.]

Scientific risk and real world decisions
Volume 363, Issue 9411 , 6 March 2004, Pages 747-749
doi:10.1016/S0140-6736(04)15714-0
Copyright © 2004 Elsevier Ltd. All rights reserved.
The lessons of MMR
Richard Horton

The Lancet, London NW1 7BY, UK

Available online 4 March 2004.

This week, The Lancet prints a partial retraction – a retraction of an interpretation – from the majority of authors of a paper published in February, 1998, by Andrew Wakefield and colleagues. Wakefield and one other co-author, Peter Harvey, have not signed this retraction statement. We hope to publish their response very shortly. The original report made clear that the authors 'did not prove an association' between measles, mumps, and rubella (MMR) vaccine and a newly described syndrome of bowel disease and autism. But the authors did raise the possibility of a link, on the basis of parental and medical histories, and they suggested that 'further investigations are needed to examine this syndrome and its possible relation to this vaccine'. This interpretation of their data, together with a suggestion made by Wakefield during a separate press conference held at the Royal Free Hospital that there was a case for splitting the MMR vaccine i nto its component parts, triggered a collapse in confidence in the UK’s MMR vaccination programme. It is the interpretation expressed about a connection between the vaccine and the new syndrome that is now being retracted. Today's retraction comes after debate following the release of new information 2 weeks ago about the circumstances surrounding the publication of this work. An enormous amount of effort has gone into reviewing and analysing the events before and after publication of the 1998 article. It is now time to look forward.