Science Seminar Report - Everyday Scientific Practice

The opening address, given by Professor Douwe Breimer, Rector Magnificus of Leiden University, concentrated on the necessity of an ethical code in everyday scientific practice. He highlighted many qualities that are indispensable for the development of good scientific practice: honesty, trust, reliability, objectivity, fairness, independence, transparency, accuracy, reproducibility, credit, openness and personal responsibility. If any of these values is not adhered to, the foundations of the research may be undermined and the scientific conclusions unreliable.

If I have seen farther than others, it is because I was standing on the shoulders of giants." — Isaac Newton, in a letter to Robert Hooke, 1676

As most new research relies to some extent on previous work it is also of vital importance that all scientists adhere to the same ethical standards. To 'stand on the shoulders of giants' you must be able to trust that the giants can take your weight.

All the qualities mentioned may seem obvious and easy to adopt, but Professor Breimer also highlighted the pressures and temptations that scientists are faced with, and how these can lead to various forms of scientific misconduct. Plagiarism is apparently still widespread, in particular in minor theses and dissertations, perhaps as a consequence of the wealth of information available on the Internet.

Other, subtler, forms of scientific misconduct are possibly even more rife than plagiarism, which is after all easily proven and condemned. The manipulation or selective interpretation of data are harder to identify and easier to succumb to, yet they could lead to false conclusions.

Sloppiness and negligence in research can also be dangerous, such as not designing experiments carefully, or being too focussed on the desired result, ignoring any other possible outcomes. Conflicts of interest, particularly when commercial parties are involved, can also lead to misconduct and they should always be clearly declared.

Luckily, given all the types of misconduct that can occur in scientific practice, the social context in which science occurs to a large degree acts as a self-correcting mechanism. The peer review process endeavours to ensure not only the quality, but also the novelty and value of scientific research, aiming always to extend human knowledge beyond what is already known. Moreover, in the Netherlands at least, each university has academic regulations regarding scientific conduct and if a violation is suspected a committee is appointed to investigate.

Paradoxically, despite these self-correcting measures, the atmosphere in which science occurs can also be the spur to misconduct. The pressures which researchers face, essentially to 'publish or perish', may lead them into the various temptations offered by cutting corners or the fabrication or manipulation of data.

'Publish or perish'
Scientists are generally judged on how many scientific articles they have written, and where these articles were published – journals such as Science and Nature scoring highly. If the scientists do not publish well, they are less likely to receive funding and future positions.

As a way of dealing with these temptations head on, Professor Breimer described scientific conduct courses that were introduced as part of the PhD and masters programmes in Leiden. These include training in good clinical practice, good academic research practice and the interpretation of data, and end up with the student producing a written code of conduct. This type of training could prove to be a useful blueprint for similar courses, both elsewhere in the Netherlands and in the UK. Despite the belief of some senior scientists that an understanding of good scientific conduct is intrinsic, many situations that researchers face which require moral judgement are not black and white cases, but rather grey areas which are subject to interpretation. Therefore in the current scientific climate where scientists are faced with a multitude of different pressures, a healthy understanding and respect for codes of scientific conduct should be a pre-requisite for any researcher, and is certainl y worthy of specialised t raining courses. Professor Breimer finished by recommending a useful resource for students and active researchers alike, the leaflet 'On being a Scientist' available at www.nap.edu/readingroom/books/obas.

Peer review

The best option, or the only option?
Before it is published a scientist’s work is reviewed by his peers in an effort to sort, judge and selectively incorporate the ideas of individuals into the wider scientific body of knowledge. Problems arise with this system however, as it is clear that neither the authors nor the reviewers of articles can always be completely objective and detached. During the workshop anecdotal evidence showed that influential authors, by putting pressure on editors, may be able to push through articles which might otherwise be rejected. From the other side reviewers may sometimes be guilty of rejecting or delaying publication of an article, either because of personal objections towards the researcher or in order to minimise competition. The general protocol of peer review, where the reviewers, but not the authors, remain anonymous throughout the process, fosters opportunities for misuse of the system. In this way the reviewers are not he ld directly accountable for their opinions, and it was generally agreed that this coul d be changed. Various suggestions were made on how the process could be improved, including the reversal of anonymity, where reviewers are not aware of the identity of the author of the article, and at the same time can be held accountable for their decisions. Essentially, more transparency in the review process was suggested, to avoid situations where articles are either wrongly rejected or accepted for political or competitive reasons rather than scientific ones.

Peer review
The authors of a scientific paper submit their paper to a journal. The editors of the journal then send it out for review to independent scientists who are themselves experts in the field of research. These referees decide whether the paper can be published in its current form, published after revision, or if it should be rejected completely. Generally the referees remain anonymous; they could be good friends of the authors, or rivals........

The radical idea of completely dispensing with peer review was also raised. Alternatives, such as institute driven or web based reviewing were discussed, but ultimately it was considered that these could not guarantee the quality of the research published as adequately as peer review can. The danger would be that uncontrolled and unreliable research would be freely available for open misinterpretation. However, the open access option, which has retained peer review but made research openly available over the Internet, was applauded. This was considered to be a good way of making scientific research more accessible, without necessarily having any reduction in calibre. It was suggested that perhaps new research fields should take the lead in this, and help to establish the open access option as valid way of publicising results.

Open access
Journals (usually web-based) which are free to read. This means that everyone interested in the research has access to it, without having to pay a, sometimes large, subscription fee.

At the end of the feedback session the general feeling was that, despite its obvious advantages, the peer-review system has several flaws. The nature of the process favours the status quo over innovation, and established names over junior researchers. It would be useful to find out exactly how common abuses are, perhaps by polling researchers on personal experiences, and then to lobby for changes which would promote a fairer system.

Statistical honesty

There are lies, damned lies and amateur statistics
Classical statistics assesses the probability that the result of an experiment could occur randomly by chance. If this probability is low enough, the experiment is considered to have 'worked' and the result 'significant'. However, different thresholds of probability, different statistical methods and different sample numbers all affect the final decision about significance, and borderline results could well be considered significant in one analysis but not significant in another.

Statistics can be misleading:
"According to statistics, humans have one testicle and half a womb"

It is in these grey areas where researchers could try several different methods of statistical analysis before they finally pick the one that shows the most impressive significance, rather than that which would most suit the experiment performed. The scientists present at the meeting accepted that in some fields this is a common temptation, mainly due to the pressures that researchers face to publish. Ambiguous results are rarely welcome, either in peer reviewed journals or in the media. This can lead to the deliberate abuse of statistics, but accidental misuse also occurs, as many researchers have no specialised training in this complex subject.

One solution that was proposed by several researchers was the use of trained statisticians to help design and carry out statistical analyses. It should also be established right at the outset of an experiment which statistical method would be most appropriate for analysing the final results. Finally it was generally agreed that a better basic understanding of statistics was vital and most scientists would benefit from appropriate training, if only so that they would know when they needed to contact a professional.

Funding

Blue skies or turbulence?
The process of funding research is similar to the peer review process, often with the same reviewers sitting on both editorial boards and funding boards. This means that the problems are also similar – the more established you are, the easier it may be to get funding; the more funding you receive, the more established you become, ad infinitum. This system could squeeze out the more junior researchers, who do not have friends in the right places. Again, more transparency in the review process would clearly be beneficial, with reviewers challenged over their choice of successful applicants. Securing funding for a substantial period is also difficult and this could be resolved by having rolling grants with clear milestones, which if reached would lead automatically to more funding.

As well as the disadvantage to the individual, the current funding system can also damage the pursuit of knowledge itself, as money generally goes to mainstream work rather than so-called blue skies research, which may be high-risk but innovative. In this way creativity is suppressed and the status quo is preserved. This could be avoided by the allocation of a certain amount of funding specifically for high-risk research, which may have a slim chance of succeeding but could pay off in the end. This 'portfolio' type approach is already being applied by the MRC in the UK who, as well as funding more traditional research, have a programme calling for blue skies research in particular fields. This must, in the end, be beneficial to science, because often the most interesting questions are those with unpredictable answers.

Vernieuwingsimpuls – Veni, Vidi, Vici
The aim of this scheme is to promote innovation in the academic research field in the Netherlands. The scheme is directed at providing encouragement for individual researchers and gives talented, creative researchers the opportunity to conduct their own research programme independently.

Another flaw with the funding system is the fact that some grant applications request money for research that has already partly been done. If the grant is successful the funds can then be put to other, possibly more risky, research. Although this was not considered a flaw by some, the views of the participants on this point were polarised. Some participants argued that lying is presupposed, and the system funds those who are good at cheating. Others, most prominently Mr drs Jan Staman from the Rathenau Institute, suggested that fairness should be presupposed and that when scientists cheat they undermine their own system. There was lots of talk of good principles but yet again the pressure that researchers face, to 'publish or perish', could result in principles taking second place.

Continue > Science and the media