Tuesday, September 18, 2012

Where's a good place to talk?

There's been a bit discussion over at sciblogs and a few other places that have got me thinking. Recently Prof. Jack Heinemann released a report, being a risk assessment of a form of GE wheat that the CSIRO in Australia has developed. It has been criticized, for amongst other things being presented direct to the public rather than appearing in a peer reviewed journal. Which in some peoples opinion makes it an attempt at science by public opinion. The response to this argument is (and was made) that neither are the risk assessment regulators reports. And I agree that it's a good idea that risk assessments made by the regulators should be both peer reviewed and published. I'm not sure though, that this is an adequate defense against the criticism.

To fully appreciate a risk assessment, I think you need some background in the area. Prof. Heinemann certainly has this. I'm not sure that the general public does. I'm not suggesting that risk assessments should be locked up for privileged eyes only, that's pretty much what's causing the problem in the first place. Presenting them to the general public rather than in an appropriate venue after blind peer review is counter-productive though.

Presenting it to the general public sans peer review, even if it gets reviewed after it is made public, allows it to be presented without criticism, as indeed I believe it has been, as a tool to enable an argument from authority. Which I shouldn't have to say, is a terrible thing. In this case, even more so because Prof. Heinemann raises potential problems that should (and could easily) be addressed.

There is merit to the idea that a risk assessment should be published. I am firmly of the opinion that the argument we should be having is with the regulators - that risk assessments should be both peer reviewed and published in a suitable venue. That way valid risk assessments are a lot harder to dismiss as populist science by public opinion. And scare mongering press releases can be more easily dismissed. This is not suggesting that Prof. Heinemann's assessment is the latter btw. It is however suggesting that published where and how it has been, it loses credibility (as Prof. Dearden of Southern Genetics suggests). It becomes less useful as a piece of science used to help us mange our tools and our world and more useful as a PR tool, a piece of spin. Which, I think is unfortunate.

Prof. Heinemann says:
Science routinely shows prevailing assumptions, such as those made earlier about dsRNA, to have been wrong.  The proper response to challenges to assumptions is further research.  This, not denunciation of the challengers, is the way to maintain public trust in the regulatory system, and in science.
With this I concur. Further, open research is the proper response. Questions raised by research should be answered. Questions raised by press release though?  That imposes extra unneeded burdens on legitimate research.

3 comments:

  1. Ben
    I would say that there are different standards being applied. All risk assessments released by regulators are issued through a process of press release. When FSANZ issues an assessment, they mail that to anyone on their contact list including any journalists. The same for other regulators. Journals also issue press releases when they are trying to attract attention to a new issue or an article.
    Who is meant to decode these for the public?
    All writing meant to be read is announced somehow. Only secret science, covered by intellectual property or other kinds of restrictions, isn't somehow announced. The degree of notice does vary though.
    But for me your key point is that there must be a better way for everyone. I certainly would like to find that and keen to discuss further.

    ReplyDelete
  2. Hello again Ben
    Following on from our conversation last year, our peer-reviewed GM wheat report has become part of a blind peer-reviewed paper published by a top ranking journal. Interestingly, despite using this form of peer review too, the criticisms leveled at us haven't changed. Thus, the peer review argument used before appears to have been manufactured to mask minds that were not prepared to take on board the evidence we presented. Free download of paper from the journal's site. http://bit.ly/14i7pyG
    All the best

    ReplyDelete
  3. I wouldn't go so far as to say that the argument was manufactured as a defensive measure.

    To be fair, if I recall correctly, at the time the discussion was under way, your article hadn't been reviewed/published. If it had been published before going to the press, I don't think the arguments would have appeared.

    As in, again, if I recall correctly, two strains to the argument - one regarding the actual science, which I understood but didn't comment on and one regarding the manner in which we, as scientists go about things.

    Publishing means that the former argument moves into the traditional realm of, shall we say, trial, for a scientific paper. It doesn't invalidate the discussions around the methods by which we communicate science to the general public.

    Will have a look at the paper sometime shortly - mid-PhD life is ... hectic atm.

    ReplyDelete