Thursday, September 20, 2012

When is a good time to talk?

One of the things I like about blogging (seriously), is that occasionally I write about something that I think is rational and reasonable and shortly after that I realise that it conflicts with something else that I think is rational and reasonable. Or at least appears to conflict. It makes you think about your opinions with a little more depth than usual, I'm aware of cognitive dissonance, but not a fan.
Releasing un-reviewed papers or risk assessments direct to the public, places an unnecessary burden on not only a specific GE product, but on GE research in general. I am also of the opinion that scientists should try and engage more with the public - I seriously don't like the idea of demarcating science and saying "this bit is for scientists, that bit is for release to the general public", an attitude that is both underestimates those who are interested in science and distances science from public life where I think it belongs.

The question becomes then, when is it appropriate for scientists to engage more directly with the public? I'm new at this whole writing scientific papers business, but I get the impression that few scientists put pre-published results out to the general public. I would go so far as to say that results don't generally get discussed outside of a very small group. Results get talked about widely after peer review, if they make it past. Any results that do make it past should be open for the public to access - whatever problems there are with the peer review process.
A lot of the communication of science that I have seen and the method that I try (not always successfully) to emulate is speculative. Telling people about what this particular piece of research might mean, how something that we're doing might help with a particular problem. This is a good thing, long gone are the days of the scientist who can answer every question with authority. It needs to conveyed that the larger part of what we do is question. Its not as much about the message then but at about what stage in the process you decide you have a message is it when you start to question or is it when you have poured over the other research and your own data for months on end, discussed it with some of those who have done similar work?

After some thought, I don't think my two positions are in conflict. The results science produces should be open to all, public and scientists alike. As should the process. I'm not so sure that a result can or should be released without some independent checking, i.e. peer review.
This all changes when the commercial world rears it's head. The commercial world doesn't want to let everybody know their secrets lest someone copy them.  I think it's overdone and quite silly sometimes, but I'm generally fine with that. Prof. Heinemann suggests that "This boundary has long ago been transgressed by the commercial sector, using scientists and writings of scientists in ads and other ways". And I am tempted to agree. I think the commercial sector has gone a step further though. They use the appearance of scientists and imitations of the writings of scientists. And it's not just the commercial sector that does it, special interest groups that use these tactics as well (using both scientists and the appearance of scientists).

This causes a problem. Without a transparent regulatory process then it becomes difficult for the public to tell good science from the appearance of good science. When it comes to products that have potential for harm, then regulatory agencies need to both put their assessments through review and publish the results. Prof. Heinemann suggests that this might slow the progress of good products and differentially penalise small business. This doesn't hold up in my mind. If a product has potential for harm, then there is cause to believe that it might not be a good product - something that should be checked. And the time compared to the cost of developing a GMO, I don't see the time and cost of an independent, reviewed risk assessment being that onerous (I could quite easily be wrong here).

All of this boils down to three points:
  1. Open, transparent regulatory processes
  2. Full and open communication of how the risks and results are being assessed over the whole process.
  3. Full and open communication of risks and results after review - there's enough confusion about from commercial and special interest groups already, best practice should be to reduce it if at all possible.

This is still a work in progress in my head, so criticisms would be welcome.

2 comments:

  1. Please, we need more from this blogger! Great read.
    I think we need more categories than "peer-reviewed" and "not peer-reviewed". Peer review is described by our regulatory agencies as seeking comment from peers, that is, those with, in their view, the scientific background to make informed criticism. A tremendous amount of science is and always has been limited to this form of peer-review (e.g., developers studies on their products that are the basis of safety assessments and later, advertising their benefits; government commissioned and internal reports; and my recent report used this form of peer-review).
    Those of us in academia primarily (but not exclusively) use anonymous peer-review. This requires someone serving as an intermediary to protect the identity of the reviewers. While all types of peer-review have their weaknesses, I agree that anonymous peer-review can be the most stringent, but anonymous reviews are also sometimes guilty of using their anonymity to suppress good work too.
    Nature magazine devoted much thought to the issues of peer-review recently. Check it out here http://www.nature.com/nature/peerreview/debate/
    Hybrid systems are emerging. For example, PLoS considers that the traditional anonymous review system as practiced by many journals allows too many non-scientific, normative, judgements to determine what is publishable. "Too often a journal's decision to publish a paper is dominated by what the Editor/s think is interesting and will gain greater readership — both of which are subjective judgments and lead to decisions which are frustrating and delay the publication of your work. PLOS ONE will rigorously peer-review your submissions and publish all papers that are judged to be technically sound. Judgments about the importance of any particular paper are then made after publication by the readership (who are the most qualified to determine what is of interest to them)." That model is discussed more here http://www.nature.com/nature/peerreview/debate/nature04992.html.
    What it comes down to is that there is a long standing acceptance of peer-review that is not anonymous and peer-review that is, and those most familiar with one tend to support that way. Personally, I think both have a place, but the end goal must still be a quality assurance. Neither form of peer-review is clearly better on this. There are both good and bad anonymous referees, and we know that many papers that go through this process are still disputed just as vigorously despite having been approved by anonymous review. Same for the other method. What we need is active engagement with the science as much as will the process of reviewing it. We cannot excuse ourselves from being critical just because one form of peer-review was used any more than we can dismiss a finding just because another form was used.
    Thanks again for raising this important topic.

    ReplyDelete
  2. The form of peer review might need to be changed, but I'm generally happy with the idea of peer reviewed/not peer reviewed. Or at least the idea of a control mechanism. Without a gate keeping process/device of some sort it is to easy for the commercial and special interests to muddy the waters. Peer review doesn't necessarily stop that but it at least it means the discussion is not a free for all where it's impossible to tell informed from uninformed opinion.

    Judgements regarding the importance of a work have always been made after publication as best I can tell - that's what I've always thought citations were. Again not necessarily the best way to judge the importance of a work, but it's what we've got.

    The problems with peer review weren't what I was trying to get at here though. I was more thinking about the process of science and scientists interacting with society. We don't generally release our data to other scientists before review/publication so I don't see releasing it to the wider public before then being necessarily a good thing. After that initial step though, when it has been vetted for technical rigour, then yes, I think work (and data and methods etc) should be released to both the science communities and society in general.

    ReplyDelete