simple solution to - Trends in Data Fabrication

From: aaleshin
Date: 3 April 2012 18:44

Hi James,

My previous message on this matter remains unnoticed, but I also suggested a very simple solution to the data fraud: the crystallographers should submit to PDB  partially processed data, like unmerged partial reflections. These files are much smaller than the images, and only a few people in the world are capable to forge them. This simple solution would kill any attempt to fabricate crystallographic data. 

Alex

On Apr 3, 2012, at 7:11 AM, James Whisstock wrote:

Hi

I was thinking about the last statement in the Acta editorial  - "It is important to note, however, that in neither of these cases was a single frame of data collected. Not one.".  This brought me back to the images..  

To date there is no "global" acceptance that original diffractiom images must be deposited (though I personally think there should be).  Many of the arguments around this issue relate to the time and space required to house such data.  However (and apologies if this has already been raised and I have missed it), if our sole intent is to ascertain that there's no trouble at t'mill then deposition of a modest wedge of data and / or a 0 and 90, while not ideal, may be sufficient to provide a decent additional check and balance, particularly if such images, headers etc were automatically analysed as part of the already excellent validation tools in development.  

I'm sure there are a number of clever ways (that could be unadvertised or kept confidential to the pdb) that could be used to check off sufficient variables within such data such that it should (?) be very difficult to falsify images without triggering alarm bells.

Of course this would probably then drive those that are truly bonkers to attempt to fabricate realistically noisy false diffraction images, however I would hope that such a scheme might make things just a little more difficult for those with fraudulent intent, particularly if no one (apart from the developers) knows precisely how and what the checking software checks!

While it seems sad that it's come to this cell biologists and biochemists have had to deal with more and more sophisticated versions of the "photoshopped western" for years.  Accordingly, most high profile journals run figures through commercial software that does a reasonable job of detection of such issues.

J

Sent from my iPhone

On 03/04/2012, at 11:10 PM, Dyda  wrote:

I think that to review a paper containing a structure derived from

crystallographic data should indeed involve the referee having access

to coordinates and to the electron density. Without this access it

is not possible to judge the quality and very often even the

soundness of statements in the paper.

I think the argument that this may give a competitive advantage

to the referee who him or herself maybe working on the same thing

should be mute, as I thought article refereeing was supposed to

be a confidential process. Breaching this would be a serious

ethical violation. In my experience, before agreeing to review,

we see the abstract, I was always thought that I was supposed to

decline if there is a potential conflict with my own work.

Perhaps naively, but I always assumed that everyone acts like this.

Unfortunately however, there is another serious issue.

After a very troubling experience with a paper I reviewed, I discussed

this issue with journal editors. What they said was that they already

have a hell of time to find people who agree to referee, by raising the

task level (asking refs to look at coords and density) they feared

that no one would agree.  Actually, perhaps many have  noticed the  

large number  of 5 liner referee reports saying really not much about a

full length research article. People simply don't have the time to

put the effort in. So I am not  sure how realistic is to ask even more,

for something that at some level, is pro bono work.

Fred

http://www2.niddk.nih.gov/NIDDKLabs/IntramuralFaculty/DydaFred

******************************************************************************* [m

----------
From: aaleshin

Moreover, PDB does not need even to store the raw data, just validate their consistency with the scaled structural factors and then trash them.