REFMAC Riding Hydrogens

From: Paul Smith
Date: 5 March 2012 19:15

Hello CCP4 community,

I'm posting at large regarding a previously raised issue for REFMAC for which I cannot find the conclusion in the old threads.

Specifically, does REFMAC add riding hydrogens during default refinement?

Though I've not requested that any hydrogens be added, only used if in the input, I see the following in the output header:

REMARK   3  HYDROGENS HAVE BEEN ADDED IN THE RIDING POSITIONS

Does anyone know exactly what this remark is referring to?

Are riding hydrogens added for stereochemical restraints, or are their scattering contributions calculated regardless of they are explicitly defined?

Can the hydrogen behavior in REFMAC be more explicitly controlled.

Thanks,

--Paul

----------
From: Matthew Franklin

Hi Paul -

Refmac does indeed add riding hydrogens by default; see here:

http://www.ccp4.ac.uk/dist/html/refmac5/keywords/restraints.html#makecif

The REMARK 3 is referring to what you think it is: riding hydrogens have been added.  (They are not, however, written to the output file unless you request that.)

I believe the H atoms are used for both stereochemistry and scattering, although there may be some fudging on the scattering factors.  Adding the riding hydrogens generally gives you some improvement in R factors even with a good quality (i.e. stereochemically correct) model.

If you look at the link I gave above, you will see that you can do three things with hydrogens: never use them, use the riding hydrogens, or only use H atoms given in the input file (which you could of course tailor to your needs).

Hope that helps,
Matt

--  Matthew Franklin, 

----------
From: Garib N Murshudov

On the interface under Refinement Parameters there are three options

1) Use if present

2) Do not use

3) Generate all

if you are using script then default is generate all (add them riding positions). You can switch to other options:

make hydr all    # generate all - default option

make hydr yes # use if present in the file

make hydr no   # do not use

Controlling output

make hout yes  #   write hydrogens to the output file

make hout no   #   do not write - default optoon

I hope it helps

regards

Garib

Garib N Murshudov 

----------
From: Pavel Afonine


Hi,

On Mon, Mar 5, 2012 at 11:52 AM, Matthew Franklin  wrote:

Adding the riding hydrogens generally gives you some improvement in R factors even with a good quality (i.e. stereochemically correct) model.

and here are the results of more or less systematic test that prove this:

see "On contribution of hydrogen atoms to X-ray scattering"

here:

http://www.phenix-online.org/newsletter/

 

Pavel

----------
From: Tim Gruene

Dear Pavel,

you may want to add to the structures mentioned in [1] one or two
organic structures present in the Cambridge Database.

"Until recently it was customary to ignore hydrogen atoms throughout the
process of crystallographic X-­‐ray structure determination." [1]

'recently' as in 1997 [2]? Even though 1997 is probably a poor
estimation of the corresponding year...

Cheers,
Tim


[1] "On contribution of hydrogen atoms to X-ray scattering"
http://www.phenix-online.org/newsletter/
[2] http://shelx.uni-ac.gwdg.de/SHELX/shelx.pdf

- --
Dr Tim Gruene

----------
From: Pavel Afonine

Dear Tim,

good catch, thanks; I could craft that phrase more carefully! Although often it may not be quite fair to take phrases out of context: this newsletter article was written in the context of macromolecular refinement. And yes, "recently" may be a broad term -:)

All the best,

Pavel

----------
From: Robbie Joosten

Hi Everyone,

 

Pavel's statement is likely a bit of an exaggeration, but he has a valid (yet hard to prove point). The default in CCP4i was (and is?) to use hydrogens only if present in the input file. This is IMO not a safe default.

Because there were some reporting errors in the past (http://proteincrystallography.org/ccp4bb/message18808.html) it is hard to tell from the PDB when refinement with hydrogens became hip. Discussions on this BB show that at the use of riding hydrogens is still not fully accepted, especially at low resolution (where they actually help most).

 

Cheers,

Robbie

----------
From: Ian Tickle

> Pavel's statement is likely a bit of an exaggeration, but he has a valid
> (yet hard to prove point). The default in CCP4i was (and is?) to use
> hydrogens only if present in the input file. This is IMO not a safe default.

I agree: what I find confusing here is that the current CCP4i default
is indeed to use H atoms only if present in input, whereas the current
documentation (http://www.ccp4.ac.uk/html/refmac5/keywords/restraints.html#makecif)
says (quoting verbatim):

HYDR [ Yes | No | All ] (Default HYDRogens All)
   How to deal with the hydrogen atoms.
   Y - if hydrogens are present in the input file, use them,
   N - ignore hydrogens even if they are present in the input coordinate file,
   A - add all hydrogens in their riding positions and use them for
geometry and structure factor calculations. They also contribute to
the gradient and second derivative matrix of geometry residual.

The documentation in this case also corresponds to the source code default:

         MAKE_HFLAG = 'A'

Which do we believe: GUI or source code?  Personally I'm much more
inclined to trust the defaults in the code over either the
documentation or the GUI, because I have a strong suspicion that the
CCP4i defaults (and yes even on occasions the documentation!) lag
somewhat behind the "community view" on this as represented by the
current code version.  Or is this an incorrect perception?

> Because there were some reporting errors in the past
> (http://proteincrystallography.org/ccp4bb/message18808.html) it is hard to
> tell from the PDB when refinement with hydrogens became hip. Discussions on
> this BB show that at the use of riding hydrogens is still not fully
> accepted, especially at low resolution (where they actually help most).

Documentation says: "For low resolution and early stages it is better
to use MAKE HYDR N, i.e. do not use hydrogens even if they are present
in the input file.".

Cheers

-- Ian

----------
From: Paul Smith

Hello CCP4bb,

Firstly, thanks to all for your comments.  However, I'm still unsure how to sort all of this riding hydrogen business out.

Robbie's comments seem particularly apt:

"Because there were some reporting errors in the past (http://proteincrystallography.org/ccp4bb/message18808.html) it is hard to tell from the PDB when refinement with hydrogens became hip."

Is there any foolproof way to know if a recently deposited file was refined with riding hydrogens in REFMAC, especially since some such structures do not yet have publications associated with them? How about the value of NON-BONDED CONTACTS OTHERS? If that is other than NULL, does that mean riding hydrogens were present?

Also, how about refmac version numbers?  Is there a clear demarcation when riding hydrogens a) became available, b) became default, or c) became default in the CCP4i GUI?

Thanks again,

--Paul

----------
From: Tim Gruene 

Dear Paul,

I doubt there is a foolproof way - if the authors did not describe the
refinement procedure properly in their publication and the PDB file
lacks this information in its header, I guess you cannot tell.

Tim

----------
From: Garib N Murshudov

Hi

1) refmac's default option has been "generate all" starting from version 5 (it is around 14 years). The reason is for most tests generating hydrogens gave good R or geometry. And if you look at the original paper by Ramachandran (I think it was pulished in 1963) you see using hard ball approximation and including all atoms generated plot is very similar to what we know as Ramachandran plot. A little bit more work on included hydrogen plus phi/psi torsion angle restraints (in free form, not from Ramachandran plot) should generate faithfully Ramchandran plot. It needs to be tested properly and carefully. refmac version 4 did not have hydrogen generation option.

2) ccp4i's option has always been use hydrogens if present in file (I personally do not like this option: this option should be used with care)

3) If there is no non-bondedn ... others then it is very likely that hydrogens were not used. 

4) I am not sure that there is a fullproof way to know if hydrogens were added or not.

Regards

Garib

Garib N Murshudov

----------
From: Martyn Winn


I have changed the GUI default to be MAKE HYDROGEN ALL. This should be
in the next release.

In any case, the option is in the folder Refinement Parameters, so you
can check.

m