Well, if you are literally looking for the fastest possible hammer, then you should be skipping all of those other modifiers and just using redstone redstone redstone. And redstone.
Furthermore, in my experience (which is based on pickaxes, not hammers, so it may be different) one full set of...