Excessive end play on the distributor shaft will cause at least some of it. You want to set the play to ~ 0.017"-0.020". I've seen them over 0.100".
Mr Gasket and others have shims for the HEI. The shaft is ~0.490" OD.
What you're probably seeing is a combination of the timing set, distributor end play AND the cam being forced rearward.