There is going to be some free play in the pinion because of backlash settings. Different gear ratios will have different amounts. A 4.56 gear will have more than a 2.73 gear because of the difference in the size of the pinion gear. You are also feeling the slop between the axle and pinion gears and the splines in the axles and axle gears. What you are feeling here is a stack up of several things in the rear end that have some clearance in them. If you were to remove the cover and hold the ring gear completely still you would feel less slop because you would only be feeling the clearance between the pinion gear and the ring gear. If you were to hold the pinion gear completely still and set up a dial indicator on the ring gear you could accruately measure the backlash between these 2 gears. It should be between .008 and .010 inch on a used gear.
While you were turning the drive shaft did you check for any up and down or in and out movement in the pinion? If you have any movement in the pinion other than rotation then you have a pinion bearing problem or you have a loose pinion nut.