Replace is the distributor. It's not the shaft that drive the pump giving you problem, but the housing where the bottom part meets the block. If the distributor does not properly sealed you will loose oil in lifter valley. That is part of an oil passage.
I had to replace the distributor twice because of low oil pressure on my 307sbc and again on my 406sbc.