The material from that cam lobe and lifter (and probably some others) has travelled through your engine. It has scored the crank journals and cam journals. It has imbedded itself in your main, rod, and cam bearings. Some, if not all, of the pistons have metal particles imbedded in the skirts and will remain there to scratch the cylinder walls unless they are removed by glass beading the skirts. The cylinder walls are also scratched by the metal particles and will need to be honed. Your oil galleries are full of abraisive oil. Plus, the rocker stud will never stay put until you replace it with a screw-in stud.
You will have to remove the engine and tear it down to repair it, regardless of what the old man says.