Did this reg come with a low pressure spring? From what I could see, this is a 4.5 - 9.0 PSI unit. If it did not come with a low(er) pressure spring, I'd still be a bit worried about the gauge.
You might also want to test the sealing of the float valve. Since you're not running the engine, any leakage thru the valve, particularly if coupled with very slight leakage at the regulator, would cause this inability to fully shut down the flow. Have you tested the float valve with the bowl off the carb?