Sorry, I thought you had the 12-803b regulator. From what I see the 12-803bp is preset (hence the p?) at 7 PSI. Is it the adjustable regulator with the pressure simply pre-adjusted to 7 PSI? I'd have thought that any adjustment would've been locked out, but I'm not acquainted with this particular regulator.
With your finger lifting the float, clearly you could easily exert much more pressure on the valve's sealing surface than the float (on gasoline) would be able to exert. You might want to try an underwater bubble test looking for defects in the float bowl or very small leakage paths through the seated valve. Remember, with the engine running, a very small amount of leakage can be accomodated by the engine's demand for fuel. With the engine not running, any leakage will result in fuel slowly trickling out through an open sight glass.