You mention that your vac. adv. is already connected to the manifold port, which is good. Do you know the specs on the can you are using? In order to make the most of the manifold vacuum setup, I had to use Crane Cams' adjustable vacuum advance kit (which also includes a vac. adv. limiting plate that is VERY useful).
The goal here is to get the vac. can set to peg out below the least amount of vacuum (which is at idle in gear of course). I set my 406 to 18 initial and with the vac. advance it idles with about 33 degrees total. The motor runs great like that (and the throttle blades are almost entirely shut). My 355 was the same - Crane Cams adjustable kit, 15 initial, for a total of 32 at idle.
The adjusting plate that comes with the kit is nice because you can limit the amount of vac. adv. that you get, and then you use the screw in the vac. port to adjust the "Hg that allow it to open.
All this will help your fuel economy too. I've found a lot better mileage after setting the vac. adv. up well. Although both motors still run a little rich at idle- but never load up or foul plugs.
Another thing: I drilled holes in the 670 HOlley for the 355 - it actually didn't seem to do that much for me. I had better luck getting the idle correct with the vac. adv., but the holes don't seem to have hurt me. The 406 is much hotter, only has a 750 holley, and I cracked the secondaries just a little. Have the primaries almost totally shut here, too, made possible by vac. adv.