You would have to stretch the body? It can't be done? I am sure glad Carol Shelby or Richard R. Hovis didn't listen to that talk!
Here's a 302 Ford in a Bug eye Sprite, it doesn't get much smaller than a Bug eye!
But here lies the problem, it will take a boat load of fab and modification and those $20K estimates are pretty conservative if you ask me. The shops who gave you the $120K they aren't out of line either if you are talking about "building a car". Building that car by a quality shop who actually makes money (unlike a lot of rod shops who make none) it's going to be a six figure car. I am sure they were just blowing you off or setting a figure to see if you still are the real deal and they should even take the time with you. But either way we are talking a LOT of money.
So then we come down to the simple fact that most of these cars are built by the owner. I have been into a few big name rod building shops and I am always amazed at how "assembly line" they are building the same basic car over and over. When you throw something out of the ordinary at them they aren't ready for it, it isn't their thing. Stuffing a V8 into a Honda DelSol isn't their thing, period. So of course you are going to get them trying to escape the question with a big price or pushing you out the door.
My suggestion is to start building the car yourself and one day you will pull out of your garage with it done.