You may find it useful to read up on the German "passive house" standard. The first major point is nothing new, its just super insulate, put in triple paned windows, seal up all the cracks.
The problem in a very well sealed house is the air gets stale (CO2) and moist from people breathing, so a certain amount of air needs to be turned over each hour with the outside air.
The second major point is that incoming air should be pulled through earth tubes, so 20F outside air will be warmed to to perhaps 45 F from the 52F temps you've seen.
The third point is to have a long heat exchanger that allows the 45F air to slowly pass alongside the exiting house air that has been warmed by your heating system to perhaps 75F?
as the 75F air is leaving it warms the 45F air to about 60F. You then still have to heat the air from 60F to 75F, but the heating system and your fuel costs (whether its wood/propane/other) will be greatly reduced for the life of the house.
If your cave has cracks at the other end that will feed air inward, you may be able to seal it off by your house and draw 52F air from it (one way) as your "earth tube".
There's dozens of good ideas at "builditsolar.com". Best of luck.
