Stopping air leaks is important, too. But that makes for stale air inside unless you have or install an air to air heat exchanger to force fresh air turnover and maybe a dehumidifier depending on the climate.
With the right insulation and air-tight construction you don't need supplemental heat.
http://www.thermablok.com/ The stuff is still more expensive than conventional insulation, but for irregular or curved surfaces it works well. Adding it to door and windows frames and stamped steel studs is an economical use for it.
This stuff has a big future in manufactured housing AKA "mobile homes."