Assuming the natural gas meter was at zero when it was installed, I use exactly 1/2 as much natural gas as the previous tenants of the house.  We are all away from the house during working/school hours on weekdays, so the thermostat allows the temp to drop.  It also drops overnight, but not during the day on weekends, when we are around.

The issue of thermal mass is a tough nut to crack.  The variation in house temperatures during the day is only 4-5 degrees C (7 to 8 F).  The daily variation in external temperatures, where I live, is about 10-15 C (18-27 F), and in winter it falls in a range far below the house's mean temperature.  At -20C (about 0F) there is a wide gap between the house temperature range and the outside, no matter what time of day it is.

Tweaking the timing of the house's temperature changes according to the sun's schedule, instead of the family's schedule, may lead the "gremlins" to make adjustments of their own.

Unless away on vacation for several days, I wouldn't set the thermostat below 10C, because there are many subtle consequences of reducing the house temperature.  Pipes under the house hover between ambient and house temp, so reducing the house temp too much might allow them to freeze.  Plants, electronics, perhaps other things that don't come to mind right away, might be affected by long periods of cold.

Oh yeah, some houses have humidifying systems.  If the absolute humidity of the house is high, and the temperature drops, then the relative humidity can quickly go to 100%.  If this happens in the winter, ice can form on the insides of the windows.