The main factor for heat transfer between a component and a heat sink is contact area. This translates to surface flatness of both pieces. Air has a much higher thermal resistance than metal-to-metal contact. The idea behind the heat sink compound is to fill in the minute air gaps with something of a lower thermal resistance.
If you want the best results, lap both pieces flat before assembly. Extruded heat sinks (which most are) are particularly bad, with little ridges running along the length. The method I use is to lay a wet sheet of wet/dry sandpaper on a piece of plate glass and work the piece against it. Start with 400 grit and work your way up to 1500 grit. You should end up with a near mirror finish.
Most people use far too much heat sink compound which can actually make the thermal connection worse than none at all. One method is to put a piece of plastic film (a baggy works) over a finger and rub a small amount of compound on both pieces. Work it in well. Than, use a paper towel or rag and wipe the surface clean. A small dollop of compound is then again applied to each piece. The dollop should be about half the size of a BB for the typical bridge rectifier block. Using a razor blade or an edge of a credit card, spread the compound to an even film. Applying adequate clamping pressure is also important. A small amount should squeeze out at the edges.
Heat sink compound can also be obtained at most computer stores. Gizmo is right, it's not a glue.
