A heatsink is supposed to dissipate the heat from the CPU to the surrounding air.
Quite a few hours have been put down to find the "secret formula" to the ultimate heatsink. There are, however, a few common things to look for when you are going to put your greens into a new heatsink.
The bigger surface area it has, the better it dissipates the heat generated by your CPU. What you want is a heatsink with lots of fins or pins and a shroud to prevent the air from looping.
A heatsinks efficiency is measured in K/W or C/W where K=Kelvin and C=Celsius. Lower K/W is better.
Let's assume we have a 0.4 K/W heatsink and we want to overclock our PIII 700 to 931. At 931, the PIII 700 puts out approximately 35W with the core at 1,8V:
0.4 x 35 = 14 degrees higher than your case temperature.
If you have a lot of high RPM harddrives, DVD players and other heat producing devices and a case that is not vented properly, you might have case temps at 30C to 40C.
This gives us a CPU temperature at: 40C + 0.4 x 35 = 54C.
It is obvious that such high temperature is not desired, and what you must look for here are other solutions to bring the temperature down.
A lot of retail heatsinks nowadays are being sold with a thermal pad attached to their bases. Remove that one and use thermal paste instead. But keep in mind that certain CPUs have a fragile "slug". Apply a thin and even layer of thermal paste, and your CPU temps might drop by a few to several degrees.
Another highly recommended, and free, way to increase the efficiency of a cooler is to reduce the thermal resistance between the CPU and heatsink: lapping the heat sink. I did this on a retail Intel heatsink just for fun, and I was shocked at how concave it really was. Not only did the temperatures drop 3C°, but it let me run my CeleronII566 at 918MHz instead of 892MHz!
When it comes to lapping the CPU core it self, I think I would be a little bit careful, both because of ESD and, as I just mentioned, the vulnerability of certain CPUs