Even though windows and doors don't consume energy, they can be a significant source of energy loss. A third of all heat from your home escapes from gaps and cracks.
There are a few ways you can check to see if you are losing precious heat in the winter or letting in heat in the summer.
Check for drafts using a thermal detector
Point the detector anywhere you suspect heat could be escaping or getting in. If you see a 5°C or more change, in either direction, you know you have a leaky spot. Check out Dave's DIY to see how a thermal heat detector works in action.
This video was filmed during the summer but the same draft checks can be done during the colder winter months.
Other ways to check for drafts
DIY smoke test: On a cool and windy day, light a stick of incense and hold it up close to the areas you suspect air might be leaking. If the smoke moves back and forth, or if it's blown out, there's a leak.
Noise check: Open the window or door a fraction and listen to the noise level coming from outside, then close the window or door. If you can still hear the outside noise clearly, there's a draft. A good seal would cut out most of the noise.
Blower door test: If you want a professional assessment, energy auditors can come in and use a powerful fan to evaluate the overall air tightness of your home.