My rule of thumb is; I never ride when it's below 50 degrees, but ahh, sometimes 60 degrees is my limit. But if I must ride in the the 30's and 40's, I wear a TourMaster suit which is similar to a snowmobile suit. It's basically a sleeping bag with arms and legs. I've riden in below freezing temps while wearing it and still managed to be comfortable. Suits like it offer little armor and no crash protection though.
The way I see it, when it's that flippen cold, a crash is a maybe, but freezing to death is a gaurantee. So I take my chances and ride warm, looking kinda like the Michelin Man in the process.
In a worse case scenario; I eat an Oldsmobile at 65 mph and the feathers fly, at least I'll die as snug as a bug in rug.
Death is inevitable, but pain is something that I'd rather avoid. Make mine fast please, make mine very fast.