It seems the keys are: 1. How far you're riding. 2. How fast you're planning on riding.
I think you'll find the hands are the hardest part to keep warm, but I ride down to 35 by switching to a pair of nylon-shell Thinsulate lined hunting gloves. They keep my hands warm, but don't offer much in the way of crash protection.
As long as you don't mind putting on a bunch of layered, wind-resistant clothing, you should do fine with that as long as you're not riding great distances at great speeds.
I've never had a problem with my feet as long as I keep my boots tucked up tight against the engine.
I've considered heated grips but just really don't want to mess with them. One day I probably will invest in some better gloves with a good combination of insulation and padding/protection.
As long as you're not riding hundreds of miles cross-country, I think the above advice of good thermal underclothing, layering and snowmobile-type jacket and pants would enable you to remain relatively comfortable in some pretty low temperatures.