Goggles are used in dirt riding to prevent dust, dirt and roost from coming up through the bottom of the helmet and entering the eyes. Yes they also protect front debris, but so would a visor. But a visor doesn't protect under debris, hence the only choice to use goggles for dirt bike riding.
Street riding does not have these concerns. Even low speed off roading does not have these concerns. I would say most of the riding people do with KLR's does not require goggles. Dirt helmets are also made to fit more snug so the helmet won't move much.
A proper fit dirt helmet will be uncomfortable for extended periods. In most aspects, goggles would be extremely annoying and uncomfortable for extended periods. They are constantly putting pressure on and rubbing against the face and nose. When you stop, you must pull the goggles off vs just flipping a visor or opening a flip up front. Then you have to get them on again if the band slips off the back. If you have glasses, you can forget about them, even the OTG ones because I have a pair for my KDX and they are uncomfortable as hell.
A visor allows adjustments at many positions to allow air flow without exposing the eyes. I've yet to experience a situation where I could not get adequate air flow to keep things cool.
Some people dislike flip up helmets, but they offer an advantage when stopping during dual sport rides. You just flip up the helmet to get more air. No need to remove a pair of goggles and then have to take the helmet completely off to get air while stopped.
There are advantages and disadvantages to either option. You already mentioned how debris can find it's way in if the visor is open for ventilation. That said, everyone should wear some type of eye protection even with a visor helmet. However, with a visor you won't need full on goggles, just a pair of sunglasses or other eye protection made to fit comfortably directly on your head, not over the helmet. You won't have to remove them in order to remove the helmet.
Now, one could say why not use low profile eye protection with a dirt helmet. This would leave your nose, cheeks and forehead vulnerable to head on debris which could sting like hell and cause an incident.
With a visor, you have the option of closing it while in heavy debris and then cracking it open again for ventilation. The likelyhood of high speed debris finding it's way through a visor ventilation gap isn't great even though it does and has happened to everyone. I count my times on one hand and really wouldn't consider any as necessarily high speed.