I would imagine your hunch is right that the guy banked on you not wanting to go home empty-handed after a 2-hour drive. When you stuck to your guns, he got pissed and was willing to lose the sale rather than do the deal.
I don't know how much money we're talking, but can't imagine a 2011 with 3600 miles on it was offered for a particularly attractive price as compared to a new one. If it was a substantial amount less than a new one and seemed to be in good shape, I would be leery about why the seller was trying to unload it if he knows how much it's worth.
For me, too many questions with a used bike unless I've got a really good feeling about the person selling it. If I'm wrong about that, it's on me. I'd rather just buy a new one. If you're willing to lay out a substantial amount cash for a used 4-year-old KLR, you might as well just drop a substantial down payment on a new one and finance the rest if you don't have it or save up until you do.
I guess that is if you're sure you're going to like it and are going to ride it for years. If you wind up not liking it and having to sell it, you're going to take a loss.