I think you have it correct there. You would push it through (I'd screw the bolt in and push that, so the rubber stretches as it goes in) and then when you tighten it, the bolt will be pulled toward the flat flange, pressing the rubber out behind the fairing surface and pulling everything tight.
Regarding the buzzing, I wanted to mention that I created a post about this a while back. I'm not sure if it's the same with your bike, but with mine this was coming from the joint between the side and middle fairings. Just last weekend I finally got around to disassembling the front fairings, and creating rubber gaskets to line the joints where the side panels lay against the middle. The buzzing is now gone.
I made the patterns by getting them right on paper first, then cutting pieces from 1/64" sheet rubber (eBay). I'll post the patterns to my original post soon, so you wouldn't need to mess around with that process if you wanted to silence this. In the next few days you'll find a link here:
That was my assumption, but I figured it prudent to check in here. I am surprised it fell out and am guessing it wasn’t inserted properly at the outset. The one in the same position on the right side was clearly put in cockeyed and then tightened that way as it is pretty severely distorted.
As to the noise dampers, did you buy self-adhesive rubber sheet? Or just tuck in place and tighten the screws?
I am thinking of replacing the oil line banjos with the ones that pdwestman listed somewhere (got to find that post) to try to get more oil to the top end and less to the crank. If I order those, then I will add on one or two of these well nuts as I will be paying shipping already so adding on a cheap part isn’t so painful.
On that topic, are there any favorite suppliers for Kawasaki parts? I have used Partzilla and Bike Bandit in the past for parts for my Polaris ATV, but haven’t bought Kawasaki parts in nearly 20 years.