I've thought about this also.
I just bought an Alpinestars jacket, that has CE-rated "armor" - but it feels squishy to me. If I put the jacket on, I still wouldn't want to wail my arm into the edge of a door.
And it seems if they simply bonded the soft-ish padding that's in the sleeves with even a thin layer of something like polycarbonate - or maybe here's where a dry-layup (vacuum bagged) dual layer of thin carbon fiber would really shine.
I was wondering, having seen some pretty inexpensive CE-rated replacement jacket armor on Amazon, if you could take a set like that and modify it to be actually protective... more like those shin/knee guards up there ^^^