My boyfriend hit me during sex

A several nights ago during sex, my boyfriend slapped me randomly. It was with an open hand, but it shocked me a lot. He's my first and I've never had anything like that. I leaved his house right after.

I talked to him the next day to explain that I was scared and upset, but he didn't understand why. He told me he was just being 'kinky'. I know that slapping in the bedroom is different than if he hit me in another context, but it really hurt my feelings. I feel somewhat justified for being upset with him, but also a bit guilty, because I'm worried that I'll always be too 'vanilla' to enjoy that kind of thing.

Any advice?