feeling insecure about the college that i went to

I live in area with lots of talents and high achivers, but also lots of people who are somewhat snobby (myself included unfortunately). I noticed that my surroundings all went to a good college, Ivy league or prestigious schools in general. But I went to a state school in middle of nowhere, and whenever I got asked that question and I told people where I went for college, people tend to give me unpositive feedback about it, either through verbal or non-verbal cues. And along with that I sort of sensed that with my answer people kind of question my intelligence and family values/ background in general.

To provide more context, I am a foreign national that only moved to the States since college, and when applying for college my family didn't do enough research and just picked an area with my parents' close friends lived in. Eventhough it's a state school but being a foreign national I still paid international tuitions. But my family values education a lot and my parents always made sure to put me into the best schools in my home country growing up, so my social circle back in home country has always been filtered by the schools I went to / social economical status through neighborhoods or after class activities. So this demographic is what I am familair with.

But now when realized how frequent this "where did you go for college" always got brought up in my current social circle and how much assumptions people make base on the college that you went to (family value, intelligence, social economical status, social circle, personal value, career, etc.) I found myself feel insecure when getting asked that question and feel very uncomfortable and even inferior when receiving negative feedback about it from people that I currently socialize with. What can I do to change my view about it and stop feeling insecure?

I did go to a good graduate school after college, but seems like people don't really care or ask where did you go for Grad school (bc not everyone go to grad school) and people always ask where did you go for college. What should I answer to people who ask me that question to not giving off the impression that I am not in their league?

To provide more context, I have a decent job, I tend to socialize with people who have good career and come from good family (upper middle class) and went to good schools. Those people and I have similair outlook for life and financial mindset. In addition, I am not too smart but not dumb either, somewhere in the middle, I am not trying to impress, I just want to feel belong to the social economical bracket that i am in.