It seems that heavy usage over time would do damage. That seems fairly obvious, to be honest. What would be surprising is if a single dose had long lasting negative effects.

You have to also consider that people like me are essentially broken. Being completely unable to socialise with people puts you on a path of loneliness and misery. Maybe curbing some of my brain's "abilities" is exactly what I needed. I've often noticed that stupid people are happier.

I didn't become stupid, though. If I became stupider then it doesn't matter because that is completely offset by being more able to apply my skills properly. There's no point being a genius if you can't communicate with people.