At the same time, many people (we'll call them "left-wing") claim that there is no agency behind wealth or poverty, and that therefore we should punish the "lucky" rich people to help the "unlucky" poor. This attitude is utterly simplistic and destructive in a society when taken to the extreme (the left-wing tends to be extreme).
"punish the rich"... "the left-wing tends to be extreme"; this is just blatant partisanship. There is nothing to support, just opinion... what productive conversation could start this way?
Yes, there are nut jobs. But, yes, you are engaged in a fallacious generalization. It's also called a prejudice. But here's a softer restatement of what you said for those of us who at least think we're not nut jobs:
"At the same time, many people (we'll call them "fellow humans") claim that there is sometimes wealth and poverty without agency and that those with means should help those without means, particularly in cases without agency or in cases with resistant intertia, because of the direct or indirect causal relationship between the source of the means and the reduced or diminished means of those living in or near poverty. This attitude is either selfish or altruistic on an individual level depending on the means of the proponent of this attitude and destructive in a society when taken to the extreme (some fellow humans tend to be extreme when you largely ignore fellow humans who do not fit your definition of extreme)."
How was that? I don't assume that all "right-wingers" like you might be are extreme. I'm related to many of them, friends with many of them. They have good ideas and a different opinion on fairness and entitlement. It's possible that everyone's partially right and it's a good idea to mingle more often.