It's a pain when people get to know you too well, because their frustration at your shortcomings is legit. It's not just something they perceive. No, it's real.
On whole, the benefit of being around you has to be more than the cost.
Why are so many people homeless? Because the cost of being around them is more than the benefit. Essentially, they've ended up being the kind of people who need more than they can give. It may have been abuse or neglect or any kind of mental illness. But, if they provided more joy than pain, lots of people would take care of them. Like we take care of our animals. They don't do much but give us something to take care of and get joy from.
If you don't have the number of friends you want it's because you perceive the cost of maintaining them to be too high for the benefit.
Relationships and close relationships are expensive in terms of time and opportunity cost. The time you're spending with one person is the time you can't be spending with every other person. When you constantly think you could do better, it's easy to not want to settle.
I'd have more friends if I could settle and treat people like a pet. You show up, you feed them, you support them and pick up their shit -- in return, they just have to make you smile once in a while.