Give the opportunity to the people who love your app to reach back to you.
The ones who are willing to reach back to you are:
1. The power users, the one you should try to get a hang of. The ones who are going to tell you if there's a problem. The ones who are going to tell you what's the best feature to add.
2. The really annoyed users. If there is a big and real problem most people are going to leave your app and never come back without letting you know. A small percentage of people will first try to reach out to you before giving up on your app. Don't ignore them! By promptly resolving the issue and by being open you can turn them into your most loyal customers.
Remember that even if you originally made the app for yourself, when you put it out there it became something more. Real people are using it and you should at least take the time to listen if you don't have the time to answer to everyone.
If people want to use it in some way to didn't anticipate who are you to deny it, just based on principle? Of course, I'm not saying that if one user asks for a feature you should implement it, it should always be something that enriches the app and follows your vision for its future.