Don't fear the upvote button
The company I work for uses, like many other companies, NPS (net promoter score) to measure customer satisfaction.
The results are to be taken with a grain of salt, to say the least. On top of that, many customers don't take the survey or just reply: "I never recommend companies."
Every now and then, my boss forwards me lovely replies from happy clients. It's a nice little confirmation, but it doesn't change the way I do my work. I keep doing what I do no matter the score.
I feel the same way about the upvote button. It's nice if a post gets some extra attention, but it's not important. It won't change the way I blog.
Another thing when it comes to upvotes is that they don't say much, really. It's just a small piece of a big pie. You may have 500 readers who love your writing, but never even see the toast button in their feed reader.
I understand that some people choose to hide the upvote button for various reasons. Blogging should feel fun and free. If that toast thingy interferes with that in any way, by all means hide it if it makes you feel better.
But if you hide it just because you feel that the posts don't get enough upvotes, it's probably not the button that's the problem. That mindset will just shift to the next thing: not enough guestbook comments, not good enough stats, not enough reader mail...
Don't focus on collecting numbers, focus on delivering words.
Don't fear the upvote button, but don't praise it either.