This really was the best decision we made before launching:
We recently launched VimTricks, an email newsletter serving up tips and tricks about the Vim text editor. After reviewing many email platforms, we went with Substack for it’s easy of use and seemingly well-documented path to transitioning from free to paid newsletter.
One problem we wanted to solve before we even launched was how to know if the tips and tricks we published were too easy or too hard. It’s really impossible to know the skill level of our audience without feedback. To accomplish this, we ended up adding a little section at the bottom of every email that looks like this:
Was this useful? Help us improve!
With your feedback, we can improve VimTricks. Click a link to vote:
- ?Great trick, very useful. Thanks!
- ?I already knew this, but still helpful.
- ?Not interesting to me.
Each of those bullet points is a link and each link goes to a very simple, 10 line web app written in Ruby with Sinatra and hosted on Heroku. The app records the question and the response (3, 2, or 1) in a simple Postgres database. This isn’t exactly “no code” but it is “not very much code”. The whole thing took a couple of hours to put together.
With the data in hand, we were able to slap together some dashboards in Metabase. The results get displayed like this in a live dashboard we can keep tabs on after every VimTricks issue is released:
With this, we can easily see the reaction to each post as it comes in. It’s been remarkably effective and extremely helpful. When the posts are too easy, we get a lot more “I already knew this, but still helpful.”, which show as yellow on our graph. Interestingly, we get the same 5 or 6 votes for “Not interesting” on each post (maybe the same people!).
We’ve been averaging about 10% response rate, which is better than I expected. And it’s drastically more helpful than the ❤️’s available in Substack. Only 2 or 3 people click those, but more than 100 click our feedback links, which offer a much more helpful level of granularity.
Having this at launch was the best decision we made and we’re looking forward to tailoring VimTricks to the responses we get.
