At Air360, we always like to keep things as simple as possible. Here are a few (really) simple steps that will help you effectively measure how your app or website is making progress and when it's not.
For each product version update you are doing, you are always pursuing the goal
to make your product deliver more value to your users.
Here is a quick recipe on how to measure how your product delivers value and track it over time to see if your product efforts are contributing to make your users use and love your product more and more.
1. Define a user event that reflects product value
This might seem a bit abstract but it's actually a very practical thing to do.
Here are a few examples:
- Whatsapp helps you send messages to your friends. Hence a user event that would reflect delivered value could be the "send message" event. The more of these are done per day, the better you are doing.
- AirBnB allows you to book rooms. Hence, a good user event to track to measure your product value would be a "night booked" event.
If you were AirBnB and you just shipped a new version of your website, a good way to measure if the newer version performs better would be to keep an eye on how the number "night booked" event per session evolves.
Why not simply measure the # of nights booked?
The problem with measuring the number of nights booked is that it could be rising if you get more visitors than usual. Let's say you shipped your new AirBnB website just before holidays and you get a rising number of visitors. Just looking at the number of nights booked will show a rising trend when actually your website just got much worse to use.
So checking the ratio of night book event performed vs. unique sessions is a great way to remove noise here. We get a pure signal of how many percent of our users are booking a stay when they browse our website. Pretty simple and efficient right?
As you can see, these user events are actually the foundations of your business core value and as a result, they usually directly contribute to your North Star Metric. If you don't know what a North Star Metric is, please check our article here.
2. Track it in Air360
The event could be linked to a URL that the user navigates to when he completed the action. For example, for AirBnB it could be your user is taken to www.airbnb.com/booking/confirmed
Or it could be your user clicked on a button on your website
Or in your native mobile app
Air360 allows you to track any of these with a few clicks with no need to involve your engineers. No tech skill required 😍
To see how you can track these events with Air360, please check our articles here.
3. Make it visual and practical!
Now that we have our user defined in Air360, we want to track the popularity of this user event. In other words, we want to measure how often it is done by our users.
We're going to create a simple dashboard to do just this. If you don't know how dashboard work in Air360, check our article here.
Let's add a chart that allows us to measure how often the user event "confirm booking" is done per user per day.
Let's add one of these with an explicity name : Booking done per user per day. Sounds pretty explicit?
Then we choose the event we want to track which is the "Booking confirmed" event.
Now let's add it to our dashboard.
We now have a dashboard that reflects how our business is doing. This should be your compass to evaluate how the different actions to improve your website or app are contributing (or not) to your overall success.
In our example above, we can see that our new product release on April 25th 2018 clearly didn't help to make our bookings easier 😲
To learn more about how to use Air360 dashboard, please check our introduction on dashboards here.
4. What to do from here? 😶
Based on our previous example where we could see our bookings are decreasing, which is not good. We need to identify the "why" here 🔍
A few ideas of possible causes we could have here:
- The new search bar is not visible enough
- The search feature results are less relevant than before
- Properties displayed on the homepage perform less than previously
A good idea is to repeat the process we did above for other user events that are directly contributing to the booking. We could add the same chart than above using these user events:
- Click on search bar
- Click on a search result
- Click on a property listed on homepage
This is the kind of dashboard we like at Air360. It's incredible what we can achieve with 4/5 charts like these on a single dashboard! ❤️
5. Going deeper
Sometimes real life is a bit more complex than in the previous example and the problem can get a bit trickier to find.
When a metric is falling, the culprit could actually be hiding a problem that touches only some of your users.
A few examples (taken from actual customer stories) could be:
- Your search bar is not displayed properly on Internet Explorer 😲
- Your search results are performing poorly on users using 1920x1080 screen size 😕
- The booking confirmation button is broken on the latest iOS version and you didn't notice it 😱
Fortunately, Air360 gets us covered (one more time 😇) to find these kind of problems: Air360 automatically performs automated analysis for you on an hourly basis via the Insights feature. This allows to find these kind of problems before it gets too bad.
You can also find about these kind of problems with Air360 Data Explorer as shown below. When we are on a dashboard, let's click "Add a chart" and then, "Data Explorer".
The Data Explorer lets you explore any metric and split this metric by any dimension you want: Browser, device, country, screen resolution and so on.
In order to find if there is something wrong with a given browser, what we can do is track how many times the event "Booking confirmed" is done per session and split it by Session browser.
Pretty simple right? Now when we add it to our dashboard, we get something like this.
These charts will quickly let us see if a browser performs much less (or not at all) compared to the other browsers. For instance, in the chart above, we can see that something clearly got wrong on Chrome from May.
This should prompt you to immediately test your website or app with this browser.
Remember, always keep things actionable, if a metric is here and you can't do much about it, then it's not a metric, it's noise. Here are a few examples below of an action sheet we could create
6. Always reevaluate
As your business grows, it will be useful to reevaluate what your dashboard should show. Remember: Overtime, always reevaluate your goals and readapt. Analytics is an ever-going process, not a one-time event.