A/B testing the hamburger menu label
...or how just one word can make a difference
In the Mobile UX course during my first NN/g conference, it was mentioned that in a previous study labeling the hamburger menu icon resulted in a 20% increase in user interaction with the hamburger menu.
The Problem
Getting Buy-in from the stakeholders
This info stuck with me even after the conference as I realized that hamburger menus were common across all of our pages and wondered whether they were hurting any of our UX metrics. I hunted for data sources beyond what I heard at the conference and put together a small presentation for the stakeholders, citing hamburger menus' lack of discoverability, potential accessibility concerns and low engagement that could bar the users from viewing our content.
Additionally, I reached out to our Analytics team to check whether our users were browsing different pages across our websites. As it turned out, one of our sites with about 80% mobile traffic didn't see a lot of browsing at all with most users staying on the homepage.
Stakeholders agreed to my suggestion of launching an A/B test after I pointed out such a test would be low risk and likely a quick win as it was quite literally just a single word.
Hypothesis
By labeling the mobile menu more clearly, we'd like to see an increase of mobile users exploring beyond just the homepage. We determine the experiment successful if we see more users engaging with content on the sub pages of the website.
Preparations
The page in question is localised for 6 languages, so I acquired the needed translations. Turns out the word “Menu” stays as is in a majority of languages, just with different variations on accents or umlauts. This way, the button would stay more or less the same size with no extra design changes required.
The Prototype
I created an interactive prototype for the Analytics team as a reference, showing how the button would look as well as determining the size. After coordinating with the development team, we launched the test about a week later on the website through Optimizely.
As there was an update to the website that would increase traffic, we timed the experiment to use this increased traffic to validate our thesis. After 3 weeks, the experiment was turned off.
Results
The data showed an increase of clicks on the menu button as well as an increase of sub page visits by 6%.
The design team has taken this info on board and applied it on new designs for our upcoming projects. These findings have also resulted in a new best practice guideline suggesting the use of a labeled Menu button where permissible.
← Back to Portfolio Overview