I'm a huge proponent of user research; understanding why and how customers engage with a product is an incredibly important aspect of design process.
I've been fortunate enough to use a variety of research tools to inform my design work, some of which are highlighted below.
A/B and Multivariate Testing testing (MVT)
I used MVT testing to tailor editorial content and imagery in customer diagnostics flows.
A series of presentation formats for articles was tested, including splitting content over several pages and presenting it all on a single page. We found customers responded best when content was provided through multiple pages, though importantly this only applied if the instructions reflected 'phases' of activities. Where a single phase comprised a series of steps, these were more successfully engaged with when kept on a single page.
The visual style for imagery was also tested. Our team discovered customers responded better to photos compared with line drawings when required to identify a product, whereas line drawings outperformed photos when used to provide a visual representation of the written instructions.
Contextual study & analysis
I worked closely with a research consultant to set up a scenario in which we could test the effectiveness of a new format for self-service content. The first journey was looking at how customers would follow steps to solve a no satellite signal error, baring in mind the customer would need to switch between reading material and operating their TV and set top box.
The content was originally presented as a single solution, if that didn't work a series of questions were asked to show the next relevant solution; if that didn't work, another set of questions would lead to another solution and so on. What we discovered was this question-answer pattern was difficult for customers to follow in the context of following the instructions as it meant they had to switch between different 'modes' of operation. We switched to a series of upfront questions which led to the relevant solutions being shown back-back and this tested far better in subsequent testing. With this new approach, once a customer had answered the initial questions they could stay in 'instruction-following' mode and simply follow each set of instructions if the previous set had not sufficiently solved the problem. This made the process feel simpler as there was less context switching, which also benefitted the customer by enabling them to pay closer attention to the information in the instructions.
Another interesting outcome from the contextual research identified an important link that had been overlooked in the original plan. Effort was being ploughed into building an online self-service journey for no satellite signal, with a view to encouraging customers to go online to solve the issue themselves, instead of relying on a call centre operator taking them through the steps. As part of the contextual research, I simulated the trigger point for no satellite signal and found the on-screen-message that would appear was explaining to customers they should call the contact centre. Virtually no amount of improvement to the online journey would encourage customers to go online if they on-screen prompt was asking customers to call Sky instead! As a result of the finding, I put together a business case with the project manager to update the on screen message on the relevant set top boxes in order to direct customers into the online journey.
Eye tracking
Most recently I've used eye-tracking studies to help shape the layout of a home screen and navigation pattern. The team wanted to understand whether a deeper menu with fewer upfront items faired better in comparison to a shallower menu with more items upfront. There was some apprehension amongst stakeholders that a longer list of items would result in items further down becoming lost or ignored. Using eye tracking studies helped establish that a longer menu did not inherently mean items further down were ignored; in fact overall task accomplishment was most effective with a shallower structure, which necessitated a longer upfront list. The results from the eye tracking sessions enabled us to make an evidence-based decision to use longer lists with confidence this reflected the most suitable approach for customers.
A secondary discussion centred around the placement of My Account, Settings and Help menus; we had pressure from stakeholders to put these in a place believed to offer more prominence (closer to the top of the list, just above the main area of focus). Eye tracking helped uncover participants naturally looked towards the end of the list for account related activities as well as Help, seeing this as the natural location for these tasks (further research to expand on this suggested the reason for this was the peripheral context of these activities that was not central to the main system being designed). When looking at eye tracking data for the option which had Account and Help menus placed nearer the top of the list, it was clear to see participants were initially looking towards the bottom of the list to find these items. This data uncovered the most intuitive location for the Account and Help menus, helping to reassure stakeholders they were not 'losing prominence' by appearing further down the menu, and enabling us to design a suitably ordered menu which was aligned with customer behaviour and reflected their expectations.
Card sorts + Tree testing
I assisted with a project to improve the Help & Support taxonomy on Sky.com, with a view to surfacing content categories that were more relevant and easily identifiable to customers.
This work coincided with an overhaul of help content articles, which over the years had become significant in number, often without clear indication about which was most up to date.
A combination of card sorting exercises, click tests and tree testing was carried out to establish a new information architecture (IA), which enabled customers to find content faster and more intuitively than before. I helped develop a new metadata framework which saw each piece of published content tagged with a range of relevant keywords that would help surface the article in the relevant areas.
Introducing keywords significantly improved the the search functionality, allowing this to become more prominent on the landing page and a useful method for finding content. As each piece of content was tagged with metadata to define which categories it belonged to, it was possible to experiment with different layouts and approaches for content discovery. One approach which tested well was the 'sentence builder' which allowed a customer to interrogate the content by completing a phrase that reflected what they needed help with. Using the categories shaped by the card sorting exercises helped ensure the content remained structured inside a clear taxonomy that customers understood and could easily work with.
Personas
I created a set of personas for the Sky homepage project, based on visitor profiles which were provided by the in-house IQ team. These were particularly useful for stakeholders and the development team during feature discussions as the personas inspired a full range of use cases which might otherwise not have been considered or prioritised.
As an example, our research and customer insight drew attention to a significant number of users who were using the homepage as a destination in its own right as a portal to the internet. Typically, these were older customers who had set sky.com as the default webpage on their browser (perhaps unknowingly) during the legacy Sky broadband setup process. These frequent users relied on the Sky homepage for access to information about weather forecasts, news and their Sky email, so it was important these features (particularly Sky email - who we had wrongly thought most customers would not often use) remained within easy reach to address this set of users' needs.
Using personas helped create a tangible representation of Sky's broad range of customers, and ensure their core needs were addressed in the redesign.
Usability testing
The research I'm most accustomed to is lab testing. For the Sky homepage, we recruited participants who were customers as well as non-customers to gain feedback from both groups. Each group was asked to complete a series of tasks (e.g. buy Sky or find out what's on TV this evening) for us to observe how easily these were achieved and what issues were encountered. During a similar session for the Help & Support redesign, a pattern emerged which suggested the content should be adjusted to use more solution-focused language, as this was in line with customers' mental model when seeking assistance.
Analytics
Throughout projects I'm keen to use analytics to uncover areas where effort could be most beneficial, as well as monitor ongoing performance of releases. I've found it useful to look at existing data when starting a project; for example I was able to show a significant portion of customers seeking to update their email address and phone number were doing so on a page which related to their household account details rather than their Sky ID. Using this data led to forming a hypothesis that customers were subsequently calling Sky to update their email 'again' because the first change had been made in the 'wrong' area so was not having the desired effect. I have also used data to pinpoint pages which have high bounce rates during the sales process, with a view to assessing whether more or less information on the page is beneficial, and what content is most relevant for customers to complete the sales journey.
Competitor analysis
I've carried out competitor analysis on several occasions to provide input and inspiration to carry in to new project work. I've also used competitor analysis in a more formal sense to provide a benchmark against other companies - which can be useful to highlight areas requiring focus to ensure the customer experience on Sky in on par or above that of the competition.
guerrilla testing
The local supermarket and work canteen have provided ideal locations for guerrilla testing. I was able to get useful feedback on early concepts for the billing self-service project and new Sky homepage by observing users' interaction with basic versions of the proposed design. A combination of sketches and simple prototypes were used at varying stages of the project as a quick means to get further insight from real users.