Case Study
Discovery
The Challenge
During system errors, account balances and tools on the Member Home screen could appear blank or unavailable. Members were shown a vague page-level error message, which led to frustration, concern, and calls to the contact center. The initial assumption was that members believed their account balances were incorrect. The request was to mock and test new error message copy to address this concern. Before designing solutions, we needed to validate whether this assumption reflected the actual member problem.
Client
USAA
Industry
Enterprise Financial Services
Year
2023
Role
UX Strategy, Research, Behavioral Analysis
Tools
Mural, Figma, UserZoom, Box, Wiki, Zoom, Slack
Discovery
What We Learned
We analyzed existing mobile MSAT feedback (January-July) and member verbatims prior to testing.
This analysis revealed:
The majority of members were annoyed or blocked, not confused
Many members were already familiar with refreshing the app
Only a small subset were worried their account balances were impacted
This reframed the problem from balance confusion to lack of clarity and guidance during system disruption.
The Outcome
Problem Statement
~1.2M members per month receive a vague page-level system error on Member Home, causing frustration and concern when account information, tools, and functionality are unavailable or incorrect, resulting in member complaints.
Change Statement
We'd like to change members' sentiment and actions when encountering unavailable information or tools in order to increase confidence in USAA and fulfill our mission.
These statements became the criteria for evaluating all message options.
From Problem Definition to Message Selection
Together, these statements established clear criteria for success. I guided the Product partner through human-centered design exercises to ground our assumptions in member feedback that I analyzed and synthesized, and we co-created the problem and change statements to align on the real issue. Using this shared criteria, we rewrote and tested three message options. Through unmoderated research we learned that Message B, which paired clear reassurance with an explicit Refresh CTA, most consistently prompted the desired behavior, refreshing the app, and Message B was selected for implementation.
