Improving part accuracy with VIN collection
Background
We are unable to correspond 8.5% of our customers’ vehicle information (YMMS) with a windshield part. When this happens, we ask users a series of “problem part” questions to try to determine their specific vehicle. However, depending on the vehicle, these questions can be difficult to know and answer correctly, which can result in us bringing the wrong glass to an appointment.
This problem then results in a loss of jobs, time, revenue, and an overall poor customer experience because we have to reschedule while we wait for the new, correct windshield to come in, often after the customer has already waited longer than a week for their original appointment.
My role:
Lead designer & researcher
Impact:
+36% VIN collection
+3.2% part accuracy
Tools:
Figma, Maze, Feedback Loop
How might we obtain more user VINs and increase part accuracy?
Challenges
To combat this problem, we need to collect VINs from customers. This is a difficult hurdle to overcome as many customers (1) don’t know their VIN, (2) don’t want to provide their VIN due to privacy concerns, or (3) don’t have an easy, accessible way to provide it in the moment.
From left to right:
(1) vehicle selection where users provide their YMMS or VIN (control)
(2) vehicle selection redesign and removal of tabs (test variation 1)
(3) VIN collection modal if there’s no part match (test variation 2)
(4) VIN look-up modal if there’s no part match (test variation 3)
How will users react when asked to provide their VIN first?
Method:
A/B Testing
Sample size:
10,000+
Confidence level:
95%
Problem statement
When users provide their vehicle information in the funnel, they have the option to enter their vehicle’ year, make, model, and style, or to enter their VIN. However, only 3% of users provide their VIN over their YMMS.
This can be a problem for certain vehicles because the YMMS doesn't always result in a direct match with a windshield part. There are so many different trims, features, and variations available and obtaining a VIN is the only way for us to ensure part accuracy.
There’s an opportunity to highlight VIN entry as an option and encourage users to provide that information instead of their year, make, model, and style – while still leaving either as an option to move forward.
On the left, the Control variation / On the right, the Test variation
Findings
VIN entry increased by 12.5% (significant)
Page progression from vehicle selection decreased by 4%
Appointment rate decreased by 1.3% (significant)
Recommendations
Explore why users convert less when VIN entry is promoted
Understand what users know about their VIN and any pain points in retrieving this information
Why do users avoid providing their VIN for vehicle selection?
Method:
Unmoderated usability
Sample size:
25
Problem statement
From our A/B testing, we knew that users converted less when we asked for their VIN upfront. However, we didn’t understand why users were so opposed to providing the information. Was it due to unfamiliarity, inconvenience, privacy, confusion, or something else?
A couple screens from the interactive prototype that users could interact with when completing assigned tasks
Survey results collected after task completion from our unmoderated usability sessions
Findings
User frustration surrounding VIN collection could be sorted into three separate categories:
Uncertainty: “Why are you asking me this?”
Effort: “Having to go outside and take a pic of the VIN would be annoying”
Privacy: “This is kind of intrusive, VIN is personal”
Recommendations
Understand what information might motivate the user most so that providing their VIN is worth the effort and/or inconvenience
Personalize the content so it doesn’t feel like a generic attempt at obtaining unnecessary information and instead a crucial part of service
What language resonates with users and most clearly communicates why we need their VIN?
Method:
Survey
Sample size:
300
Margin of error:
6%
Confidence level:
95%
Problem statement
Many users don’t realize the difference in accuracy a VIN provides over a vehicle’s YMMS. There’s an opportunity to clearly communicate this importance to users, in the hopes of encouraging more users to provide their VIN, while also not decreasing the appointment rate due to the added inconvenience.
Copy test variations and collected survey results
Mock-ups to understand which visuals most clearly communicated VIN placement
Findings
Users are frustrated when they don’t understand WHY we’re asking for their VIN
Not explaining the “why” makes users wonder if we want it only to sell to other companies
A straightforward explanation resonated with users more so than any type of monetary compensation, statistic, or veiled threat
“Four windshields match your vehicle” wasn’t direct enough – users assumed that if four matched, any of them would work
Users felt the illustrated VIN dropdowns and the bullet point VIN description were the easiest to understand / read
Recommendations
When there are multiple windshields that match the vehicle, clarify that only one of them is correct so users understand why providing their VIN is so important
Dynamically display the actual number of windshields that match the customer’s vehicle YMM to personalize the experience
How can we lower the barrier to VIN entry with other information?
Method:
Survey
Sample size:
300
Margin of error:
6%
Confidence level:
95%
Problem statement
VIN is a crucial element to part accuracy, however, there are multiple ways a VIN can be found in addition to just finding the number on your vehicle dashboard. Using APIs, we would be able to search through publically available BMV records to locate a customer’s VIN if we had their license plate number or their home address.
For this to be successful though, we needed to understand how comfortable users are providing this information.
Copy test variations and collected survey results
Findings
In terms of customer comfort level, we found that when providing:
Home address:
53% felt comfortable
27% felt uncomfortable
100% have their address memorized
License plate:
51% felt comfortable
28% felt uncomfortable
41% have their plate memorized
VIN:
40% felt comfortable
37% felt uncomfortable
0% have their VIN memorized
92% of users said finding their license plate number would be easier to find than VIN
The majority of users who were uncomfortable or unlikely to provide their home address said so because of privacy
Recommendations
Provide users an option to look up their VIN in the way that best fits their personal needs and comfort
Guide users so they understand the benefits of each look-up option
Impact
After finalizing the designs based on our research findings, we tested each update while measuring (1) conversion rate (users who finished scheduling), (2) VIN collection (users who provided their VIN), and (3) part accuracy (users who had the correct windshield ordered and installed at their appointment).
VIN Collection
Profit: $108K
VIN collection: +34%
Part accuracy: +3%
Retention: +2%
VIN Look-Up
Profit: Flat
VIN collection: +1.9%
Part accuracy: +.2%
Retention: +.4%
On the left, VIN collection (test variation 1)
On the right, VIN look-up (test variation 2)
Team
Lead Designer & Researcher: Me 👩💻
Product Manager: Chris Roblee & Lindsay Stout
UX Writer: Natalie Nichols & Laura Lampe
Data Analyst: Jake Riddle
Director of UX: Jordan Monson
Director of Product: Tim Simeone