Improving part accuracy with VIN collection

Background

We are unable to correspond 8.5% of our customers’ vehicle information (YMMS) with a windshield part. When this happens, we ask users a series of “problem part” questions to try to determine their specific vehicle. However, depending on the vehicle, these questions can be difficult to know and answer correctly, which can result in us bringing the wrong glass to an appointment.

This problem then results in a loss of jobs, time, revenue, and an overall poor customer experience because we have to reschedule while we wait for the new, correct windshield to come in, often after the customer has already waited longer than a week for their original appointment.

My role:
Lead designer & researcher

Impact:
+36% VIN collection
+3.2% part accuracy

Tools:
Figma, Maze, Feedback Loop

How might we obtain more user VINs and increase part accuracy?

Challenges

To combat this problem, we need to collect VINs from customers. This is a difficult hurdle to overcome as many customers (1) don’t know their VIN, (2) don’t want to provide their VIN due to privacy concerns, or (3) don’t have an easy, accessible way to provide it in the moment.

From left to right:
(1) vehicle selection where users provide their YMMS or VIN (control)
(2) vehicle selection redesign and removal of tabs (test variation 1)
(3) VIN collection modal if there’s no part match (test variation 2)
(4) VIN look-up modal if there’s no part match (test variation 3)

How will users react when asked to provide their VIN first?

Method:
A/B Testing

Sample size:
10,000+

Confidence level:
95%

Problem statement

When users provide their vehicle information in the funnel, they have the option to enter their vehicle’ year, make, model, and style, or to enter their VIN. However, only 3% of users provide their VIN over their YMMS.

This can be a problem for certain vehicles because the YMMS doesn't always result in a direct match with a windshield part. There are so many different trims, features, and variations available and obtaining a VIN is the only way for us to ensure part accuracy.

There’s an opportunity to highlight VIN entry as an option and encourage users to provide that information instead of their year, make, model, and style – while still leaving either as an option to move forward.

On the left, the Control variation / On the right, the Test variation

Findings

  • VIN entry increased by 12.5% (significant)

  • Page progression from vehicle selection decreased by 4%

  • Appointment rate decreased by 1.3% (significant)

Recommendations

  • Explore why users convert less when VIN entry is promoted

  • Understand what users know about their VIN and any pain points in retrieving this information

Why do users avoid providing their VIN for vehicle selection?

Method:
Unmoderated usability

Sample size:
25

Problem statement

From our A/B testing, we knew that users converted less when we asked for their VIN upfront. However, we didn’t understand why users were so opposed to providing the information. Was it due to unfamiliarity, inconvenience, privacy, confusion, or something else?

A couple screens from the interactive prototype that users could interact with when completing assigned tasks

Survey results collected after task completion from our unmoderated usability sessions

Findings

  • User frustration surrounding VIN collection could be sorted into three separate categories:

    • Uncertainty: “Why are you asking me this?”

    • Effort: “Having to go outside and take a pic of the VIN would be annoying”

    • Privacy: “This is kind of intrusive, VIN is personal”

Recommendations

  • Understand what information might motivate the user most so that providing their VIN is worth the effort and/or inconvenience

  • Personalize the content so it doesn’t feel like a generic attempt at obtaining unnecessary information and instead a crucial part of service

What language resonates with users and most clearly communicates why we need their VIN?

Method:
Survey

Sample size:
300

Margin of error:
6%

Confidence level:
95%

Problem statement

Many users don’t realize the difference in accuracy a VIN provides over a vehicle’s YMMS. There’s an opportunity to clearly communicate this importance to users, in the hopes of encouraging more users to provide their VIN, while also not decreasing the appointment rate due to the added inconvenience.

Copy test variations and collected survey results

Mock-ups to understand which visuals most clearly communicated VIN placement

Findings

  • Users are frustrated when they don’t understand WHY we’re asking for their VIN

    • Not explaining the “why” makes users wonder if we want it only to sell to other companies

  • A straightforward explanation resonated with users more so than any type of monetary compensation, statistic, or veiled threat

  • “Four windshields match your vehicle” wasn’t direct enough – users assumed that if four matched, any of them would work

  • Users felt the illustrated VIN dropdowns and the bullet point VIN description were the easiest to understand / read

Recommendations

  • When there are multiple windshields that match the vehicle, clarify that only one of them is correct so users understand why providing their VIN is so important

  • Dynamically display the actual number of windshields that match the customer’s vehicle YMM to personalize the experience

How can we lower the barrier to VIN entry with other information?

Method:
Survey

Sample size:
300

Margin of error:
6%

Confidence level:
95%

Problem statement

VIN is a crucial element to part accuracy, however, there are multiple ways a VIN can be found in addition to just finding the number on your vehicle dashboard. Using APIs, we would be able to search through publically available BMV records to locate a customer’s VIN if we had their license plate number or their home address.

For this to be successful though, we needed to understand how comfortable users are providing this information.

Copy test variations and collected survey results

Findings

In terms of customer comfort level, we found that when providing:

Home address:

  • 53% felt comfortable

  • 27% felt uncomfortable

  • 100% have their address memorized

License plate:

  • 51% felt comfortable

  • 28% felt uncomfortable

  • 41% have their plate memorized

VIN:

  • 40% felt comfortable

  • 37% felt uncomfortable

  • 0% have their VIN memorized

  • 92% of users said finding their license plate number would be easier to find than VIN

  • The majority of users who were uncomfortable or unlikely to provide their home address said so because of privacy



Recommendations

  • Provide users an option to look up their VIN in the way that best fits their personal needs and comfort

  • Guide users so they understand the benefits of each look-up option

Impact

After finalizing the designs based on our research findings, we tested each update while measuring (1) conversion rate (users who finished scheduling), (2) VIN collection (users who provided their VIN), and (3) part accuracy (users who had the correct windshield ordered and installed at their appointment).

VIN Collection

  • Profit: $108K

  • VIN collection: +34%

  • Part accuracy: +3%

  • Retention: +2%

VIN Look-Up

  • Profit: Flat

  • VIN collection: +1.9%

  • Part accuracy: +.2%

  • Retention: +.4%

On the left, VIN collection (test variation 1)
On the right, VIN look-up (test variation 2)

 

Team

Lead Designer & Researcher: Me 👩‍💻

Product Manager: Chris Roblee & Lindsay Stout

UX Writer: Natalie Nichols & Laura Lampe

Data Analyst: Jake Riddle

Director of UX: Jordan Monson

Director of Product: Tim Simeone

Previous
Previous

Increasing attachment rate by 18% with service packages

Next
Next

Improving our Google rating from 3.8 to 4.0 with +1.5K reviews/week