Improve Metrics Opt-IN

A desktop computer uses label design software to create and print a warning label.

Design Ops

This project is part of a larger desktop software design epic.

(2 weeks)

Situation

When installing software, users must opt-in to send anonymous usage statistics. These statistics and error reports help developers identify bugs and prioritize improvements.

  • This particular software was typically installed on standalone computers without internet access.
  • Only 3% of known installations opted-in to send performance metrics.
  • Legally, users can not send metrics by default (opt-out later).
  • Previous user research did not exist.
  • End users were unknown because the software was primarily sold through 3rd party vendors.
EULA of a software installer, with opt-in unchecked and Install button active.
Original installation wizard with a checkbox to opt-in to usage and error reporting.

Task

Redesign the form to improve the opt-in rate.

Action

First, I conducted a survey using usertesting.com to understand how users feel about these statistics and error reports. I found that half agreed to send the reports even though some were not entirely clear what they were. Some declined because they did not want the software using their network. A majority declined because had conspiracy theories about the reports.

Pie chart of opt-in user types, described below.
Graph showing whether users typically opted-in or opted-out of usage and error reporting.

Next, I conducted a heuristic analysis of the existing form, and found that users could simply install the software without even reading the checkbox for anonymous statistics and error reports.

Finally, I redesigned the form to such that the Install button was disabled until the user read the reporting option and selected a radio button to opt-in or opt-out. I tested the new design on usertesting.com with a positive success rate.

Result

Once deployed, this new wizard increased metrics opt-ins to about 20% of known installations, nearly a 700% increase.

EULA of a software installer, showing two deselected radio buttons for the usage statistics reporting, and a disabled Install button.
Revised installation wizard with a disabled Install button until the user chooses one of the radio buttons.