E Source Review of North American Electric and Gas Company IVRs: History and Methodology
Since 2004, E Source has been benchmarking interactive voice response (IVR) systems to gauge how successful they are in serving residential customers. Our study has become the industry standard for assessing IVR improvements as they relate to customers’ contact preferences.
- In 2007 (PDF), we evaluated 11 tasks and functions, including customer service self-service options that required account access. About 27 percent of U.S. and Canadian utilities provided the E Source team with secure access to their IVRs during the research period.
- In 2009 (PDF), we evaluated 13 tasks and functions, including customer service self-service options that required account access. About 28 percent of U.S. and Canadian utilities provided the E Source team with secure access to their IVRs during the research period.
This year, our methodology is similar to that used in the 2009 study. We will assess 13 tasks and functions that were identified by an E Source market research study as the options residential customers most strongly think that utilities should offer on the IVR. New this year, we will assess two of the features using mobile phones: report an outage and make a payment. We will collect data from 95 IVRs to bring you the 2011 rankings.
The E Source review is based on actual residential customer IVR use and feedback. It includes the largest gas and electric companies in the U.S. and Canada. To ensure that we get a good overall sense of how customer-friendly your IVR is, multiple residential reviewers will call into each IVR system and every task and function will be double-checked. The benchmark gives equal weighting to functionality and usability, and the usability of a given task or function is compared against best practices gleaned from IVRs for all types of businesses, not just from utility IVRs. Shown below is a graph from the 2009 Review of North American Electric and Gas Company IVRs that highlights the usability and functionality ratings for all of the companies included in the review.
Our reviewers are asked to assess the overall experience they have while using the IVR, including the use of secure-access self-service options. (E Source works directly with each utility to obtain secure access to the utility IVR to enable this in-depth evaluation. Accounts are only used with the permission of the account holder; no transactions are completed, and all information provided is managed with strict confidentiality.) Researchers are not given time limits for their reviews. By allowing each reviewer to work through the system at his or her own pace, we feel we capture a more accurate IVR experience from each of them.
The E Source benchmark study provides a great deal more information than an audit of features and functions of the utility IVR. However, the results aren’t as comprehensive as the feedback from usability testing, in which companies observe multiple customers as they try to complete the same task through an IVR.
Q: How did E Source select the tasks and functions that were included in the review?
A: The choice of features and functions was based on market research we conducted, in which we asked customers what they want to find and use on their utility IVR. We also considered advice from thought leaders in the utility industry and within E Source.
Q: When will the IVR benchmark rankings be available?
A: The rankings will be announced August 2011 and the final report will be available in late September.
If you would like information about participating in the next IVR assessment, please submit your name and e-mail address.