E Source Review of North American Electric and Gas Company IVRs: History and Methodology
Since 2004, E Source has been benchmarking interactive voice response (IVR) systems to gauge how successful they are in serving residential customers. Our study has become the industry standard for assessing IVR improvements as they relate to customers’ contact preferences.
- In 2007 (PDF), we evaluated 11 tasks and functions, including customer service self-service options that required account access. About 27 percent of US and Canadian utilities provided the E Source team with secure access to their IVRs during the research period.
- In 2009 (PDF), we evaluated 13 tasks and functions, including customer service self-service options that required account access. About 28 percent of US and Canadian utilities provided the E Source team with secure access to their IVRs during the research period.
- In 2011, we evaluated 13 tasks and functions, including customer service self-service options that required account access. Approximately 45 percent of US and Canadian utilities provided the E Source team with secure access to their IVRs during the research period.
The E Source review is based on actual residential customer IVR use and feedback. It includes the largest gas and electric companies in the US and Canada, plus a representative set of small and midsize utilities. To ensure that we get a good overall sense of how customer-friendly your IVR is, multiple residential reviewers will call into each IVR system and, we’ll perform quality-assurance checks. The benchmark gives equal weighting to functionality and usability, and the usability of a given task or function is compared against best-in-class IVRs for all types of businesses, not just from utility IVRs. Shown below is a graph from the 2011 Review of North American Electric and Gas Company IVRs that highlights the usability and functionality ratings for all of the companies included in the review.
Our reviewers are asked to assess the overall experience they have while using the IVR, including the use of secure-access self-service options. (E Source works directly with each utility to obtain secure access to the utility IVR to enable this in-depth evaluation. Accounts are only used with the permission of the account holder; no transactions are completed, and all information provided is managed with strict confidentiality.) Researchers are not given time limits for their reviews. By allowing each reviewer to work through the system at his or her own pace, we feel we capture a more accurate IVR experience from each of them.
The E Source benchmark study provides a great deal more information than an audit of features and functions of the utility IVR. However, the results aren’t as comprehensive as the feedback from usability testing, in which companies observe multiple customers as they try to complete the same task through an IVR.
Q: How did E Source select the tasks and functions that were included in the review?
A: The choice of features and functions was based on market research we updated in 2013, in which we asked customers what they want to find and use on their utility IVR. We also considered advice from thought leaders in the utility industry and within E Source.
Q: When will the IVR benchmark rankings be available?
A: The rankings will be announced and the final report will be available by the end of 2013.
If you would like information about participating in the next IVR assessment, please submit your name and e-mail address.