The challenge
The TECH Initiative, a $265 million effort to achieve carbon neutrality in California homes by 2045 through the market adoption of low-emissions space and water-heating technologies for existing single-family and multifamily residential homes. Utilizing market incentives, supply chain engagement, workforce development, consumer education, regional pilots, and Quick Start Grants, the initiative installs low-emission space- and water-heating technologies in California homes.
As the evaluator for the TECH Initiative, E Source integrates evaluation within the program design, implementation, and reporting processes to support evidence-based decision-making and continuous program improvement.
The solution
E Source conducted timely surveys and interviews with manufacturers, distributors, customers, and contractors to enable near-real-time process evaluation, device-level performance analysis, and consumption analysis. This approach allowed us to assess impacts alongside market and program studies to establish baselines for long-term impact assessments. Our data collection activities encompassed:
- Mystery shopping
- Web usability testing
- Customer surveys
- Market actor interviews
- Post-training assessments
- Measurement and verification
The results
Creating a program theory logic model early helped us understand how TECH works, making it easier to choose important metrics. It also showed us where to collect data to support these metrics, allowing us to give real-time feedback. This helped implementers make quick changes to improve the program.
We recommended metering infrastructure to monitor device performance, which is important for deploying heat pumps and managing flexible loads. Many contractors and consumers are skeptical about heat pumps, especially in cold climates. Demonstrating energy and greenhouse gas savings, along with customer satisfaction, quickly is more impactful than providing this information after the fact.
We were careful to construct a baseline measurement of market indicators from which we can judge program impacts, including equity indicators, marketing metrics, market indicators, and workforce indicators. Together, these activities will paint a holistic picture of the program’s market impacts
Our evaluation of the TECH Initiative will help inform the scalability of program activities. Considering the program’s cost-effectiveness alongside its equity goals will identify which strategies are important to continue and which should not be absent from the program. Our work also guides updates to cost-effectiveness frameworks for building electrification, requiring a rethink of traditional costs and benefits to account for the added electric load and reduced greenhouse gas emissions. Findings such as these will inform the state’s policy design and future program planning.