EM&V Forum
The Council’s 2023 Evaluation, Measurement & Verification (EM&V) Forum:
Expanding Opportunities in a Changing Landscape
Wednesday, February 15, 2023
ABOUT
The EM&V Forum, Expanding Opportunities in a Changing Landscape, on February 15, 2023, was a great success, with more than 150 participants, 2/3rds of whom attended in person. This year’s EM&V Forum focused on new opportunities for our members in the face of many industry and policy changes. The Forum was rich and diverse in topics, speakers, attendees, and opinions.
Prominent speakers from the CPUC, Program Administrators, Implementers, and EM&V practitioners kicked the day off by painting the big picture. Speakers and participants addressed CPUC-led market transformation initiatives (e.g., TECH BUILD and the newly launched California Market Transformation Administrator (CalMTA), analyzing and engaging in other markets, suggesting a broader view of evaluation, innovations in EE programs and policy and consequences of the changing landscape. These sessions stressed the need for more deliberate coordination for data collection and access as the number of implementers and programs increases.
Speakers and participants covered the importance of appropriate metrics for assessing both programs and markets through many of the day’s sessions. Presenters focused on experiences with newly emergent or newly applied metrics, the importance of capturing value with these metrics, and the challenges of assembling sufficient data to measure that value. Collectively, the experiences underscored the call for more concerted coordination and accessibility of in-period data collection.
Another recurrent thread raised by participants and panelists was the need for data to enable evaluations and support metrics better. Practitioners provided stories of successes and challenges with new measurement technologies and analytical methodologies across the sessions, addressing measurement, data access, analytics, and NMEC*. The Energy Commission updated participants on their ongoing development of a database warehouse that accepts, processes, and shares large-scale program data, including billing and interval-level meter data, rate information, and program participation-related data. That data stored in the “warehouse” could be used to: understand and improve program performance, verify contract performance, evaluate cost-effectiveness, support electricity system planning, validate demand flexibility value, and support energy policies and programs.
The day closed with participants and panelists suggesting topics and issues for future venues, be it Council Member meetings or major events, such as the Spring Forum and Fall Conference). Those suggestions included: PA updates on their priorities, planned studies, and program evaluation challenges; expanding SEM to other markets; best uses for embedded EM&V; best EM&V practices from other industries; and engaging community-led organizations and groups to share their experiences and work in EE policy, design, implementation, and EM&V.
The Council greatly appreciates those who volunteered their time and expertise and truly made our 2023 EM&V Forum a success! We look forward to continuing these collaborative efforts and further engaging these topics with our members.
* Normalized Metered Energy Consumption – Overall, NMEC is a set of tools and standards that, when applied to interval data, provides quantifiable and statistically significant reporting of normalized energy usage and energy savings due to an intervention, such as an energy efficiency project.
AGENDA
Market Transformation – The Big Picture
Which markets are ripe for transformation and can work towards achieving the State’s clean energy, climate, equity, and resiliency goals? What are the differences and commonalities between strategic planning to meet goals, and programs to transform specific markets and end-uses? How can the industry support and evaluate decarbonization, electrification, and other energy and climate efforts? How can market transformation strategies help overcome differences in segment visions, barriers, and necessary technologies? Hear directly from the new California Market Transformation Administrator and other experts in this opening session.
- Moderator: Jay Luboff, Principal, Jay Luboff Consulting, LLC
- Margie Gardner, Vice President of Market Transformation, Resource Innovations
- Abhi Wadhwa, Senior Analyst – Building Decarbonization/Climate Adaptation, CPUC
Innovation in Energy Efficiency Programs and Policy
What Does Innovation Look Like and How Do We Measure it?
- How are Program Administrators, developers, and implementers navigating innovation? How can the industry drive policies that reward innovation rather than punish risk?
- What implications, challenges, and opportunities have arisen in the changing policy and program landscape? How to identify challenges and leverage opportunities when they are rapidly changing. What is the industry doing differently to adapt?
- Practitioners’ perspectives of the relevance of different methods for evaluating programs. What does the industry need to change, do we need a new evaluation guidebook?
- Moderator: Jarred Metoyer, Senior Vice President/Managing Director, DNV
- Dr. Ellen Steiner, Vice President, Opinion Dynamics
- Christopher Healey, Director of Utility Regulatory Affairs, Enervee
- Dr. Liza Legaspi, Behavior Programs and Financing Products Segment Supervisor, SoCalGas
- Dulane Moran, Principal/Research and Evaluation, Cadeo Group
- Tami Rasmussen, Senior Vice President, Evergreen Economics
- Jeorge Tagnipes, Supervisor, CPUC (via Zoom)
Market Access Programs (MAP)
Review EM&V plans, challenges, methods, and opportunities with the Market Access Program offered across the utilities and other load serving entities.
- Moderator: Coby Rudolph, Senior Regulatory Analyst, CPUC
- Alison Erlenbach, Lead, Meter-based Energy Efficiency, PG&E
- Jennifer Kreutzer, Senior Customer Programs Manager, MCE
- Brian Maloney, Senior Portfolio Advisor, Southern California Edison
- David Sagara, Managing Director, Mendota Group representing SDG&E’s Summer Reliability Market Access Program
Measurement, Data Access and Analytics – A Look Ahead
This session will address emerging issues related to baselines, data access, submetering, and measurement among newer and emerging technology and programs. What works? What doesn’t? And what new data services and analytic tools are expected to be available in the next few years to ensure M&V remains up-to-date alongside modernizing programs.
- Moderator: Joe Desmond, Executive Director, CEDMC
- Kari Binley, Senior Manager, Energy Partnerships, ecobee
- Jason Harville, Director for Energy Data and Analytics, CEC
Normalized Metered Energy Consumption
Through a series of technical presentations, experts will discuss NMEC methods and applications of this methodology as it applies to various programs. Recommendations on how to account for differences between electricity and natural gas applications will be discussed in detail.
- Moderator: Carmen Best, Vice President, Policy and Emerging Markets, Recurve
- Justin Kjeldsen, Regional Director, Franklin Energy
- Kevin Ehsani, P.E., EM&V Supervisor, SoCalGas
- David Jump, Director, kW Engineering
Metrics for Evaluating Programs and New Initiatives
What are the metrics to track new initiatives and technology performance? What do those metrics mean for bill and rate impacts? How can Distributional Equity Analysis (DEA) help track the state’s progress for equity issues in disadvantaged communities?
- Moderator: Clark McIsaac, Director of Policy and Strategy, CEDMC
- Anna Frankowski Nelson, Project Manager II, Energy Solutions
- Dr. Ellen Steiner, Vice President, Opinion Dynamics
- Steve Schiller, Principal, Schiller Consulting and Founder, CEDMC
Open Mic
Now it’s your turn! Is your pet idea not on today’s agenda? Bring it and make a 2-minute pitch that we can consider for a “deep dive” in an upcoming Council event or meeting.
- Moderator: Joe Desmond, Executive Director, CEDMC
2023 Gold Sponsors
2023 Silver Sponsors
2023 Bronze Sponsors
COVID-19 Precautions:
The EM&V Forum will adhere to all COVID-19 restrictions that are in place on February 15, 2023, as specified here. If there is a mandated shutdown, we will switch to a fully virtual event. We appreciate your understanding.
