Read this newsletter online Visit our website  

SQ consult. Sustainable quality
  Towards effective NAMAs: tapping knowledge from policy evaluation  

  In spite of large efforts spent on the development of Nationally Appropriate Mitigation Actions (NAMAs) only limited progress has been made in terms of actual implementation. Uncertainty regarding NAMA effectiveness in reducing GHG emissions tempers the willingness of potential funders to make investment decisions. We argue that using insights and methodologies available from the policy evaluation community could provide a fresh impulse in supporting the development and implementation of effective NAMAs.  

  The origin and future of NAMAs  

Nationally Appropriate Mitigation Actions (NAMAs) are sets of development policies and actions aimed at reducing greenhouse gas emissions, which were first mentioned in the Bali Action Plan in 2007. NAMAs are voluntary actions defined by developing countries and they have become one of the key topics on the agenda of international negotiations as the means for boosting social, economic and environmental transformations necessary to promote low carbon development in developing countries.

The ‘intended nationally determined contributions (INDCs)’ for the agreement in Paris in 2015 will be a next important step in this process. All Parties are expected to define their contributions - including developing countries. This will likely also provide a further boost to international support for NAMA implementation. The fact that the Green Climate Fund is now operational and initial capitalization is almost reaching the goal of 10 billion US$ also points towards enhanced funding, which makes it even more important to address some of the challenges NAMA implementation faces today.


  Going beyond project based activities  

While the NAMA concept has been under discussion for some years, concrete implementation still need to take off. The NAMA Status Report in 2013 identified the lack of clarity of the concept as one of the main reasons for the relatively slow implementation of NAMAs. Given that there is no internationally agreed definition, each country is free to define NAMAs as they see appropriate. The full list of NAMAs submitted to the UNFCCC  is a good indication for the broad interpretation of the term. Three types of NAMAs are emerging, differing in level of detail and concreteness, as shown below.

It is, however, important to realise that NAMAs are in itself not a new concept. Fundamentally, it is about good development policies and actions that now have the additional - and sometimes leading - objective to reduce GHG emissions. So what is new then? NAMAs can provide new sources of funding through for example the Green Climate Fund as well as many bilateral and other multilateral sources. For the international climate community it is also relatively new to think about large-scale, structured mitigation in developing countries beyond project-based activities. This leads to many approaches that are still fundamentally rooted in project-style thinking, rather than aiming at a smart design and implementation of policies. This is driven not only by existing experience - or lack thereof, but also by the complexity of determining the impacts of policies.


  Understanding the effectiveness of NAMAs  

The international climate community has years of experience with assessing the effectiveness of project-based activities under the CDM. Assessing the effectiveness for NAMAs, however, becomes more complex as the scope of mitigation actions under NAMAs is broadened, both in geographic coverage and the types of activities. Especially for policy NAMAs the CDM experience is of limited use when looking for methodologies to assess their effectiveness.

While being faces with similar questions, policy-based NAMAs require a different approach to solve issues, such as:

  • How to determine the contribution of a policy NAMA towards achieving emission reductions?
  • What is the causal relationship between the policy NAMA and the achieved reductions?
  • How to set a standard for determining policy NAMA effectiveness in “diffused” sectors like households and transport or for cross-cutting policies, such as carbon taxes?
  • What needs to be monitored in order to be able to determine if a policy NAMA is effective or not?

Discussions around baseline setting and additionally within the CDM demonstrate the challenges already faced at a project level. At a policy level these questions are even becoming more complex to answer. Determining cause-and-effect relationships and assessing policy interaction are typical problems faced in evaluating effectiveness. Data availability and quality beyond project activities become crucial challenges.


  Involving the policy evaluation community  

Extensive experience with these issues exists in the wider policy evaluation community. Substantial effort has been put into the development of both ex-post and ex-ante policy evaluation methodologies, e.g. in relation to energy efficiency, renewable energy and GHG emission policies and programs. This experience can be valuable input into the efforts to move towards the implementation of effective NAMAs. In doing so, potential funders of NAMAs can be provided with crucial information to enable, ex-ante, the prioritisation of NAMAs looking for international support in line with their own selection criteria and, ex-post, assess the cost-effectiveness of their investment. At the same time, robust policy evaluation can support implementing countries to enhance the overall effectiveness of their policy NAMAs, thus maximizing GHG reduction and sustainable development benefits.


  A smart policy design process  

A policy design process in the ideal case starts with clear objectives and the identification of barriers that hamper meeting these objectives. Barrier refers to any factor that inhibits the effectiveness of the policy. This is taken into account in designing and formulating the policy in such a way that barriers are addressed to a maximum extent. Policy implementation is followed by a continuous cycle involving monitoring and evaluations and possible reformulation of the policy to increase effectiveness of the policies. A successful policy design process includes an explicit understanding and logic of the causal relationship between the policy (NAMA) and policy objectives, like emission reductions to be achieved. In other words: How is the policy instrument (or a package of instruments) expected to achieve its impacts and who is expected to take action at which point in time. This relation is “translated” into a monitoring plan holding information for the key indicators that needs to be collected to assess its effectiveness. Experiences from a large number of ex-post evaluation studies have resulted in a listing of ingredients for effective policies: typical circumstances in which to apply different types of policies to improve energy efficiency and GHG emission and characteristics that determine the success of the policy.


  Key challenges in policy impact evaluation  

Various challenges exist when assessing the impact of policies. Experiences from policy evaluation can help to address these challenges. Some of these include:

  • Setting a baseline: The baseline represents - in the case of policy NAMAs - the level of GHG emissions that most likely would have been emitted in the absence of the policy being assessed. Like in CDM, the baseline is not a historical reference point but an assumption about what would have happened in the implementation period if the assessed policy were not implemented. Attributing changes in emissions to specific policies and actions is not straightforward. GHG emissions can change due to a variety of factors, including: the policy or action being assessed; other policies or actions that affect the same emissions sources; and various external drivers that affect emissions, such as changes in economic activity, population, energy prices, weather, autonomous technological improvements, and structural shifts in the economy. In policy evaluations it is crucial to be very transparent on what is included in the baseline and what not, which assumptions are applied to determine the level of GHG emission for the baseline and what the uncertainties are in the calculations. Experiences from the policy evaluation community allow identifying key non-policy drivers for baseline development and robust methods for making projections and analysing uncertainty related to such projections.
  • Establishing cause-and-effect relationships: For investment projects - like most of the CDM - it is relatively clear how the cause leads to the GHG effect. For the installation of, for example, a facility to capture landfill gas will reduce methane emissions from landfills and the amount can be estimated relatively accurately using available data on the landfill and the installation specifications. For policies this is often not quite as straight forward. Policies can impact investment and consumer behaviour in different ways - some more, some less direct. For directly impacting policies, such as regulations, the cause-and-effect relationship is rather clear, while this is less the case for indirect measures, such as taxes, incentives or research and development initiatives. Unlike the CDM, not only the relationship between the policy and the intended GHG effect needs to be considered, but also potential indirect and often unintended effects that can reinforce or counteract the policy’s mitigation goals. A range of methods and tools supports analysts in determining these relationships, like for example logical modelling. For many policies there are established relationships that can be adjusted based on the specific national circumstances.
  • Evaluating multi benefits: NAMAs are not only about reducing greenhouse gas emissions but aim at contributing to broader sustainable development goals. Listing and assessing these multi benefits can increase stakeholder buy-in, enhance motivation of targeted actors, expand financial support for policies, and improve the cost-benefit ratio for policies.

    A good example for the multiple benefits is provided by the recently published IEA overview on how investment in energy efficiency provides benefits to different stakeholders. Improving energy efficiency in households can - next to GHG emission reduction – for example results in increase in disposable income by households. Energy efficiency improvement in industry is often accompanied by important increases in productivity of the firm, reduced air pollutant emissions and waste and improved public and labour health. Policy evaluation in the past did not focus on GHG emissions, but aimed to assess the broad spectrum of potential impacts of policies to provide governments with robust analysis for decision-making. A wide range of methods is available to evaluate different types of effects and to support an integrated assessment of multiple impacts.
  • Determining success: For investment projects, success is normally easy to determine, with the planned installations up and running. For policies, success is much harder to assess, and depending on the type of policy implemented. A framework that includes the monitoring of inputs, activities and intermediate outcomes can facilitate a better understanding of successes and problems and help fine-tune policies to make them more effective. Policy evaluation is working with a multitude of indicators, depending on the type and scope of the policy. This allows targeted data collection to inform decision makers and provide confidence on the effectiveness of policies.

  Concluding remarks  

Policies are an essential element in delivering the needed social, economic and environmental transformations necessary to promote low carbon development in developing countries. To tap the potentials provided by smart policy making, a robust analysis of expected impacts and sound monitoring of achievements is key. Using insights and methodologies available from the community of policy evaluators can help to provide confidence in the effectiveness of policies proposed as NAMAs. Better understanding of expected impacts can enhance successful implementation, facilitate peer-to-peer exchange and unlock international support.
Experiences from many years of policy evaluation at different levels of jurisdiction can support governments in identifying the most promising policies to achieve the given objectives within the national circumstances. This starts with identifying favourable circumstances for different policy instruments in general to the detailed analysis of expected impacts in the given national context. The tools, methods and experience from the policy evaluation community can support governments in selecting, implementing and monitoring policy NAMAs to ensure effectiveness and create multiple benefits for the climate as well as the national communities.


  Ingredients for effective policies  
Type of policy Typical circumstances in which to apply the policy Characteristic that determine the success
Energy perfor-mance standards for buildings, cars or applianc-es
  • When dealing with a tar-get group which is:
    • unwilling to act (e.g., voluntary agreement of pro-ducers not fulfilled) or
    • difficult to address (e.g., land-lord – tenant problem)
  • When aiming at remov-ing the worst products or services from the market with regard to energy consumption
  • Is the standard well-justified? E.g. through life-cycle cost studies.
  • Is the target group well prepared for the standard? E.g. through information campaigns, demonstration projects, feasibility studies, training programs etc.
  • Is the target group sufficiently skilled to apply the standard?
  • Is there resistance among the target group to apply the standard?
  • Are there sufficient resources (knowledge, capacity, time, budget, priority) in place to enforce the legislation?
  • Are there penalties in place for non-compliance?
  • Are the penalties at a sufficiently high level to stimulate meeting the standard?
  • Is the standard timely adjusted to technology progress?
Mandatory targets/tradable savings (demand-side) for energy companies
  • When aiming at energy savings in the house-holds or services sector, i.e. large target groups being difficult to address by energy efficiency ser-vices.
  • When knowledge, finan-cial and institutional bar-riers play a role.
  • As an alternative or complement to an ener-gy saving fund
  • Is the target clearly set beyond business-as-usual?
  • Is measurement and verification of savings possible at low cost, e.g. by standardization of energy saving measures?
  • Is the cost-recovery mechanism (energy companies’ costs passed to end-users) clear and transparent?
  • Are there penalties in case of non-compliance (or are there other incentives in place to prevent non-compliance)?
  • Are penalties set at such a level that target achievement is stimulated?
  • Are financial incentives needed to stimulate households and companies to implement EE measures
  • Is the market for tradable certificates transparent and relia-ble?
  • Is there undesired overlap with other instruments?
Labelling of appliances, cars, buildings
  • When there is a knowledge / infor-mation barrier
  • When dealing with large consumer or service sec-tor groups
  • When dealing with ra-ther uniform technolo-gies
  • When it is not feasible to implement a very effec-tive standard
  • Is it planned to adjust the label to technology progress and market transformation?
  • Is the label well-justified by respective life-cycle cost studies?
  • Is the target group timely and sufficiently informed? E.g. through information campaigns.
  • Is the label clear and transparent?
  • Are there complementary incentives (eco-tax, subsidy, tax exemptions) for stimulating action?
Financial / fiscal instruments such as soft loans, subsidy schemes, investment deduction schemes, rebates
  • When there is a financial barrier in place.
  • When an informative instrument (e.g. energy audit) needs financial incentives to attract the target group
  • Is the target group aware of the existence of the instru-ment?
  • Is the financial support sufficient to attract new investments or to carry out energy audits?
  • Is the annual budget for the instrument linked to the tar-get?
  • Is the procedure for getting financial support sufficiently known by the target group and simple enough?
  • Is it clear for the target group which technologies are eligi-ble for financial support?
  • Is the list of technologies regularly updated to limit free riders?
  • Is the instrument implemented for a long time period to ensure security for investors?
Energy tax / en-ergy tax exemp-tion
  • When dealing with large target groups
  • When aiming to internalize external costs
  • Is the target group well informed on existence and planned future development of the energy tax?
  • Is use of tax income properly justified and marketed to market actors?
  • To what extent does the energy tax take account of global or regional competition aspects (e.g., by tax exemptions for large industries)?
  • To what extent are energy tax exemptions used as an in-centive for implementing EE measures (e.g. in a voluntary agreement scheme)
Information / knowledge transfer / educa-tion / training
  • When there is a knowledge barrier
  • When dealing with large target groups
  • Is the information well linked to the customer type within the target group?
  • Is the information clearly linked to other instruments (regulation, financial/fiscal, voluntary agreement, etc.)?
Voluntary agreements to save energy (industry, services sector) or improve energy efficiency (e.g. cars or appliances)
  • When dealing with a small number of actors with which you need to negotiate or a strongly organized sector
  • When there is much relatively cheap saving potential (low hanging fruit)
  • Is the target group motivated to participate in the voluntary agreement?
  • Is the target set beyond business-as-usual?
  • Are there penalties in case of non-compliance (or are there other incentives in place to prevent non-compliance, e.g. a rebate on energy tax, or is there a regulatory threat in case of non-compliance)?
  • Is there a good monitoring system in place?
  • Are supporting instruments in place (such as audits, energy monitoring systems, demonstration projects, financial in-centives)?
Procurement program
  • When there are sufficient possibilities to bundle large buyers
  • When there is a limited number of market actors supplying EE technologies
  • When potentials for further development and market transformation of new technologies are large enough.
  • Is the program management qualified and engaged? Can the buyers and suppliers group be motivated in principle? Is the buyers group involved in the program set up?
  • Is the buyers group sufficiently sized? Are the results of the program well documented to facilitate market deployment? Is the program well tuned with other policies (energy efficiency standards, labelling, research & development)?
Source: Harmelink et al (2008)

  Contact the authors: SQ Consult

Mirjam Harmelink, SQ Consult
Marion Vieweg, SQ Consult
Dian Phylipsen, SQ Consult

P.O. Box 8239
3503 RE Utrecht
The Netherlands
  To unsubscribe click here