anti-circumvention of standards for bettermarket surveillance

This project has received funding from the European Union’s Horizon 2020 research and innovation program-me under GA number 785122. The sole responsibility for the content of this document lies with the authors. It does not necessarily reflect the opinion of the European Union. Neither the EASME nor the European Com-mission are responsible for any use that may be made of the information contained therein.

ANTI-Circumvention of Standards for better market Surveillance

Online workshop for test laboratories11 May 2021

OEKO

2

OEKOAgenda

ANTICSS project in brief

Circumvention: current legal framework

ANTICSS definitions / understanding of circumvention

ANTICSS testing procedure to target circumvention

Examples

Questions & Answers

3

OEKOAgenda





Examples

Questions & Answers

4

OEKOANTICSS Project in brief

• Study within the Horizon 2020 programme, GA 785122https://www.anti-circumvention.eu

• Duration: 04/2018 – 09/2021

• Project lead: Oeko-Institut e.V. (DE)

• Project partners: AT: AEA, BMDWBE: ECOS, BHTCCZ: SEVEn, SEIADE: OEKO, UBONN, GRS, VDEES: FFII-LCOE, CMIT: ENEA, CCIAA Mi, IMQNL: Re/gent, NVWAPT: ADENE, ASAE

https://www.anti-circumvention.eu/

5

OEKOANTICSS project in brief

Kathrin Graulich (ANTICSS project manager)OEKO, Oeko-Institut – Institute for applied Ecology, Germany,

Sonia Martin – FFII-LCOE, Spain, test lab partner

Christoph Türk – VDE, Germany, test lab partner

Randolph van Kasteren – Re/genT, Netherlands, test lab partner

6

OEKOANTICSS Project in brief

7

OEKOAgenda





Examples

Questions & Answers

8

OEKOCircumvention: current legal framework

Energy Labelling Framework Regulation

Product-specific Energy Labelling delegated acts

Product-specific Ecodesign implementing acts

Standardisation Requests

standards should deter intentional and unintentional ‘circumvention’Prohibits the inclusion of software or hardware that automatically alters the performance of a product in test conditions

Verification procedure Non-compliance

Article on ‘Circumvention and software updates’ Verification procedure Non-compliance

Standard shall be designed to minimize the risk of circumventionStandards shall set test procedures that aim at minimizing the risk that the performance of a model can be automatically altered in test conditions

9

OEKOArticle ‘Circumvention’ in recently adopted Ecodesign regulations

10

OEKOCircumvention: current legal framework ! Constraints

Verification procedure: reference to ‘verification’ of non-compliance but without any reference to ‘identification’ of CV Definition:

No definition of CV in Article 2… but there are elements…software or hardware that automatically alters the performance of the product in test conditions……products designed to be able to detect they are being tested (e.g. by recognising the test conditions or test cycle), and to react specifically by automatically altering their performance during the test with the aim of reaching a more favourablelevel for any of the parameters declared by the manufacturer, importer or authorised representative in the technical documentation or included in any documentation provided…

Other conditions/situations of CV are not consideredLimitations to the application of elements in the ANTICSS approach:

Use of alternative test procedures for compliance verificationAssessment of deviations & interpretation of results

11

OEKOAgenda





Examples

Questions & Answers

12

OEKOANTICSS definition of ‚Circumvention‘

Circumvention is the act of designing a product or prescribing test instructions, leading to an alteration of the behaviour or the properties of the product, specifically in the test situation, in order to reach more favourable results for any of the parameters specified in the relevant delegated or implemented act, or included in any of the documentations provided for the product.

The act of circumvention is relevant only under test conditions and can be executed, e.g.,a) by automatic detection of the test situation and alteration of the

product performance and/or resource consumption during test, orb) by pre-set or manual alteration of the product,

affecting performance and/or resource consumption during test or c) by pre-set alteration of the performance within a short period

after putting the product into service.

13

OEKO

Jeopardy effects encompass all aspects of products or test instructions, or interpretation of test results, which do not follow the goal of the EU ecodesignand/or energy labelling legislation of setting ecodesign requirements and providing reliable information about the resource consumption and/or performance of a product.

ANTICSS definition of ‚Jeopardy effects‘

These effects may be not classified as circumvention but become possible due to loopholes or other weaknesses in standards or regulations.

14

OEKOCircumvention: Difference to ‚non-compliance‘

Ecodesign and energy labelling legislation states that ‘non-compliance’ can be determined only by Market Surveillance Authorities through product inspection, i.e. laboratory testing, and/or checking of the data and information provided in the technical documentation and/or any other information provided by the manufacturer or supplier against the requirements and conditions as defined in the legislation and standards.

In contrast, circumvention and jeopardy effects do not make a product appear as non-compliant during testing. In the first instance products appear to comply with all the requirements and conditions, but the test results are specifically influenced, resulting more favourable for the manufacturer, by the use of circumvention behaviour or by the exploitation of (possible) weaknesses or loopholes in standards and legislation.

15

OEKOCircumvention: Delimitation to ‚missing representativeness‘

Circumvention and jeopardy effects as defined by ANTICSS should not be confused with the fact that test standards might not always reflect typical consumer usage and for this reason measurements in real life might deviate from the declared performance parameters.

However, the more harmonized standards• deviate from typical user behaviour, • entail very specific conditions or • include ambiguities and loopholes, the higher is the likelihood that products are designed to be able to detect these test conditions or that manufacturers exploit the loopholes in a way to achieve more favourable results for their products, i.e. the risk of circumvention or jeopardy effects is increasing.

16

OEKOAgenda





Examples

Questions & Answers

17

OEKOANTICSS guidelines for test laboratoriesto target CV

https://www.anti-circumvention.eu/storage/app/media/D22_ANTICSS_Guidelines_test-labs_final.pdf

https://www.anti-circumvention.eu/storage/app/media/D22_ANTICSS_Guidelines_test-labs_final.pdf

18

OEKOIntro to the ANTICSS testing procedure to target CV

19

OEKOIntro to the ANTICSS testing procedure to target CV

20

OEKOPotential sources of suspicious behaviour

• Claims or prior information / reported cases on suspect behaviourfrom test labs, NGOs, consumer organisations, other MSAs, etc.

• Own analysis of technical documentation (for example declared values of a model more favourable than the ones of other similar models)

• Analysis of the Ecodesign and Energy Labelling regulations and applicable harmonised standards aiming at the identification of weaknesses/loopholes

• Information from the test lab that is performing the tests of the model according to the harmonised standard.

• Consideration of specific habits or situations prone to circumvention reflected in the ANTICSS guidelines

• Etc.

21

OEKOSuspicious habits detected within ANTICSS

22

OEKOInitial confirmation and categorization of the “suspect behaviour”

Is the reported “suspect behaviour” related to the suspect of circumvention behaviour?

If it is NOT considered as ‘hint for circumvention’ or ‘jeopardy effects’ END of the process.If YES process continues, following the next steps to confirm (or not) the initial categorizationthrough testing.

23

OEKODevelopment of analternative testing procedure

• Is there an applicable alternative existing procedure?

• Criteria for the development of an alternative testing procedure:

- It has to be developed on a case-by-case basis dependent on the specifically encountered type of suspicious behaviour.

- The aim is to detect inexplicable changes in the measurement results due the (slight) variation of preferably just one test conditionthat may indicate a circumventing behaviour of the tested product.

- It shall be designed as close as possible to the standard procedures.- It should be easy applicable by MSAs and test labs.- From time to time, it should be varied to avoid that manufacturers

aiming at circumventing otherwise get the chance to adapt their appliances to specifically to well-known alternative test procedures.

• It is advisable that the development be carried out in close cooperation between MSAs and the test laboratories.

24

OEKOTesting: Harmonised standard VS alternative test procedure

• The model is tested according to the harmonised standard or the transitional method of measurement.

• Afterwards the model is tested according to the alternative testing procedure. In this case, the same parameters as in the previous tests will be measured and the results will be compared with the corresponding values obtained in the tests according to the harmonised standard or the transitional method of measurement.

25

OEKOAnalysis of test results

USE OF VERIFICATION TOLERANCES FOR ASSESSING THE SIGNIFICANCE OF DEVIATIONS

If the deviation between the values obtained after having conducted tests according to the harmonised standard and the alternative test method exceeded the verification tolerances set out in the Ecodesign and/or Energy Labelling regulations, the result of the alternative test is considered to be‘significant’.

USE OF OTHER PARAMETERS FOR ASSESSING THE SIGNIFICANCE OF DEVIATIONS

For some alternative testing methods, the use of verification tolerances might not be the right approach if deviations can be expected already due to methodological differences between the alternative test method and the harmonised standard.

26

OEKOANTICSS categorization of test results

27

OEKOFinal interpretation of the test results (I)

The following aspects should be investigated to better understand the reasons for the deviations between the test results before the final judgement on the specific behaviour can be given:

• “Expected deviations” due to methodological differences between alternative test methods and harmonised standards.

• The alternative test procedure might differ too much from the harmonised standard, e.g. due to variation of more than one parameter.

28

OEKOFinal interpretation of the test results (II)

Assessing if the deviations of the tested product model in the alternative test could be a consequence of ‘circumvention’ or ‘borderline to circumvention’ was the most complicated step according to the experience of the ANTICSS project.

The following aspects were discussed thoroughly: Does the tested model reach more favourable results in the harmonized test? Does the tested model reach the more favourable results exclusively in the test situation, or is it

possible that the suspect behaviour could also occur in real-life use, even if only theoretically or in(extremely) rare situations? Not considered, as not verifiable just by testing: possible intention of the manufacturer, (deliberate or

unintentional)

For the final interpretation of the results, different views and opinions of experts areadvisable:

Discussion of the test results between the MSA in charge and the test laboratory Exchange with manufacturers to better understand the potential underlying (technical) reasons for

certain test results, manufacturers’ instructions or the reaction of a specific model to the alternativetest conditions. Exchange with further experts, e.g., other MSAs via the ADCO network, standardization experts

and/or policy representatives. In the end, also legal experts could be consulted.

29

OEKOAgenda





Examples

Questions & Answers

30

OEKOTest results of ANTICSS – Example 1: Dishwashers – specific loading instructions

DescriptionAccording to regulation or standard “The dishwasher manufacturer’s instructions regarding installation and use shall be followed.” In many dishwashers it is necessary to remove or alter the position of “accessories” fitted to the appliance when supplied. If the parts are not removed, the dishwasher cannot be loaded with the claimed full capacity (16ps). Instructions on removal of parts are only given in the ‘Instructions for Test Laboratories’, thus unlikely to be carried out in households in day-to-day use.

Methodology1. Tests according to harmonised standard EN 50242:2016, following the manufacturer’s instructions and loading scheme2. Alternative testing method:

− An alternative loading scheme is designed, fitting the maximum number of place settings, when the machine is loaded “as supplied”

− Tests are conducted according to standard conditions and manufacturer’s instructions without removing or altering accessories.

31


Standard loading schemeaccording to manufacturer‘s instructions(16 place settings)

Many accessories and third rack had tobe removed, cutlery basket split into half

ANTICSS alternative loading scheme(12 place settings)

Machine testedas supplied

32


Standard test results

ANTICSS alternative test results Deviation

Standard place settings (ps) 16 12 -25%

Specific energy consumption (Wh/ps)

47.2 60.9 +29%

Specific water consumption (L/ps)

0.68 0.91 +34%

Energy efficiency class A+++ A+++ No difference

33

OEKO

Conclusions about this case: circumvention

Loading instructions how to reach the declared capacity exclusively provided in instructions for test laboratories.

No information how to reach the declared capacity in the user instructions.

The specific loading instructions lead to reaching more favourable results (capacity) specifically under testing.

Capacity is a declared parameter on the Energy label (purchase criterion).

Higher capacity might facilitate a better Energy efficiency class.

Test results of ANTICSS – Example 1: Dishwashers – specific loading instructions

34

OEKOTest results of ANTICSS – Example 2: Washing machines. Testing with different loads

DescriptionSome machines can be optimised in a way to gain more favourable results for theenergy and water consumption specifically at the declared full load and half load used instandard testing, whereas the consumption values are comparably worse when themachine is run at different loads.

Methodology1. Tests according to harmonised standard EN 60456:2016, (2 cycles half load 40°C+ 2 cycles half load 60°C + 3 cycles full load 60°C) with the rated capacity declared by the manufacturer.2. Alternative testing method: (2 cycles half load 40°C+ 2 cycles half load 60°C + 3 cycles full load 60°C) Full load =6 kg; Half load =3 kg + testing at further partial loads additional loads

in case of suspicious test results

35


Energy consumption (60 °C) VS load

0,555

0,81

0,55

0,82

0,606

0,453

0,670,625

0,58

0,950,92

0,8750,82

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

1

0 2 4 6 8 10 12

Ener

gy c

onsu

mpt

ion

(kW

h)

Load (kg)

Standard Declared Expected Extra +alternative

36


Water consumption (60 °C) VS load

44,7

56,8

47,12

39,8

44 43,7

44,6

55,8 55,6 56,5

56,2

0

5

10

15

20

25

30

35

40

45

50

55

60

0 2 4 6 8 10 12

Wat

er c

onsu

mpt

ion

(l)

Load (kg)

Standard Expected Extra+alternative

37


Conclusions about this case: Borderline to circumvention

Test results for the energy and water consumption at full and half rated will also be achieved in real life when consumers load the machine with these capacities, even if this would only be theoretically or in (extremely) infrequent situations.

The regulations define the most efficient programme in terms of combined energy and water consumption. Further, the regulations prescribe a method of calculating the annual energy and water consumption using an approximation of the probable use of the washing machine by consumers (taking full / partial load for testing and calculation);

Although the measurement method does not include any reference to the other loads, this does not imply that the programme shall be efficient only for full / partial load as declared by the manufacturer.

ANTICSS discussions on the general categorisation of case: - Categorising the case as “jeopardy effect” (i.e. resulting in “borderline to circumvention” for the

tested model)- Categorising the case as “hints for circumvention” (i.e. resulting in “circumvention” for the

tested model)- Categorising the case as “missing representativeness of the standard

38

OEKOTest results of ANTICSS – Example 3: COLD appliances – Door opening test

Description• Energy saving modes (“ECO” modes) are more common these days• The harmonized standard (EN IEC 62552-1,-2,-3: 2020) tests lacks door openings, but

incorporates the possibilities of several energy saving functions (e.g. varying the defrost interval) which provides a better representativeness of the standard

• Modes are activated based on detection of door switch activations• Unknown what the difference is between “Normal” and “ECO” mode

• Within Anticss the deactivation of an external display was observed

39


Test method• Installation according to the harmonized standard• Randomizing parameters for:

• Amount of door openings per hour• Total open time per door opening

• Physical or simulated door openings

Testing tool• A testing tool is under development with a built-in randomizer• Developed as a first line check for suspicious behavior of an appliances

40


Testing tool• Information

41


Testing tool• Definitions

42


Testing tool• Test 1

43


Test results – What to look for• Additional component activation• Transition period between the “ECO” and “Normal” mode

• I.e. time between last door opening and switching to “ECO” mode

Test results – Conclusion• An impact assessment can be performed based on the additional energy

consumption of components and the transition period, e.g.:• Energy consumption increases 15% for a period of 10 minutes starting with a

door opening detection• Defrost heater behavior changes significantly due to door opening detection of

the appliance

44

OEKOAgenda





Examples

Questions & Answers

45

ANTICSS 11/05/2021Follow-up activities

• Recommendations for MSAsThe test lab provides the test results and a recommendation about thecategorisation of the model according to the results obtained (no circumvention,borderline to CV, circumvention)

If revisions of the applicable Ecodesign and/or Energy labelling regulations do notaddress the detected issue of circumvention or jeopardy effect, the testlaboratory is recommended to inform the MSA about this situation to promotethe relevant modifications

• Recommendations for standardization committeesIf revisions of the applicable test standard do not address the detected issue ofcircumvention or jeopardy effect, the test laboratory is recommended to informthe standardization committee about this situation to promote the relevantmodifications.

46

ANTICSS 11/05/2021Follow-up activities (II)

• Further investigation: Sampling of new modelsIf the MSA considers it necessary, it could be investigated if the circumvention case can occur in further models, either from the same manufacturer or in other models with the same characteristics from different manufacturers. The selection of the models, specifically targeting suspicious behaviourin terms of circumvention to be investigated will be crucial.

• The model selection procedure applied within ANTICSS was specifically targeted at finding appliances with a high probability of having a ‘circumvention’ behaviour. The method was a combination of models identified in previously reported cases of suspect ‘circumvention’ and ‘jeopardy effects’ and a semi-random, risk-based selection, which relied on framework technical/characteristics-based criteria in order to maximise the chances of identifying ‘circumvention’

47

ANTICSS 11/05/2021Follow-up activities (III)

48

OEKOAgenda





Examples

Questions & Answers

49

OEKO

50

OEKOANTICSS Contact & Disclaimer

Oeko-Institut e.V.Kathrin Graulich – ANTICSS project coordinatorE-Mail: [email protected]

www.anti-circumvention.euwww.twitter.com/AntiCircumventwww.linkedin.com/company/anticss/

https://cordis.europa.eu/project/rcn/213579_en.html

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 785122.This project presentation reflects only the author's view. The Agency and the Commission are not responsible for any use that may be made of the information it contains.

mailto:[email protected]

http://www.anti-circumvention.eu/

http://www.twitter.com/AntiCircumvent

http://www.linkedin.com/company/anticss/

anti-circumvention of standards for bettermarket surveillance

Documents