Why they say they did it this way
NCR also disclosed in that same page that they sell research reports to manufacturers and ran a paid installer-verification program — worth weighing alongside the "impartial" framing. Full context on our sources page.
Setup of the test stand
In NCR's own words, describing the physical rig every product was tested on:
Simulated roof
“The simulated roof measures 47 inches across, containing 4 isolated (sealed from each other with silicone) test bays measuring 8 inches across. Roof pitch can be made to vary from 4/12 (18.43 deg) to 12/12 (45 deg). The roof was covered in Owens Corning Oakridge Desert Tan AR Laminate Shingles Constructed from Asphalt & Fiberglass. The gutters were Amerimax 5-in White Metal K Gutters.”
Rain simulator channels
“The Rain Simulator Channels establish a consistent water-flow across all four testing bays... A controllable In-Line Ball Valve at each ‘bay-channel’ provided the most consistent results – each valve was adjusted to ensure that the flow was even to all four bays.”
Simulated accelerated rainwater
“We used unfiltered and untreated well water and added 100ml of sulfuric acid (to simulate acid rain), 100ml of molasses (for simulated foliage debris leaching), 50ml of 2 cycle motor oil (to simulate shingle leaching) as well as 1 tablespoon of sea salt (to simulate higher salt climates). We measured pH, nitrogen, phosphorus, and potassium levels before and after.”
Effective Protection Coefficient
Shingle grit & fine particle testing
“This test is performed by adding one (1oz) full shot-glass (about 61 grams) of shingle grit material to a marked point on the roof... Following complete drying of the gutter and protection product, a 31cc 2-Cycle Gas Blower is engaged on maximum power from a set distance and angle... Then the amount of Shingle debris that remains on top of the gutter cover as well as beneath (within the gutter) is measured with a digital scale.”
Leaf & pine needle penetration and clog test
“This test is performed by adding one cup of both dry and green leaves, as well as one cup of pine needles... When the gutter and gutter protection are completely dry, a 31cc 2-Cycle Gas Blower is engaged on maximum power... The amount of leaves and pine needles that remain on top of the gutter cover as well as beneath (within the gutter) is measured with a digital scale.”
Shredded leaf / twig penetration and clog test
“This test is performed by adding one cup shredded mixture of green and dry leaves, pine needles and twigs of various sizes... to a marked point on the roof (white line) while the water is flowing.”
Strength
Impact testing
“This test is performed by dropping a 2, 3 and 5 pound weights onto of the gutter cover. Following each weight drop the gutter is going to be tested to see if it still is #1 intact and able to process water and #2 is still within the gutter. The weights are padded with a rubber coating, to simulate a tree branch.”
Water throughput
UV dyed water flow testing
“This test is performed by adding a UV dye to the water and flowing the water over the simulated roof. Next we put up a high powered UV light and our cameras are outfitted with a special lens. Then we film and photograph the water flowing into (and sometimes over) the gutter cover. We adjust the roof pitch at 4/12 and 12/12.”
Over-flow
“This test is performed in conjunction with the Leaf, Pine Needle, Shredded Leaf and twig penetration testing. We placed a marked container at below the lip of each bay of covers and following 10 minutes of the conclusion of the clog testing the measurement of the amount of water that is within the marked container at that specific bay.”
Durability
UV laser weathering
“This test is performed by inserting the gutter covers into a CNC laser machine and running a uniform elliptical pattern on all of the gutter covers with about 65W of power... to simulate the UV rays that would occur naturally when the gutter covers are installed atop a roof.”
Accelerated weathering
“For the duration of about two years, we kept the gutter guard covers within a cooler that contained the base of the fluid that was the simulated rainwater – and to that we added an additional 300ml of sulfuric acid, 330ml of molasses, 200ml of 2 cycle motor oil as well as 5 tablespoons of sea salt. The cooler was placed outside as to further accelerate the testing with heat variation.”
Product sustained integrity
“Product sustainability and integrity is determined by how well the products stand up to all of the testing over 24 months. This evaluation is a major factor in determining the rating for the durability of the product.”
Value and Industry Reputation were not physical lab tests — NCR based those on installed pricing research and on tracking each brand's standing and complaints over time. — NCR Consumer Advocacy Group, “Testing Methodology,” archived March 17, 2017