Methodology
How we calculate the numbers on our comparison pages. Every metric is derived from stored review data using published formulas.
Data Collection
We scrape product reviews from public retailer APIs (Ace Hardware, Costco, Home Depot) and brand websites using their Bazaarvoice review widgets. All review text, ratings, dates, and product associations are stored in raw JSON files. We do not generate, fabricate, or estimate any review data.
Product Rating (1-5 stars)
The weighted average star rating across all products and sources for a brand.
Product Approval % is the percentage of all reviews rated 4 or 5 stars.
Warranty Review Classification
Not every review discusses the warranty. We use a two-tier keyword system to identify warranty-relevant reviews:
Tier 1 — Direct warranty terms (any rating)
warranty, warrante, warranty claim, warranty repair, warranty replace, warranty service, warranty cover, warranty honored, extended warranty
Tier 2 — Service interaction terms (only if rating ≤ 3, or combined with Tier 1)
customer service, support ticket, tech support, call center, RMA, service center, authorized dealer, repair center
This tiered approach prevents product quality complaints (e.g., “it broke”) and retailer return issues from being counted as warranty experiences.
Warranty Satisfaction Rate (WSR)
An NPS-style score from -100 to +100, calculated only from reviews that mention warranty topics. Uses the same promoter/detractor methodology as Net Promoter Score.
Promoters = warranty reviews rated 4-5 stars
Detractors = warranty reviews rated 1-2 stars
Passives = warranty reviews rated 3 stars (excluded)
WSR = (Promoters / Total) − (Detractors / Total) × 100
+25 or higher
Good
-25 to +24
Mixed
Below -25
Poor
Experience Gap
The difference in approval rates between overall product reviews and warranty-specific reviews.
A large gap (e.g., 70+ points) means customers love the products but have poor warranty experiences. A small gap suggests the warranty experience matches overall product quality.
Confidence Levels
WSR reliability depends on sample size. We label each brand's data:
Limitations
- Review data comes from public retail sites. Customers who leave reviews may not be representative of all buyers.
- Warranty reviews skew negative because satisfied customers rarely mention their warranty.
- Some brands have limited retail presence on our sources, resulting in smaller sample sizes.
- Keyword matching may miss some warranty discussions or include tangentially related reviews.
- Data is refreshed periodically and may not reflect the most recent warranty policy changes.