Visual Regression with OpenCV
Capture, compare, detect. Catch visual regressions before the trader sees them
OpenCV with SSIM (Structural Similarity Index) compares screenshots against baselines with intelligent masking to ignore dynamic data like real-time prices and timestamps.
As example, I built
goregress, a Go library wrapping OpenCV (via gocv) specifically designed for visual regression testing in CI. It supports three comparison methods (SSIM, pixel-diff, histogram), ignore regions for dynamic content, automatic baseline management, and diff image generation highlighting changed areas in red. The
Suite helper integrates directly with
testing.T for idiomatic Go test workflows.
// File: compare_test.go, goregress in action
func TestIgnoreRegions(t *testing.T) {
dir := t.TempDir()
img := gocv.NewMatWithSizeFromScalar(
gocv.NewScalar(128, 128, 128, 0), 100, 100, gocv.MatTypeCV8UC3,
)
defer img.Close()
// Modify only the top-left 20x20 corner
modified := img.Clone()
defer modified.Close()
roi := modified.Region(image.Rect(0, 0, 20, 20))
white := gocv.NewMatWithSizeFromScalar(
gocv.NewScalar(255, 255, 255, 0), 10, 10, gocv.MatTypeCV8UC3,
)
defer white.Close()
white.CopyTo(&roi)
bPath := filepath.Join(dir, "baseline.png")
cPath := filepath.Join(dir, "current.png")
gocv.IMWrite(bPath, img)
gocv.IMWrite(cPath, modified)
// Compare WITH ignoring the modified region, should pass
r2, err := goregress.Compare(bPath, cPath, &goregress.CompareOptions{
Threshold: 0.99,
Method: goregress.MethodPixel,
IgnoreRegions: []image.Rectangle{
image.Rect(0, 0, 20, 20), // mask out the changed area
},
})
if err != nil { t.Fatal(err) }
t.Logf("with ignore: similarity=%.4f", r2.Similarity)
}
Key insight: SSIM compares structure, luminance, and contrast, not individual pixels. It's more robust than pixel-diff against anti-aliasing and cross-browser rendering differences. With IgnoreRegions for dynamic market data (prices, timestamps, candlestick charts), you get visual regression testing at zero recurring cost, integrated directly into go test.
Next step: Visual AI with CNNs. goregress does not include CNN-based detection yet. But the research shows where this is heading: Chen et al. (2020) found that CNN models (YOLO, Faster R-CNN) significantly outperform traditional methods for GUI element detection. A trained model can distinguish a disabled button from an active one, verify an order book renders correctly, or detect a missing column in a positions table. Things that pure SSIM doesn't capture. AppFlow (Hu et al., 2018, 128 citations) proved ML can synthesize robust UI tests that survive partial redesigns without rewriting selectors. This is a natural evolution for goregress once the pixel-level foundation is solid.
References
- Repo Pondigo · go-regress: Visual regression testing with OpenCV in Go. GitHub
- Paper Yu, S. et al. (2021) · "Prioritize Crowdsourced Test Reports via Deep Screenshot Understanding" ICSE. PDF · arXiv
- Paper Yu, S. et al. (2021) · "Layout and Image Recognition Driving Cross-Platform Automated Mobile Testing" ICSE. PDF
- Paper Feiz, S. et al. (2022) · "Understanding Screen Relationships from Screenshots" ACM IUI. ACM DL
- Docs gocv.io · Go bindings for OpenCV. gocv.io
- Paper Chen, J. et al. (2020) · "Object Detection for GUI: Old Fashioned or Deep Learning?" ESEC/FSE. 207 citations. ACM DL
- Paper Moran, K. et al. (2018) · "ML-Based Prototyping of Graphical User Interfaces for Mobile Apps" IEEE TSE. 353 citations. PDF
- Paper Hu, G. et al. (2018) · "AppFlow: Using ML to Synthesize Robust, Reusable UI Tests" ESEC/FSE. 128 citations. ACM DL
- Paper YazdaniBanafsheDaragh, F. et al. (2021) · "Deep GUI: Black-Box GUI Input Generation with Deep Learning" ICSE. 56 citations. IEEE
- Paper Si, C. et al. (2024) · "Design2Code: Automating Front-End Engineering" PDF · arXiv