Just a random thought:
For each line:
Calculate if differences to previous frame is 0
Calculate if differences to next frame is 0
Should be fast to calculate, you are doing sequential read with early break, which should be really early for fast moving games, cars, or games moving the camera continously to simulate head bob, FPSs.
Classify frame as:
- Z: zero common lines
- P: common lines with Previous frame
- N: common lines with Next frame
- B: common lines with both frames
State machine (array):
State: Previous frame classification
Input: Current frame classification
Transition action: value to add to frame counter
Now fill in the table, f.ex.: Z->Z = +1
Maybe this would work, or maybe more state is needed, like both previous frame classifications or the max,min (Y coordinate) of identical lines.
The implementation and validation is left as an exercise to the reader.
Some games may need to adjust the Y-window to check, if they have horizontal slow changing borders (health meters or whatevers).