What's the best and most flexible algorithm to detect any black (or colored pixel) in a given image file?

Say I'm given an image file that could, say, have a blue background. And any non blue pixel, including a white pixel, is counted as a "mark". The function returns true if there are X number of pixels that deviate from each other at a certain threshold.

I thought it'd be fastest to just simply iterate through every pixel and see if its color matches the last. But if it's the case that pixel (0,0) is deviant, and every other pixel is the same color (and I want to allow at least a couple deviated pixels before considering an image to be "marked), this won't work or be terribly efficient.