0   1  Down            3 0
 1   1  Down    Right   3 0
 1   0  Right           0 0
 1  -1  Up      Right   0 0
 0  -1  Up              1 0
-1  -1  Up      Left    1 0
-1   0  Left            2 0
-1   1  Down    Left    2 0

 1   0  Right           0 1
 1  -1  Up      Right   0 1
 0  -1  Up              1 1
-1  -1  Up      Left    1 1
-1   0  Left            2 1
-1   1  Down    Left    2 1
 0   1  Down            3 1
 1   1  Down    Right   3 1

 0  -1  Up              1 2
-1  -1  Up      Left    1 2
-1   0  Left            2 2
-1   1  Down    Left    2 2
 0   1  Down            3 2
 1   1  Down    Right   3 2
 1   0  Right           0 2
 1  -1  Up      Right   0 2

-1   0  Left            2 3
-1   1  Down    Left    2 3
 0   1  Down            3 3
 1   1  Down    Right   3 3
 1   0  Right           0 3
 1  -1  Up      Right   0 3
 0  -1  Up              1 3
-1  -1  Up      Left    1 3 

[ [0 1] [1 1] [1 0] [1 -1] [0 -1] [-1 -1] [-1 0] [-1 1] ]
  |-- 1 ----| |-- 2 -----| |-- 3 -------| |-- 4 ------|

Check direction for start index, iterate around, break at first white,
we now have our new point, and we check if it's a min or max.