Pupil detection

The reliable estimation of the pupil position in eye images is perhaps the most important prerequisite in gaze-based HMI applications. While there are many approaches that enable accurate pupil tracking under laboratory conditions, tracking the pupil in real-world images is highly challenging due to changes in illumination, reflections on glasses or on the eyeball, off-axis camera position, contact lenses, and many more.


Description ElSe:
The input to the algorithm is a grayscale image. After normalization, a Canny edge filter is applied to the image. In the next algorithmic step, edge connections that could impair the surrounding edge of the pupil are removed. Afterwards connected edges are collected and evaluated based on straightness, intensity value, elliptic properties, the possibility to fit an ellipse to it, and a pupil plausibility check. If a valid ellipse describing the pupil is found, it is returned as the result.
In case no ellipse is found a second analysis is conducted. To speed up the convolution with the surface difference and mean filter the image is downscaled. This operation is performed by calculating a histogram for all pixels from the large image influencing the pixel in the downscaled image. In each histogram, the mean of all intensity values up to the mean of the histogram is calculated and used as a value for the pixel in the downscaled image. After applying the surface difference and mean filter to the rescaled image, the best position is selected by multiplying the result of both filters and selecting the maximum position.
Choosing a pixel position in the downscaled image leads to a distance error of the pupil center in the full scale image. Therefore, the position has to be optimized on the full scale image based on an analysis of the surrounding pixels of the chosen position.

Description ExCuSe:
The input to the algorithm is a grayscale image. In the first step the color histogram is analysed and the algorithm decides if he expects the pupil to be bright or dark.
For bright pupils the algorithm selects the curvedd edge with the darkest enclosed intesity value. This is done by refining the result of the canny edge filtered image with morphologic operations. The remaining edges are analysed if they are curved or straight. For the curved edges the eclosed intesity value is calculated and the best is selected.
If the algorithm expects the pupil to be dark a threshold based on the standard deviation is calculated. The image gets thresholded and a coarse positioning is performed similarly to the radon transformation. This coarse position is optimized by moving to near intesity values which are darker or equal dark to their neigbourhood. After the optimization the image is threasholded by an increased threashold. All edges which are to far from pixels under the threashold are removed. The remaining edges get filterd as described befor for the bright pupil. Starting from the optmized position the surrounding edges are used to fit an ellipse.


Results remote:

Results microscope :

Download Paper:
Remote evaluation
Eyelid detection
Pupil detection microscope

Sourcecode downloads:

  • ExCuSe C++ code
  • ExCuSe Matlab code
  • ElSe C++ code (morphologic split)
  • ElSe C++ code (algorithmic split)
  • ElSe C++ code (morphologic split adjustable validity threshold for blink rejection)
  • ElSe C++ code (algorithmic split adjustable validity threshold for blink rejection)
  • ElSe C++ code (remote configuration)
  • Reimplementation George Link
  • Reimplementation Timm Link
  • Reimplementation Droege Link
  • Pupil detection through a ocular from a microscope
All algorithms (ftp)

Data sets:

  • ExCuSe data set
  • ElSe data set
  • Remote data set
  • Microscope pupil images
All data sets (ftp)

Example results:

Explanatory video (ExCuSe):