Take an odd number (2*n+1) of samples, sort them, and use the (n+1)th sample.

e.g.:

5, 9, 6     <- last 3 samples

5, 6, 9     <- last 3 samples, sorted
   ^
   |
   6        <- this is the value used for processing

You could also average the middle samples after sorting, if you need the signal to be smoother.

e.g.:

5, 2, 6, 10, 7   <- last 5 samples

2, 5, 6, 7, 10   <- last 5 samples, sorted
   ^  ^  ^
   |  |  |
   ---+---
      |
      6          <- average of middle 3 samples