What About the Outliers?

I think I first heard the term "outliers" when I was taking a research class for my master's degree. 

An outlier is an observation that lies outside the overall pattern of a distribution (Moore and McCabe 1999). source

At the time, I had had personal experience with it in my classroom as a teacher. When I would survey my students in order to get feedback about my teaching, there they would be... the outliers. A few comments that didn't align with the rest. They were "outside the pattern of distribution." 

As an administrator, I have experienced it as well. A boss who prefaced the comment with, "Maybe I'm the only one who has said this to you before..." or the very few responses from teachers/staff on an end-of-year feedback survey that are not even close to where the rest of the responses are.

What should we do with these? Should they be discarded? Or written off as comments from disgruntled students/employees?

What do you do with the outliers?