An outlier is an observation that lies "far away" from other values in a random sample from a population. What constitutes "far away" is often up to the analyst given the "normal" observations look like and the context of the problem. They may be an indicator of data errors or a rare events, and should be investigated carefully to understand why they appear in our sample and they are influential or not. The analysis should be run with and without them, and if no reasonable explanation can be attached for their existence before elimination, the results of both analysis should be reported.

While doing the EDA a quick visual way to check for the outliers for continuous data is via scatterplots and boxplots. In particular, calculating the inter-quartile range (IQR) and using its multiples can help us define the outliers. The IQR = Q3 - Q1, where Q1 is the first quartile, and Q3 the third quartile. The potential outliers lie outside the range of:

[Q1 - (1.5 × IQR), Q3 + (1.5 × IQR)],

and problematic outliers lie outside of:

[Q1 - (3 × IQR), Q3 + (3 × IQR)].

While fitting models such as linear regression, we also use the residuals plots and analysis to address potential influential points, e.g. leverage plots, Cook's distance, etc...