I agree absolutely on analysing the residuals. In fact, this is obligatory when using linear regression in statistical inference. On the other hand, I think EDA on the explanatory variables is always useful, not for their correlations with the dependent variable (target), but to identify multi-collinearity among the explanatory variables themselves. Once identified, how you deal with it should ideally be guided by logic and domain expertise.