Answer:
By doing so, you avoid being distracted by what is going on around you and you give the speaker non-verbal acknowledgment that you are listening. It's okay to look away once in a while; staring intently at the person you're speaking with can make that person uncomfortable.
