What he does isn't intended to be annoying. If he's getting on your nerves, you just take a break. Spend time away from him.
9 seconds sound clip from the Rain Man (1988) movie soundboard.
You can hear this line at 01:01:22 in the Blu-ray version of the movie.
Quote context
[...]
- Who's on first. What's on second.
- Huh?
- They set up an initial program where the Pony Express would ride for approximately 20 miles
- What am I supposed to do? There's gotta be something that I can do here.
- I'm not a psychiatrist, but I do know that his brain doesn't work like other people.
- Sure, I'll just send him back.
- Sorry?
- Nothing, just kidding, it's an inside joke.
[...]