I haven't taken anything you said to heart. In fact, I can hardly recall anything you did say.
9 seconds sound clip from the The Remains of the Day movie soundboard.
You can hear this line at 01:52:52 in the Blu-ray version of the movie.
Quote context
[...]
- Then in that case aren't you as desperate as I to see him make this terrible mistake?
- He's being tricked! Don't you see what's going on here? Are you as deluded as he is?
- Oh, dear. Now I've probably offended you.
- No. Not at all, sir. Not at all.
- Well, you must excuse me. There are other gentlemen calling for me, sir.
- You mustn't take anything I said to heart. I was very foolish a little while ago.
- I haven't taken anything you said to heart. In fact, I can hardly recall anything you did say.
- I was just being very foolish.
- I simply haven't time to stand here with you, engaging in idle talk.
- I suggest you go to bed now. You must be very tired. Good night.
[...]