Look here, it's not a question of your father leaving us. You're simply being asked to reconsider his duties.
7 seconds sound clip from the The Remains of the Day movie soundboard.
You can hear this line at 00:29:06 in the Blu-ray version of the movie.
Quote context
[...]
- My friend Bremann was ruined by inflation. Couldn't get a job in postwar Germany. Killed himself.
- Shot himself in a railway carriage between Hamburg and Berlin.
- So you see, seems this conference is crucial, and we really can't run the risk of any accidents.
- Of course, My Lord. I understand fully.
- Good. Well, I'll... leave you to think about this then, Stevens.
- You... Look, I'm short-handed in the dining room tonight. I can use you in the servery.
- What? Thank you, Mr. Stevens, sir.
[...]