Well, it certainly never has been my party.
3 seconds sound clip from the After the Thin Man (1936) movie soundboard.
You can hear this line at 01:39:39 in the DVD version of the movie.
Quote context
[...]
- You knew your husband and Polly were friends, didn't you?
- No.
- But you knew there was a woman.
- Yes.
- Now, lieutenant... suppose we take up this check business...
- Wait a minute, who's doing this, you or me?
- Well, it certainly never has been my party.
- Maybe this guy is right. How do I know?
- Were you ever in this room before?
- No.
- Why, of course not! What do you think...
[...]