I'm less interested in the dialog during sex (as it is generally much the same anyway) but more in the dialog of the story surrounding the sex, so that's ok. However, github claims 8 GB of video memory should be enough for the medium model, so maybe I should try that next.
I will try the condition_on_previous_text option. I guess since it only takes a few minutes I can do a few runs with different settings and see how that affects things.
If you're trying to get away with just Small, and it only takes a few minutes, you can try to do runs with a low temperature (but not zero) and like best_of 10 or something. Part of the key to Whisper is doing settings that try to compensate for whatever your computational stuff is limited by.