I generally prefer not to use Japanese in fanfics as well. In this case, I considered using "mistress" or "milady", but neither of them had the implication that the person being spoken to was a YOUNG upper-class woman. AFAIK, there isn't really an English word with the same meaning, so I chose to go with the Japanese in this case.