Priests having female housekeepers is still very common. Because men back then did not know how to take care of themselves. And some still do not. It was common for them to take their sisters as housekeepers just to avoid rumors. Or young priests purposely have housekeepers that are really old.

So by not denying in some occasions they were living as a couple, this was never the default. But yes rumors were and still are the default.