I think your original analysis is accurate. That is how both cult and sexual assault survivors are called liars. The idea is to throw doubt into the survivor's testimony, and point out all the years that have passed to suggest nobody can remember details that far back. Then, the real key...even if the question is answered ("nope...not once") the accuser(s)' last word is far from being given.