At first I thought that the backslashes would self-escape within the double-quotes and that was the problem, but, on second-thought,
"\\[" is equivalent to
'\[' so this is not the case - it would have worked that way.
But the real problem was that
readline did not know how many characters had been drawn to the screen and how many were intercepted as terminal escapes. In fact, it likely thought no prompt had been printed at all because your prompt consisted of what was essentially an open-quoted string.
So, as I noted in the comment, you needed to close the sequence. The
\[ means begin non-printing escape sequence in the prompt - it's so
readline can keep track of how many chars are drawn on the screen. You also need to end it like:
PS1='\[non-printing terminal escapes here\]'
man bash 2>/dev/null | grep '^ *\\\[' -A5
\[ begin a sequence of non-printing
characters, which could be used to
embed a terminal control sequence
into the prompt
\] end a sequence of non-printing char‐