That should probably be
'b.txt'$'\n' in the latter, with the final single quote.
That looks like the output of GNU ls with the quoting style
shell-escape. Recent versions of GNU ls use some such quoting by default.
You can verify it's that by running
ls -l --quoting-style=shell-escape to see if that gives the same output. As the name says, the output is quoted like the shell does it, so it can be used as input to the shell. Hence, it should be possible to remove either or both files by just copypasting the quoted/escaped output from
ls to Bash:
ls -l ''$'\n' 'b.txt'$'\n
and then of course
rm ... after you verify you got the right files.
For what it's worth, the first filename is just a lone newline (for some reason the escaping adds the leading empty
''), the second is
b.txt and a newline at the end.