That is the normal/expected behaviour for an ANSI terminal.
When you type Esc the terminal starts processing an eventual ANSI escape sequence (if you went on typing [ and then A the cursor would go up).
So, after Esc the terminal waits for the next character in the sequence, if it does not belong to a know sequence then it stops processing, but both Esc and the next character you typed will be lost (they were part of an illegal sequence, and so the are dropped).
How to stop it? One thing you could do is change the inter character timeout for the terminal by calling for example
timeout() from a program. See this:
While interpreting an input escape sequence, wgetch sets a timer while waiting for the next character. If notimeout(win, TRUE) is called, then wgetch does not set a timer. The purpose of the timeout is to differentiate between sequences received from a function key and those typed by a user.