It sounds like you want to run GUI programs. When you start Emacs, if a GUI environment is available, it opens a GUI window; otherwise, it runs in the terminal. I don't know about Matlab but I suspect that the same thing happens. Your question is confusing:
matlab $ is invalid shell syntax, but I suspect that you're running
matlab &. When you put
& after a command line, the command runs in the background. This is fine for a GUI application, but can't be done for a text mode application: it'll need to go back to the foreground as soon as it wants to access the terminal.
If you want to run Emacs, Matlab or other commands in text mode in a PuTTY window and switch between them, run Screen or Tmux in the remote shell launched by PuTTY. Screen and Tmux are terminal multiplexer applications: they provide multiple windows where you can run separate applications, and let you switch between the windows. So for example, to run Matlab, open a new window and run
& since Matlab will be in the foreground of that window). Screen or Tmux have the benefit that if the SSH connection is closed due to a network glitch, the terminal session remains active and you can restart PuTTY and reconnect to the existing session.
If you want to run Matlab (or any other application) in GUI mode, you need X11 forwarding. The Linux GUI is based on the X Window System (X11), which have a communication protocol between applications and the display server that can be forwarded over the network. You'll need an X11 server on the Windows machine for that. As part of my Windows survival kit, I use Xming; download and install the free version. Run Xming (put it in your Windows session startup if you like). In the PuTTY configuration, under “Connection → SSH → X11”, make sure to enable X11 forwarding and set the “X display location” to
localhost:0 (see e.g. this tutorial). After this, when you connect with PuTTY, you'll be able to run remote GUI applications.
For Emacs, I recommend running it locally and editing remote files. Install EmacsW32, which has slightly improved Windows support from the official sources. Emacs can open remote files pretty much transparently through Tramp — just type
/remotehostname:path/to/file to open a file under your home directory or
/remotehostname:/path/to/file to type an absolute path. Set up Tramp to use the
plinkx method and point Emacs to the
plink executable from PuTTY.