My Internet company set up a local server, where different files are hosted. I can download files at much higher speed from there. But for some reason, the server is not accessible from Ubuntu 13.10 I am using, or Fedora 19. So the problem should be Linux specific.
N.B. It is perfectly accessible from windows. If it helps, it is not accessible via router too. Probably they are using some IP Address checking, which can't be done via router.