I am trying to get html data from servicenow website. It's using frames and I would like to extract the frame source using wget. I am unable to find the option/flags to do this job. I am only able to extract the page source but not the frame source.
wget -q https://company.service-now.com/task.do?sysparm_query=number=TASK0299719
A frame is a separate page with its own URL, just like a regular link. By using a frame, the creator of the first page is saying that the second page should be retrieved immediately and displayed within the first page, rather than waiting for the user to click on it and displaying it on its own.
Since wget doesn't display anything or have an interactive user, the distinction between frame and link is meaningless. Therefore you can think of this problem as a special case of "I want wget to follow a link".
wget has a recursive mode (-r) in which it follows links from the original URL and downloads them too. You probably don't want to download all of the linked URLs, so you need to narrow it down.
If this is a one-time operation, or if the frame's URL is stable, you can simply read the original page source yourself, find the iframe element's src attribute, and ask wget to get that.
If you need wget to do more of the work for you, can use the --follow-tags option to ask it to only get iframe URLs.
wget -r --follow-tags=iframe http://yourpage/
This will download the page and all of the iframes contained in it. If there are iframes inside the iframes, it will get those too, unless you also specify a recursion depth limit like -l 1.
frameset with frame elements rather than iframe elements, then of course the command should instead say --follow-tags=frame.
Nov 26, 2021 at 19:59