Hot answers tagged selenium
7
For simple cases of downloading the contents of a page, use curl or wget. Both are command line tools designed to download files over HTTP and have many options. In your case, you'll likely need to make these tools look more like a browser; lutzky's answer and penguin359's answer mention some curl and wget options that are useful in that respect.
Sometimes, ...
7
Yes, it's fully possible with curl. Most importantly will be to save and reload cookies between uses of curl with --cookie-jar. You can also post form data as needed. I usually use a Firefox add-on called Live HTTP Headers to capture what happens when I navigate a website. It will record any headers, but also any form posts which is very helpful when ...
3
My favorite is wget, so I'll give an example with that. What you want to do is replicate your browser session as closely as possible, so use the relevant commandline arguments. The ones which are necessary depend on how thoroughly the site checks your browser. --referer (sic) is usually sufficient, but you might also need --user-agent and --load-cookies.
...
3
You can use tail -f to keep reading from the file as it grows. Be careful with what you feed tail -f into. You can pipe tail -f into a filter that waits until the desired log line and quits. What won't work is if you pipe tail -f into a filter that pipes into another filter, because the intermediate filter will buffer its output. This works:
: >file.log ...
1
It is a little harder in straight shell script, but this is what I was using for quite a while for tomcat and oc4j:
perlscr='
alarm 120;
open F, "<$ARGV[0]";
seek F -($ARGV[1]*80),2;
while (1) {exit if (<F>=~$ARGV[2]);}'
window=10
scanfor="^INFO: Server startup in \d+ ms"
perl -e "$perlscr" $logfile $window "$scanfor" 2>&1 0<&1
The ...
1
Have you considered Watir?
http://watir.com/
When you've added the package, you can run it as a standalone file or from irb, line-by-line after include 'watir-webdriver'. I've found it to be more responsive than selenium-webdriver, but without the test recording GUI to help work out complex test conditions.
1
I just recently started using the WebDriver from Selenium 2 in Java. There is a driver called HtmlUnitDriver that fully supports JavaScript but does not fire up an actual browser.
It is not a light solution but it does get the job done.
I've designed the code to run from the command line and save the web data to files.
Only top voted, non community-wiki answers of a minimum length are eligible
