Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have pubmed ids for files i need to download from the pubmed site in .pdf format. I tried the following script:

#!/usr/bin/env bash

Link="http://www.ncbi.nlm.nih.gov/pubmed/"

ID=(10021369         
10051005
10051007
100562
10064668
10071185
)

for f in ${ID[@]};
do
   wget  --user-agent="Mozilla/5.0 (Windows NT 5.2; rv:2.0.1) Gecko/20100101 Firefox/4.0.1" \
         -l1 --no-parent -A.pdf ${Link}${f}/pdf/ -O ${f}.pdf
done

I got pdf files downloaded but can't open any of them

share|improve this question
3  
What error do you get when you try to open them? What is the output from file 10021369.pdf after you downloaded it? –  Hennes Nov 12 '13 at 17:55
add comment

1 Answer

Why would you expect to be able to download PDFs from that URL? PubMed does not usually offer PDFs, it gives you a link to the journal's webpage and you get the PDF from there.

Anyway, your script is fine, the issue is that the links you are giving it do not point to PDF files but XML files and that is what it is downloading:

$ ls
10021369.pdf  10051007.pdf  10064668.pdf
10051005.pdf  100562.pdf    10071185.pdf

$ file *
10021369.pdf: XML document text
10051005.pdf: XML document text
10051007.pdf: XML document text
100562.pdf:   XML document text
10064668.pdf: XML document text
10071185.pdf: XML document text

In order to have it download PDFs, you will need to give it links that point to PDF files. You can test this by pointing your browser to one of the links that your script visits. For example, http://www.ncbi.nlm.nih.gov/pubmed/10051005/pdf/. If you visit that link you will see that it is not a PDF file.

If all your PMID are links to papers that are also in PubMed Central, you should be able to get the PDFs after first converting the PubMed IDs to PubMed Central IDs:

#!/usr/bin/env bash

Link="http://www.ncbi.nlm.nih.gov/pubmed/"
PMCLink="http://www.ncbi.nlm.nih.gov/pmc/articles/"
ID=(10021369         
10051005
10051007
100562
10064668
10071185
)

for f in ${ID[@]};
do
  PMCID=$(wget  --user-agent="Mozilla/5.0 (Windows NT 5.2; rv:2.0.1) Gecko/20100101 Firefox/4.0.1" \
   -l1 --no-parent ${Link}${f} -O - 2>/dev/null | grep -Po 'PMC\d+' | head -n 1)
    if [ $PMCID ]; then
       wget  --user-agent="Mozilla/5.0 (Windows NT 5.2; rv:2.0.1) Gecko/20100101 Firefox/4.0.1" \
            -l1 --no-parent -A.pdf ${PMCLink}${PMCID}/pdf/ -O ${f}.pdf 2>/dev/null
    else
       echo "No PMC ID for $f"
    fi

done

Running this script will download the 2 freely available PDFs from the IDs you gave (10051005.pdf and 10051007.pdf) and print an error for the rest:

No PMC ID for 10021369
No PMC ID for 100562
No PMC ID for 10064668
No PMC ID for 10071185

You will have to get the rest manually or figure out how to parse the relevant URLs.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.