I have several months of Google Alerts in my email and I would like to share them somewhere on the web (for instance on a MediaWiki wiki). Is there an easy way to export them?

The alerts contain HTML of this kind (seen from Thunderbird):

<div style="color:#fff;font-size:9px;letter-spacing:0.8px"> <div style="padding:3px 0px 4px 4px;background:rgb(255,255,255);background-color:rgba(0,0,0,0.5);width:96px">La Provincia di Biella</div> </div> <!--[if gte mso 9]></div><![endif]--> </td> </tr> </table> </a> <div>  <span style="padding:0px 6px 0px 0px"> <a href="https://www.google.com/url?rct=j&amp;sa=t&amp;url=http://www.laprovinciadibiella.it/web/castelletto-cervo-nei-siti-cluniacensi-9458&amp;ct=ga&amp;cd=CAEYACoTMzc2NjcxMzA4NDYzNjc4NjUwMjIZZThjOGE1NjBlZjA5ZDRkMDppdDppdDpJVA&amp;usg=AFQjCNEW4QKACEvE_AbGrMGOBJ2y4dFSIw" style="color:#427fed;display:inline;text-decoration:none;font-size:16px;line-height:20px">Castelletto Cervo nei siti cluniacensi</a> </span>  <div> <div style="padding:2px 0px 8px 0px"> <div style="color:#737373;font-size:12px"> <a style="text-decoration:none;color:#737373">La Provincia di Biella</a> </div>

This worked for me:

  1. download the alerts from your email in a Thunderbird directory;
  2. install the ImportExportTools plugin and export the whole directory as HTML;
  3. go to the export directory and run: find * -print0 | xargs -0 -I § bash -c 'NAME="§"; DATE="{{#time:j F Y| ${NAME:0:8} }}"; grep --text -Eo "ru=([^&]+)&amp;" "§" | sed "s,ru=,;$DATE:,g" | sed "s,&amp;,,g" ; ' | sort -ru

I used ;$DATE: as prefix for each URL because that's meaningful wikitext and parser functions allow to easily translate the "AAAAMMDD" format used by the export files. You can replace with anything.

A smarter version may send the URL to Zotero or Citoid to get a proper citation, or maybe parse the original HTML to extract more information. The HTML format seems to be different for alerts before 2014-01-18.

Tested with GNU findutils 4.5.12, bash 4.3.42.

TODO: download the "see more articles" links and parse those too. In busy days, the emails only contain a random selection of the "top" news.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.