11

I have a large bibtex file with many entries where each entry has the general structure

@ARTICLE{AuthorYear,
item = {...},
item = {...},
item = {...},
etc
}

(in some cases ARTICLE might be a different word e.g. BOOK)

What I would like to do is write a simple script (preferably just a shell script) to extract entries with given AuthorYear and put those in a new .bib file.

I can imagine that I can recognize the first sentence of an entry by AuthorYear and the last by the single closing } and perhaps use sed to extract the entry, but I don't really know how to do this exactly. Can someone tell me how I would achieve this?

It should probably be something like

sed -n "/AuthorYear/,/\}/p" file.bib

But that stops due to the closing } in the first item of the entry thus giving this output:

@ARTICLE{AuthorYear,
item = {...},

So I need to recognize whether the } is the only character at a line and only have 'sed' stop reading when this is the case.

2
  • I could only modify your code a little: sed -n "/AuthorYear/,/\}$/p". Note the $ symbol. It works fine, except that it doesn't print the closing } of a bibitem. Btw, is the use of sed necessary?
    – Barun
    Dec 19, 2013 at 15:01
  • @Barun the use of sed is not necessary at all, I just thought that would be the easiest option. I have figured out a slightly different code: sed -n "/AuthorYear/, /^ *\}/p" which seems to do exactly what I want, including the closing } and correcting for spaces if there are any
    – Michiel
    Dec 19, 2013 at 15:15

7 Answers 7

8

I would recommend using a language with a battle-tested BibTeX library instead of reinventing that wheel. For example

#!/usr/bin/env perl
use strict;
use warnings;
use autodie;
use BibTeX::Parser;

open my $fh, '<', $ARGV[0];
my $parser = BibTeX::Parser->new($fh);
my @authoryear;
while (my $entry = $parser->next) {
    if ($entry->parse_ok) {
        if ($entry->key eq "AuthorYear") {
            push @authoryear, $entry;
        }
    }
    else {
        warn "Error parsing file: " . $entry->error;
    }
}

# I'm not familiar with bibtex files, so this may be insufficient
open my $out, '>', "authoryear.bib";
foreach my $entry (@authoryear) {
    say $out $entry->raw_bibtex;
}

You'll probably have to install the module: cpan install BibTeX::Parser

2

The following Python script does the desired filtering.

#!/usr/bin/python
import re

# Bibliography entries to retrieve
# Multiple pattern compilation from: http://stackoverflow.com/a/11693340/147021
pattern_strings = ['Author2010', 'Author2012',]
pattern_string = '|'.join(pattern_strings)
patterns = re.compile(pattern_string)


with open('bibliography.bib', 'r') as bib_file:
    keep_printing = False
    for line in bib_file:
        if patterns.findall(line):
            # Beginning of an entry
            keep_printing = True

        if line.strip() == '}':
            if keep_printing:
                print line
                # End of an entry -- should be the one which began earlier
                keep_printing = False

        if keep_printing:
            # The intermediate lines
            print line,

Personally, I prefer moving to a scripting language when the filtering logic becomes complex. That, perhaps, has an advantage on the readability factor at least.

1
  • Careful, there are lots of entries with nested {}s. If you can ensure the entry ends with \n}, you can stop with ^}
    – vonbrand
    Feb 26, 2020 at 16:12
1

Now we also have the Python bibparsing module, that allows to analyze BibTeX databases with Python. For example I use the following script to calculate the number of authors in collaborative papers:

#!/usr/bin/python
import sys
import bibtexparser as bp
with open(sys.argv[1]) as bibtex_file:
    bd = bp.load(bibtex_file)
    for art in bd.entries_dict:
    print("*********")
    ae = bd.entries_dict[art]
    print(ae[u'title'])
    auths=ae[u'author'].split(" and ")
    print(len(auths))
    print(auths[0]+" --- "+auths[-1])
1

Another option would be to use bibtool.

Example:

bibtool -- select{$key ”AuthorYear”} input.bib -o output.bib

Check out the manual for specific cases.

1

I made an expanded version of Michels own answer that takes bib-file with lots of entries and splits it up into one bib-file per entry. this could easily be expanded to other purposes.

Most of this is is a straightforward implementation of Michels answer into a do-loop. However, I added a comma within the bash-expansion:

 sed -n "/"${entry},"/, /^ *\}/p" work-bibliotek.bib 

instead of

sed -n "/"${entry}"/, /^ *\}/p" work-bibliotek.bib 

otherwise an entry like

 author2010fo 

will include the entry for

author2010fobar 

This caused me quite a bit of headaches today, finding that error. So perhaps I just saved someone else those hours instead :)

I also included a test to see if all entries are unique, and it excudes @comments made by jabref.

#!/bin/bash
# bibliotek-entries-for-sig.sh


cd /home/emil/Dropbox/bibliotek


# get all entries
entries=$(rg '@' work-bibliotek.bib | rg -v '@Comment' | grep -Po '(?<={)\w+')

nonuniques=$(echo "$entries" | wc -l)
uniques=$(echo "$entries" | uniq  | wc -l)




if [ $nonuniques != $uniques ]; then 
    # echo "hey"
    notify-send "citekeys not unique. exitting" --urgency critical 
    exit

else

    # remove old ones
    rm singles-entries/work-bibliotek-entries/*.bib

echo "$entries"



    for entry in $entries; do
     
     # the comma is important otherwise 
     # author2010fo 
     # will be the same as
     # author2010fobar 
     sed -n "/"${entry},"/, /^ *\}/p" work-bibliotek.bib > "singles-entries/work-bibliotek-entries/$entry.bib"
    done


fi 
0

This is a Bash script which reads each line and uses regex matching to extract each entry that has the required pattern in its head. You can call it getbibs or something:

#!/usr/bin/env bash
# usage: ./getbibs pattern input.bib output.bib

while read entry; do
    if [[ $entry =~ ^@.*{$1,$ ]]; then
        printf "%s\n" "$entry" >> "$3"
        while read item; do
            [[ $item =~ ^@.*$ ]] && break
            printf "%s\n" "$item" >> "$3"
        done
    fi
done < "$2"

To extract all entries with an author year of 1989 you could do:

$ chmod +x ./getbibs
$ ./getbibs 1989 file.bib author.bib

It might have some issues that I haven't tested yet, but it seems to work okay for the task.

0

Just to be complete, the way I figured out myself, not as nice as some of the others, but it works:

entries=( AuthorYear1 AuthorYear2 )
for entry in "${entries[@]}" do
     sed -n "/"${entry}"/, /^ *\}/p" refs.bib 
done

It can be run from the commandline or put in a bash script.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.