I have a large bibtex file with many entries where each entry has the general structure

item = {...},
item = {...},
item = {...},

(in some cases ARTICLE might be a different word e.g. BOOK)

What I would like to do is write a simple script (preferably just a shell script) to extract entries with given AuthorYear and put those in a new .bib file.

I can imagine that I can recognize the first sentence of an entry by AuthorYear and the last by the single closing } and perhaps use sed to extract the entry, but I don't really know how to do this exactly. Can someone tell me how I would achieve this?

It should probably be something like

sed -n "/AuthorYear/,/\}/p" file.bib

But that stops due to the closing } in the first item of the entry thus giving this output:

item = {...},

So I need to recognize whether the } is the only character at a line and only have 'sed' stop reading when this is the case.

  • I could only modify your code a little: sed -n "/AuthorYear/,/\}$/p". Note the $ symbol. It works fine, except that it doesn't print the closing } of a bibitem. Btw, is the use of sed necessary? – Barun Dec 19 '13 at 15:01
  • @Barun the use of sed is not necessary at all, I just thought that would be the easiest option. I have figured out a slightly different code: sed -n "/AuthorYear/, /^ *\}/p" which seems to do exactly what I want, including the closing } and correcting for spaces if there are any – Michiel Dec 19 '13 at 15:15

The following Python script does the desired filtering.

import re

# Bibliography entries to retrieve
# Multiple pattern compilation from: http://stackoverflow.com/a/11693340/147021
pattern_strings = ['Author2010', 'Author2012',]
pattern_string = '|'.join(pattern_strings)
patterns = re.compile(pattern_string)

with open('bibliography.bib', 'r') as bib_file:
    keep_printing = False
    for line in bib_file:
        if patterns.findall(line):
            # Beginning of an entry
            keep_printing = True

        if line.strip() == '}':
            if keep_printing:
                print line
                # End of an entry -- should be the one which began earlier
                keep_printing = False

        if keep_printing:
            # The intermediate lines
            print line,

Personally, I prefer moving to a scripting language when the filtering logic becomes complex. That, perhaps, has an advantage on the readability factor at least.

| improve this answer | |
  • Careful, there are lots of entries with nested {}s. If you can ensure the entry ends with \n}, you can stop with ^} – vonbrand Feb 26 at 16:12

I would recommend using a language with a battle-tested BibTeX library instead of reinventing that wheel. For example

#!/usr/bin/env perl
use strict;
use warnings;
use autodie;
use BibTeX::Parser;

open my $fh, '<', $ARGV[0];
my $parser = BibTeX::Parser->new($fh);
my @authoryear;
while (my $entry = $parser->next) {
    if ($entry->parse_ok) {
        if ($entry->key eq "AuthorYear") {
            push @authoryear, $entry;
    else {
        warn "Error parsing file: " . $entry->error;

# I'm not familiar with bibtex files, so this may be insufficient
open my $out, '>', "authoryear.bib";
foreach my $entry (@authoryear) {
    say $out $entry->raw_bibtex;

You'll probably have to install the module: cpan install BibTeX::Parser

| improve this answer | |

Now we also have the Python bibparsing module, that allows to analyze BibTeX databases with Python. For example I use the following script to calculate the number of authors in collaborative papers:

import sys
import bibtexparser as bp
with open(sys.argv[1]) as bibtex_file:
    bd = bp.load(bibtex_file)
    for art in bd.entries_dict:
    ae = bd.entries_dict[art]
    auths=ae[u'author'].split(" and ")
    print(auths[0]+" --- "+auths[-1])
| improve this answer | |

Another option would be to use bibtool.


bibtool -- select{$key ”AuthorYear”} input.bib -o output.bib

Check out the manual for specific cases.

| improve this answer | |

This is a Bash script which reads each line and uses regex matching to extract each entry that has the required pattern in its head. You can call it getbibs or something:

#!/usr/bin/env bash
# usage: ./getbibs pattern input.bib output.bib

while read entry; do
    if [[ $entry =~ ^@.*{$1,$ ]]; then
        printf "%s\n" "$entry" >> "$3"
        while read item; do
            [[ $item =~ ^@.*$ ]] && break
            printf "%s\n" "$item" >> "$3"
done < "$2"

To extract all entries with an author year of 1989 you could do:

$ chmod +x ./getbibs
$ ./getbibs 1989 file.bib author.bib

It might have some issues that I haven't tested yet, but it seems to work okay for the task.

| improve this answer | |

Just to be complete, the way I figured out myself, not as nice as some of the others, but it works:

entries=( AuthorYear1 AuthorYear2 )
for entry in "${entries[@]}" do
     sed -n "/"${entry}"/, /^ *\}/p" refs.bib 

It can be run from the commandline or put in a bash script.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.