4

I was given a long word document which I have to port to Latex. In the document all citations appear in the classic form with author and year. Something like

Lorem ipsum dolor (Sit, 1998) amet, consectetur adipiscing (Slit 2000, Sed and So 2002, Eiusmod et al. 1976).
Tempor incididunt ut labore et dolore magna aliqua (Ut et al. 1312)

This references need to get the proper key reference as it appears in a list of bib references. In other words the text should translate to

Lorem ipsum dolor \cite{sit1998} amet, consectetur adipiscing \cite{slit2000,sed2002,eiusmod1976}.
Tempor incididunt ut labore et dolore magna aliqua \cite{ut1312}

That means:

  • extract all the strings that are composed of name(s) and year enclosed in parentheses
  • strip that string of spaces, second names (everything after the first name) and capital letters
  • use the resulting string to form the new \cite{string}

I understand that this may be quite a complex task. I was wondering maybe someone has written a script fo this specific task. Alternatively any partial suggestion is also welcome. I am currently working in MacOS.

  • 1
    Is Sit, 1998 acceptable input? It's slightly different from, e.g. Slit 2000, by virtue of the comma between the author and year. (Just looking at the 2nd citation, where multiple cites are separated by commas) – Jeff Schaller Feb 4 at 18:43
  • So basically the inputs are a bit all over the place with , ; and et al. notations. Most of them are in the form (author et al. 2012). With multiple authors these are usually separated by ;. What I am trying to get at is something that would work for maybe half or two third of the inputs and do the rest manually. – Manfredo Feb 4 at 22:07
  • 3
    If you can show as many of those possible inputs, that would help testers/answerers. – Jeff Schaller Feb 4 at 23:08
  • A solution for the 3 cases I've shown will most likely solve enough entries. Also as not that experienced with this type of operation it is more to have a clue on how to proceed. – Manfredo Feb 5 at 8:37
  • Does your input file include "arbitrary" line-breaks, or can we assume that opening parentheses are closed on the same line? – AdminBee Feb 10 at 9:38
5
+50

The following awk program should work. It looks for ( ... ) elements in each line and checks if they fit the "author(s), year" or "author(s)1 year1, author(s)2 year2, ..." pattern. If so, it creates a citation command and replaces the ( ... ) group; otherwise it leaves the group as it is.

#!/usr/bin/awk -f


# This small function creates an 'authorYYYY'-style string from
# separate author and year fields. We split the "author" field
# additionally at each space in order to strip leading/trailing
# whitespace and further authors.
function contract(author, year)
{
    split(author,auth_fields," ");
    auth=tolower(auth_fields[1]);
    return sprintf("%s%4d",auth,year);
}



# This function checks if two strings correspond to "author name(s)" and
# "year", respectively.
function check_entry(string1, string2)
{
    if (string1 ~ /^ *([[:alpha:].-]+ *)+$/ && string2 ~ /^ *[[:digit:]]{4} *$/) return 1;
    return 0;
}




# This function creates a 'citation' command from a raw element. If the
# raw element does not conform to the reference syntax of 'author, year' or
# 'author1 year1,author2 year2, ...', we should leave it "as is", and return
# a "0" as indicator.
function create_cite(raw_elem)
{
    cite_argument=""

    # Split at ','. The single elements are either name(list) and year,
    # or space-separated name(list)-year statements.
    n_fields=split(raw_elem,sgl_elem,",");

    if (n_fields == 2 && check_entry(sgl_elem[1],sgl_elem[2]))
    {
        cite_argument=contract(sgl_elem[1],sgl_elem[2]);
    }
    else
    {
        for (k=1; k<=n_fields; k++)
        {
            n_subfield=split(sgl_elem[k],subfield," ");

            if (check_entry(subfield[1],subfield[n_subfield]))
            {
                new_elem=contract(subfield[1],subfield[n_subfield]);
                if (cite_argument == "")
                {
                    cite_argument=new_elem;
                }
                else
                {
                    cite_argument=sprintf("%s,%s",cite_argument,new_elem);
                }
            }
            else
            {
                return 0;
            }
        }
    }


    cite=sprintf("\\{%s}",cite_argument);
    return cite;
}




# Actual program
# For each line, create a "working copy" so we can replace '(...)' pairs
# already processed with different text (here: 'X ... Y'); otherwise 'sub'
# would always stumble across the same opening parentheses.
# For each '( ... )' found, check if it fits the pattern. If so, we replace
# it with a 'cite' command; otherwise we leave it as it is.

{
    working_copy=$0;

    # Allow for unmatched ')' at the beginning of the line:
    # if a ')' was found before the first '(', mark is as processed
    i=index(working_copy,"(");
    j=index(working_copy,")");
    if (i>0 && j>0 && j<i) sub(/\)/,"Y",working_copy);

    while (i=index(working_copy,"("))
    {
        sub(/\(/,"X",working_copy); # mark this '(' as "already processed

        j=index(working_copy,")");
        if (!j)
        {
            continue;
        }
        sub(/\)/,"Y",working_copy); # mark this ')', too


        elem=substr(working_copy,i+1,j-i-1);

        replacement=create_cite(elem);
        if (replacement != "0")
        {
            elem="\\(" elem "\\)"
            sub(elem,replacement);
        }

    }
    print $0;
}

Call the program with

~$ awk -f transform_citation.awk input.tex

Note that the program expects the input to be "reasonably" well-formed, i.e. all parentheses on a line should be matched pairs (although one closing parentheses at the beginning of the line is allowed for, and unmatched opening parentheses will be ignored).

Note also that some of the syntax above requires GNU awk. To be portable to other implementations, replace

if (string1 ~ /^ *([[:alpha:].-]+ *)+$/ && string2 ~ /^ *[[:digit:]]{4} *$/) return 1;

with

if (string1 ~ /^ *([a-zA-Z.-]+ *)+$/ && string2 ~ /^ *[0123456789][0123456789][0123456789][0123456789] *$/) return 1;

and ensure you have set the collation locale to C.

| improve this answer | |
  • Thanks for the effort, that solves already most of the entries. Now that I have tested it there are a couple of general cases that may be improved. The first one is when the name contains a special character like ö or a name with a - like john-smith. The second case is when a citations happens outside of parentheses like Lorem ipsum dolor as explained in Sit et al. (1998) which should become Lorem ipsum dolor as explained in \textcite{sit1998}. If anyone wants to try to generalize the script, otherwise I will try to give it a shot myself. – Manfredo Feb 11 at 22:10
  • 1
    You're welcome. I have adapted the program to accept names with hyphens and in general provide a more robust test of author names. The special characters are an issue because their handling in regular expressions is very locale-dependent (even the ordering, hence my caveat about the C locale). It may be necessary to provide for each possible non-ASCII alphabetical character by adding them explicitly to the [[:alpha:].+] character list, unless anyone knowledgeable has a better idea. As for the issue of "inline" citations ... that's a challenge! – AdminBee Feb 12 at 9:04
  • On my test hyphenated names do not work yet. I have a citation which is (Nabe-Nielsen 2001) and it is not picked up. – Manfredo Feb 12 at 14:49
  • Interesting. Initially I thought it was because in this case, the "single-source" citation did not conform to the comma-separated (author, year) format, but I tested it with exactly that string (using gawk, mawk and nawk) and it gets replaced correctly. I can currently not explain that behaviour. Can you tell me which awk variant and version you are using? – AdminBee Feb 13 at 8:27
  • awk version 20070501 for MacOS – Manfredo Feb 13 at 9:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.