1

I have a bunch of files in multiple directories with incorrect dates on the first line of each file. I am trying to write a script involving sed and a for loop.

Each file is in its own directory that is made up of the correct date. For example a file might be in: ./2014/06/02/record1 and I would like to replace the date on the first line to read '2014/06/02'.

There are a number of files each in their own directory. How do I use sed and a for loop to achieve this?

  • 1
    I know there are similar questions but I specifically wanted to focus on the grabbing of part of a directory and using it within a for-loop and inserting into a file. – marzo Jan 27 at 7:10
  • It would be clearer if you could draw a directory tree. – Niko Gambt Jan 27 at 8:37
  • @P_Yadav, yes that is always the format. The first line of every file must match part of the directory it lies in. If a directory is ~/records/2014/07/01 then the first line of the file must read 2014/07/01. – marzo Jan 29 at 12:11
1

To loop over all the files, assuming you are in the parent directory of e.g. 2014 and that the files themselves are called record<something>:

for pathname in 20[0-9][0-9]/[0-9][0-9]/[0-9][0-9]/record*
do
    if [ -f "$pathname" ]; then
        # ...
    fi
done

The pattern 20[0-9][0-9]/[0-9][0-9]/[0-9][0-9]/record* would match all the pathnames that you mention, assuming you only have top-level directories for years 2000 to at most 2099.

The test in the loop is to make sure that the $pathname value is the pathname of an existing regular file (or a symbolic link to one). If the pattern does not match anything, it would by default (in most shells) remain unexpanded. The test would catch this.

To get the directory path for $pathname in the above loop:

dirpath=$( dirname "$pathname" )

or

dirpath=${pathname%/*}

The dirname utility return a string which is the directory path of the given pathname. The variable substitution variation of this removes anything from the last / in $pathname. In this case, either command would generate the same result, but using dirname is generally safer (it would return a given the string a/b/, not a/b which the variable substitution would do).

To replace the first line of the file at $pathname with the string in $dirpath (using GNU sed and in-place editing):

sed -i -e '1c\' -e "$dirpath" "$pathname"

The c command in sed would remove the content of a line completely and insert something else in its place. Here, we apply it to only the first line and insert the generated string $dirname, which will be the date from the pathname.

Bringing this together into a script:

#!/bin/sh

for pathname in 20[0-9][0-9]/[0-9][0-9]/[0-9][0-9]/record*
do
    if [ -f "$pathname" ]; then
        dirpath=${pathname%/*}
        sed -i -e '1c\' -e "$dirpath" "$pathname"
    fi
done

Test it on a backup copy of your files.


An equivalent implementation using find instead (mostly equivalent, it will not process record* files that are symbolic links to regular files):

find 20[0-9][0-9]/ -type f \
    -path '20[0-9][0-9]/[0-9][0-9]/[0-9][0-9]/record*' -exec sh -c '
    for pathname do
        sed -i -e "1c\\" -e "$( dirname "$pathname" )" "$pathname"
    done' sh {} +

This employs basically the same loop as the first variation of the solution.

0

Try this script:

#!/bin/bash

for YEAR in $(ls -1)
do
    echo -n Processing year $YEAR " "
    for MONTH in $(ls -1 $YEAR)
    do
        echo -n month $MONTH " "
        for DAY in $(ls -1 $YEAR/$MONTH)
        do
            echo -n day $DAY
            sed -i "1 s#^.*#$YEAR/$MONTH/$DAY#" $YEAR/$MONTH/$DAY/record1
        done
    done
    echo " "
done

Note: Save the script in a directory above the tree of files and run it like this:

~/tree-of-files $ ../change-dates.sh 
Processing year 2014  month 06  day 02   
Processing year 2017  month 06  day 02 
Processing year 2018  month 06  day 02 
Processing year 2033  month 06  day 02 
~/tree-of-files $ 
0

Here's a slight variation on the existing approaches: loop through the files using wildcard patterns that assume a YYYY/MM/DD directory format; on each of those files, use ed to change the first line to be the corresponding name of the containing directory structure:

for file in [0-9][0-9][0-9][0-9]/[0-9][0-9]/[0-9][0-9]/*
do
  [ -f "$file" ] || continue
  ed -s "$file" <<< $'1c\n'"${file%/*}"$'\n.\nw\nq'
done

The ed command is given as a here-string with the following components:

  • 1c -- change line 1 to ...
  • ${file%/*} -- the parameter expansion of the file variable that results from removing the last forward slash onwards (keeping only the timestamp/directory structure part)
  • . -- end the change command
  • w -- write the file to disk
  • q -- quit ed

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.