3

I have a file of filenames for files in a certain directory. However, some of the file names may:

  1. Have spaces in the original filename replaced with underscores (so directory/file with spaces becomes file_with_spaces in the input file)
  2. May not actually match a file in the directory

If I didn't have these two conditions, I'd use cat inputfile | awk 'commands' to process apply the commands I want on the file. However, I would like for some way to catch filename not found errors and either:

  1. Try with different combinations of underscores being replaced with spaces until it finds a matching file
  2. Provide a list of files that had no matches, even after substituting spaces for underscores.

Is there a good way to do this? I suspect that some type of script will be needed (rather than a one-line command), but I am not yet familiar with shell scripting enough to think of the solution.

5
  • To paraphrase, you have a file inputfile which is a list of filenames, one per line. However these names might not be exact, in particular filenames with spaces in them might be listed with underscores, but maybe not every space has been changed to underscore. You want to get the real names of the files that are listed, and also any values which are in the inputfile but doesn't have a matching real file. Is this a fair summary?
    – icarus
    Jan 15, 2017 at 0:22
  • @icarus, that is mostly correct. Each filename is on its own line in inputfile. All spaces in the original filenames have been converted to underscores, but not every underscore was originally a space. I want the real filenames so I can run a different command over them and the subset of non-matching names for manual investigation.
    – cjm
    Jan 15, 2017 at 2:48
  • @icarus: at least when I typed it out like that, it sounds like the best solution is probably to generate two files: one for actualnames and one for no-matches, then run my command over actualnames and use no-matches as my to-do list. What I need the help with would be turning inputfile into those two lists.
    – cjm
    Jan 15, 2017 at 2:50
  • Great. What happens if you have 2 files "a_b" and "a b", and you have "a_b" in the input file? What happens if you have "a_b_c" in the input and files "a b_c" and "a_b c"? Can we assume this never happens?
    – icarus
    Jan 15, 2017 at 2:51
  • There should not be any files named a_b and a b. If there were, it should match both, but if it makes things simpler, there aren't any files with names that are similar in that manner.
    – cjm
    Jan 15, 2017 at 3:03

3 Answers 3

0

The approach I would use is to take the output of ls, set up an array mapping the converted name back to the original name, then process each line of the input file. If the input is in the array then output the value of the array, otherwise add the input line to the missing file. So something like put this in a file, change to the directory and run it with inputfile as a parameter

#!/usr/bin/awk
# set up an array t of translations
BEGIN {
    while (("ls" | getline )>0) {
        k=$0
        gsub(/ /,"_")
        if ($0 in t) {
            print "$0 matches more than one file" > /dev/stderr
            exit(2)
        }
        t[$0]=k
    }
    close("ls")
}
    { if ($0 in t) {
            print t[$0]
        } else {
            print $0 > "../missing"
        }
    }
7
  • I saved this as translation.sh, but when I run it, I am greeted with /usr/bin/awk: syntax error at source line 1 context is >>> /path/to/there/translation. <<< sh.
    – cjm
    Jan 15, 2017 at 5:19
  • It is at times like this I feel I need to get a mac. What happens if you say awk -f translation.sh inputfile ?
    – icarus
    Jan 15, 2017 at 6:06
  • It seems that it doesn't like how I'm sending things to /dev/stderr. Output is: awk: syntax error at source line 8… context is print "$0 matches more than one file" > >>> /dev/stderr <<< awk: illegal statement at source line 9 source file ~/translation.sh
    – cjm
    Jan 15, 2017 at 6:18
  • OK, let's just remove that line as it is essentially just error reporting.
    – icarus
    Jan 15, 2017 at 6:25
  • Removing that line worked wonderfully, and there was only one file that did not have a match (I was expecting closer to 10, but I trust that it was one).
    – cjm
    Jan 15, 2017 at 6:38
0

Transform the munged file names into patterns that match the original file.

#!/bin/bash
shopt -s nullglob extglob
IFS=$'\n'
while read -r filename; do
  pattern=${filename//\\/\\\\}
  pattern=${pattern//\[/\\\[}
  pattern=${pattern//\(/\\\(}
  pattern=${pattern//\*/\\\*}
  pattern=${pattern//\?/\\\?}
  pattern=${pattern//_/'[ _]'}
  matches=($pattern@())
  case ${#matches[@]} in
    0) echo "No match for $filename";;
    1) echo "$filename found as ${matches[0]}";;
    *) echo "$filename matches ${#matches[@]} files";;
  esac
done <inputfile
2
  • It seems to always give matching files as matching u+1 filenames, where u is the number of underscores in the inputfile entry
    – cjm
    Jan 16, 2017 at 5:01
  • @cjm I forgot to remove spaces from IFS. I also forgot to arrange to arrange for names that don't contain any underscore to be looked up as files. See my updated answer. Jan 16, 2017 at 10:55
0

With zsh, you could use its approximate matching capability:

approx-cat() {
  emulate -L zsh
  setopt extendedglob nullglob
  local err files
  for ((err = 0; err <= $1; err++)); do
    files=((#a$err)$2)
    case $#files in
      (1) cat -- $files; return;;
      (0) ;;
      (*) echo >&2 "$#files found at error count $err:"
          printf >&2 '  "%s"\n' $files
          return 1;;
    esac
  done
  return 1
}

And call as:

approx-cat 3 'directory/file with spaces'

To allow up to 3 errors in the file name.

Example:

$ approx-cat 3 /ebc/passwds
2 found at error count 2:
  "/etc/passwd"
  "/etc/passwd-"
$ approx-cat 3 /ebc/Issue
Debian GNU/Linux stretch/sid \n \l

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.