1

First I want to apologize for my lack of knowledge; this is quite literally my first ever bash script and my Linux knowledge is basically nonexistent. Explanatory answers would be much appreciated; I'd like to learn the why not just the how.

I am trying to write a script to automate a tedious process that has to be done multiple times a day. In short, this script should prepare a set of files based on certain requirements. The script must either move a template file in and assign it a version number, or copy an existing file using a new version number, based on what is found inside of one particular directory (eg create if empty, copy if there's something there).

As an example the file might be named

YYYYMMDD_##_username_testfile.json

where the ## (version number) should be 01 for the first creation, then 02 for the copy, 03 for the copy after that, etc. A single directory could contain upwards of 10+ of these files, all with different dates and sequential version numbers, as well as other files using the same format but ending in something other than testfile.json

I can't seem to find a way to have the script look in the directory "test" for the file ending in testfile.json that has the highest version number, and if found extract the version number as a variable/parameter I can pass back into the cp sequence.

At this point I'm not even sure what command to be using; the internet has offered everything from grep to awk to listing and sorting...my head's spinning!

Any help would be appreciated. I've stripped all identifying info out, please let me know if you need more information.

  • Do you want sequence numbers for a given date, or across all dates? E.g. if 20161025_02_foo_testfile.json and 20161026_01_foo_testfile.json both exist, which one is the highest? – Gilles Oct 25 '16 at 22:32
  • It's probably easier to extract the version numbers and sort those, rather than sort the filenames by version and then extract the version numbers - something like find . -maxdepth 1 -name '*testfile.json' | cut -d_ -f2 | sort -n | tail -n1 maybe? You can modify that to use null terminators if your filenames are allowed to contain newlines. – steeldriver Oct 25 '16 at 22:34
  • @Gilles That example wouldn't happen, the second one would have version 03 instead - it increases based on the version number alone, the date is just there for tracking purposes so we know when the stack's been created. – Alex Oct 26 '16 at 13:24
  • @steeldriver My overall logic was a little faulty (I made a bad assumption about pathing via parameters) and this turned out to be the answer I needed, thank you! – Alex Oct 26 '16 at 15:03
  • @Alex, I wonder if you have considered using a version control system such as Git? – Wildcard Oct 27 '16 at 6:44
1

Lets break it down into a number of steps.

We are going to write a function that will return the next number. We are going to aim for simplicity, rather than maximal efficiency

You need to give it a directory name

You need to give it the testfile.json part of the filename

We need to find all the files in that directory which end with that ending.

Parameters are passed into a function using $1, $2 etc.

getnum(){
  ls "$1" | grep "$2\$"
}

Use cut to pull out just the second field, using _ as the delimiter. (Note if I was going this I would use sed instead of grep and cut, others might use awk, but we are going for simplicity here).

getnum() {
 ls "$1" | grep "$2\$" | cut -d_ -f 2
}

Use sort to sort them numerically

getnum() {
  ls "$1" | grep "$2\$" | cut -d_ -f 2 | sort -n
}

Use head to get just the first value

 getnum() {
  ls "$1" | grep "$2\$" | cut -d_ -f 2 | sort -n | head -1
 }

Wrap this up so we get the value in a local variable

getnum() {
      local oldnum=$(ls "$1" | grep "$2\$" | cut -d_ -f 2 | sort -n | head -1)
}

If there were no values then set the old value to 0. Print out a 2 digit number that is one more that the old one.

 getnum() {
  local oldnum=$(ls "$1" | grep "$2\$" | cut -d_ -f 2 | sort -n | head -1)
  if [ "$oldnum" = "" ]
  then
      oldnum=0
  fi
  printf "%02d\n" $(($oldnum+1))
 }

Note that I could have omitted the test for oldnum not being set, as $((oldnum+1)) would expand to $((+1)) in that case. I can also use expressions like $((${oldnum:-0}+1)), consult the manuals for more information.

You could then use function like this

  dir=somewhere/logs
  ext=testfile.json
  cp template "$dir/$(date +%Y%m%d_)$(getnum "$dir" "$ext")_$USER_$ext"
  • First, thank you so much for answering! I appreciate the breakdown, it makes so much sense. I understand most of what you've said but I'm a little confused what I should be using as parameters for $1 and $2. $1 is the name of the directory, right? In which case that's being called in the actual terminal. But then what is $2? Should it just be the string testfile.json and if so how do I write that in? – Alex Oct 26 '16 at 13:35
  • *meant $s not $2 – Alex Oct 26 '16 at 14:08
  • Turns out I did my own logic wrong, go me. I ended up using your list/grep/cut functionality (and isn't that a marvel, thanks so much for introducing me to piping, how cool!) in a different setting because I was assuming the wrong parameters. Still incredibly helpful, thank you again for commenting! – Alex Oct 26 '16 at 15:31
  • Yes, $1 is intended to be the directory name and $2 is just the string testfile.json. You write it as either testfile.json or "testfile.json", either work as there are no special characters in that string. In the example for usage I set up a variable ext to hold it as I wanted to use it twice, once when looking up existing names and once when creating the new filename. Glad you found the breakdown useful. One of the powerful features that interactive environments have is that you can test out ideas in an incremental manner. – icarus Oct 26 '16 at 22:33
  • The "$s" was a mistake, edited to correct it to $2. Blame editing in the middle of the night in a web browser rather than a real editor. – icarus Oct 27 '16 at 5:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.