How to cd into a directory as part of shell script and do a word count of files?

Question

I am trying to get the latest folder inside my directory in machineB and machineC. So I came up with the below command which will find me the latest folder inside -

`/bat/data/snapshot`

And that latest folder will be in the form of YYYYMMDD. Below is my shell script -

#!/bin/bash

readonly FILERS_LOCATION=(machineB machineC)
readonly MAPPED_LOCATION=/bat/data/snapshot

dir1=$(ssh -o "StrictHostKeyChecking no" david@${FILERS_LOCATION[0]} ls -dt1 "$MAPPED_LOCATION"/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] | head -n1)
dir2=$(ssh -o "StrictHostKeyChecking no" david@${FILERS_LOCATION[1]} ls -dt1 "$MAPPED_LOCATION"/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] | head -n1)

echo $dir1
echo $dir2

Now dir1 will look like in machineB as -

/bat/data/snapshot/20131222

And dir2 will look like in machineC as -

/bat/data/snapshot/20131222

Now I need to cd into these two directory and do a word count of files and store them in length1 and length2 variable -

ls | wc -l

I thought I can do this but I thought before that I need to cd into those two directory and then do a word count -

echo $dir1
echo $dir2

length1=$(ssh -o "StrictHostKeyChecking no" david@${FILERS_LOCATION[0]} ls | wc -l "$dir1")
length2=$(ssh -o "StrictHostKeyChecking no" david@${FILERS_LOCATION[1]} ls | wc -l "$dir2")

echo $length1
echo $length2   

So now I am not sure how to do cd into those two directory and then do a ls and find the count of files as part of that shell script?

Answer 1

Your best bet is going to be to write a script rather than trying to pass a big ugly command via ssh arguments. It becomes very difficult to execute complex commands via ssh arguments because ssh joins all the arguments into a space separated string, and then re-splits them on the remote end.

Now, using a script isn't as bad as it sounds, and is actually quite simple since you can pass the script over STDIN.

function host_file_count() {
 ssh -o StrictHostKeyChecking=no david@"$1" bash -s <<'EOI'
  dir_last="$(ls -dt1 /bat/data/snapshot/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] | head -n 1)"
  count="$(ls -1 "$dir_last"/ | wc -l)"
  echo "$(hostname): $dir_last=$count"
EOI
}
host_file_count machineB
host_file_count machineC

Basically this just feeds the script into the STDIN of ssh which in turn passes to STDIN of bash -s on the remote host.
The other advantage of doing it this way is that you don't have to perform multiple ssh commands to get the result.

---

Note that many will tell you not to parse ls. So below is another script which does the same but without using ls.

function host_file_count() {
 ssh -o StrictHostKeyChecking=no david@"$1" bash -s <<'EOI'
  dirs=( /bat/data/snapshot/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] ) # get a list of matching directories
  dir_last="${dirs[${#dirs[@]} - 1]}" # get the last item in the list
  files=( "$dir_last"/* ) # get the list of files in directory
  count="${#files[@]}" # size of that list
  echo "$(hostname): $dir_last=$count"
EOI
}
host_file_count machineB
host_file_count machineC

---

note: both of these are completely untested, there may be minor errors, but the principle stands

Answer 2

Your local shell in interpreting the pipe, plus you've got the variable on the wrong command:

len1=$( ssh ... user@host sh -c "ls '$dir1' | wc -l" )