I am trying to find out which files are in machineB and which files are in machineC.

And I need to run my shell script on machineA to figure out which files are in machineB and which files are in machineC

I have around 300 files distributed in machineB and machineC.

Now I would like to find out which file is in which machine (machineB or machineC).

Below is my shell script which I am running on machineA and PRIMARY_PARTITION has some file numbers and SECONDARY_PARTITION has some file numbers. And the files are present in this directory -


in machineB or machineC .. And the file path is like this -


Here the file number is 0, 3, 5, 7, 9. Other names in the filename are always going to be same apart from the file numbers.


readonly FILERS_LOCATION=(machineB machineC)
readonly MEMORY_MAPPED_LOCATION=/data/snapshot


echo $dir1
echo $dir2

if [ "$dir1" = "$dir2" ]
    for el in "${PRIMARY_PARTITION[@]}"
        # do something here may be?
    for sl in "${SECONDARY_PARTITION[@]}"
        # do something here may be?

Now my question is how would I find out all the files in machineB and machineC. Meaning out of (0 3 5 7 9) and (1 2 4 6 8) which files are in machineB and which are in machineC.

As an example I have shown very small set of files above. In general, I have lot of files which I am thinking its pretty difficult for me to login in machineB and machineC to find out the files.

  • 1
    Why not rsync?.
    – Braiam
    Mar 20, 2014 at 1:30
  • @Braiam: I haven't worked with rsync before so there will be some learning curve for me. I would like to learn it but given the time frame I have, I thought its best to do using ssh.
    – SSH
    Mar 20, 2014 at 1:34
  • 1
    rsync can work over ssh just fine and seems easy too. Now, depending of what your final goal is rsync can in a single command do it (which may not be helpful if you are doing an assignment)
    – Braiam
    Mar 20, 2014 at 1:39
  • @Braiam: It's not an assignment. I need to do this on our production system. I can try using rsync for this. Can you provide an example if possible? And do I need to install something on our production servers on machineA? bcoz from machineA I will be running my shell script.
    – SSH
    Mar 20, 2014 at 1:40
  • 1
    In that case, please edit your question, the opening paragraph is very misleading, you state that your problem is that you need to copy files to machineA but don't know if they are on C or B. Are you trying to figure out which file came from which after having successfully copied them (which begs the question of why not before)? How do you connect to the machines? Can you simply do ssh user@machineB ls /path and collect the files that way?
    – terdon
    Mar 20, 2014 at 2:37

2 Answers 2


There are 2 approaches that I can think of off the top of my head. The first would involve using rsync the second would be to use a combination of ssh & diff.

NOTE: Both of these approaches would compare a directory on machineA that has all the files with a subset of these files on machine's B and C showing you which machine (B or C) was the originator of said files.

Example data

Say I had the following setup of files on machineA.

$ ls
t1_1980_10_200003_5.data  t1_1980_4_200003_5.data  t1_1980_8_200003_5.data
t1_1980_1_200003_5.data   t1_1980_5_200003_5.data  t1_1980_9_200003_5.data
t1_1980_2_200003_5.data   t1_1980_6_200003_5.data
t1_1980_3_200003_5.data   t1_1980_7_200003_5.data

Method #1 - rsync

Now let's find out what files in this directory came from machineB.

$ rsync --dry-run -avz machineB:~/20140317/ .
receiving incremental file list

sent 29 bytes  received 165 bytes  129.33 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

While machineC:

$ rsync --dry-run -avz machineC:~/20140317/ .
receiving incremental file list

sent 29 bytes  received 166 bytes  390.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

Method #2 - ssh + diff

As an alternative you can use some semi-basic ssh and diff to do something similar.

$ cd /dir/with/machBC
$ diff -B <(ssh user@machineB "ls -R ~/20140317/") <(ls -R .)
> .:
> t1_1980_10_200003_5.data
> t1_1980_6_200003_5.data
> t1_1980_7_200003_5.data
> t1_1980_8_200003_5.data
> t1_1980_9_200003_5.data
  • What is /dir1/ here in your example for rsync?
    – SSH
    Mar 20, 2014 at 2:31
  • I am confuse what is /dir1/ here? This folder 20140317 is in machineB and machineC not machineA. And the files are inside this folder 20140317.
    – SSH
    Mar 20, 2014 at 4:46
  • @SSH - it was just as directory that I was using for my simulation of your problem, I've replaced it with your 20140317.
    – slm
    Mar 20, 2014 at 5:23
  • Sure.. In your ssh + diff example. I would be running that diff command on machineA where the files are located right? If yes, then what is this path cd /dir/with/machBC?
    – SSH
    Mar 20, 2014 at 6:34
  • @SSH = correct, on machineA. That's a path to denote that this is some location on machineA with the files from machine's B & C.
    – slm
    Mar 20, 2014 at 12:09

If I understand your question correctly (big if), I would do it with something like this running on machineA:

#!/usr/bin/env bash

## Save the files from each machine into bash array
files_on_B=$(ssh user@machineB find "$target_dir" -name "$file_pattern")
files_on_C=$(ssh user@machineC find "$target_dir" -name "$file_pattern")

## Do whatever you like with them, here I am simply printing
echo "Files from B: ${files_on_B[@]}"
echo "Files from C: ${files_on_C[@]}"

Note that the script assumes sane file names, like in your example, with no spaces, new lines or other strange characters.

  • Thanks terdon. I am suppose to run this script from machineA where the files are located.. Right?
    – SSH
    Mar 20, 2014 at 21:22
  • @SSH yes, from machineA.
    – terdon
    Mar 20, 2014 at 22:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .