Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am trying to recursively copy a huge directory of installables over 25G and want to preserve the owner and group id, permission modes and so on.

So I ran the below commands:

bash-3.2$ date; cp -rpP /source/path/Oracle /target/path/Oracle;date;
Tue Jun  4 09:44:54 EDT 2013

...

Some observations:

  • Today (Wed Jun 5 12:09:25 EDT 2013) I notice that it still hasn't completed and appears to be stuck somewhere. I started analyzing the process and it seems to be sleeping:

    bash-3.2$ truss -p 09431;
    read(10, 0xFFBEF288, 32768)     (sleeping...)
    
  • The last entry also shows that its asleep.

    bash-3.2$ pflags 09431
    9431:   cp -rpP /source/path/Oracle /target/path/Oracle
            data model = _ILP32  flags = RLC|ASYNC|MSACCT|MSFORK
            flttrace = 0xfffffbff
            sigtrace = 0xfffffeff 0xffffffff
            HUP|INT|QUIT|ILL|TRAP|ABRT|EMT|FPE|BUS|SEGV|SYS|PIPE|ALRM|TERM|USR1|USR2|CLD|PWR|WINCH|URG|POLL|STOP|TSTP|CONT|TTIN|TTOU|VTALRM|PROF|XCPU|XFSZ|WAITING|LWP|FREEZE|THAW|CANCEL|LOST|XRES|JVM1|JVM2|RTMIN|RTMIN+1|RTMIN+2|RTMIN+3|RTMAX-3|RTMAX-2|RTMAX-1|RTMAX
        entryset = 0x00000401 0x04000000 0x00000000 0x00000028
                   0x80000000 0x00000000 0x00000000 0x00000000
        exitset  = 0xfffffffe 0xffffffff 0xffffffff 0xffffffd7
                   0x7fffffff 0xffffffff 0xffffffff 0xffffffff
     /1:    flags = ASLEEP  read(0xa,0xffbef288,0x8000)
    
  • Finally tried pfiles

    bash-3.2$ pfiles -F 09431
    9431:   cp -rpP /source/path/Oracle /target/path/Oracle
    Current rlimit: 256 file descriptors
    0: S_IFCHR mode:0620 dev:363,0 ino:12582922 uid:502012187 gid:7 rdev:24,3
      O_RDWR|O_NOCTTY|O_LARGEFILE
      /devices/pseudo/pts@0:3
    1: S_IFCHR mode:0620 dev:363,0 ino:12582922 uid:502012187 gid:7 rdev:24,3
      O_RDWR|O_NOCTTY|O_LARGEFILE
      /devices/pseudo/pts@0:3
    2: S_IFCHR mode:0620 dev:363,0 ino:12582922 uid:502012187 gid:7 rdev:24,3
      O_RDWR|O_NOCTTY|O_LARGEFILE
      /devices/pseudo/pts@0:3
    3: S_IFDIR mode:0750 dev:377,1 ino:1135681 uid:502012187 gid:502012187 size:4096
      O_RDONLY|O_NDELAY|O_LARGEFILE FD_CLOEXEC
      /source/path/Oracle
    4: S_IFDIR mode:0750 dev:377,1 ino:1135682 uid:502012187 gid:502012187 size:4096
      O_RDONLY|O_NDELAY|O_LARGEFILE FD_CLOEXEC
      /source/path/Oracle/Middleware
    5: S_IFDIR mode:0750 dev:377,1 ino:14640507 uid:502012187 gid:502012187 size:4096
      O_RDONLY|O_NDELAY|O_LARGEFILE FD_CLOEXEC
      /source/path/Oracle/Middleware/user_projects
    6: S_IFDIR mode:0750 dev:377,1 ino:14640508 uid:502012187 gid:502012187 size:4096
      O_RDONLY|O_NDELAY|O_LARGEFILE FD_CLOEXEC
      /source/path/Oracle/Middleware/user_projects/epmsystem2
    7: S_IFDIR mode:0750 dev:377,1 ino:6480147 uid:502012187 gid:502012187 size:4096
      O_RDONLY|O_NDELAY|O_LARGEFILE FD_CLOEXEC
      /source/path/Oracle/Middleware/user_projects/epmsystem2/EssbaseServer
    8: S_IFDIR mode:0750 dev:377,1 ino:6480149 uid:502012187 gid:502012187 size:4096
      O_RDONLY|O_NDELAY|O_LARGEFILE FD_CLOEXEC
      /source/path/Oracle/Middleware/user_projects/epmsystem2/EssbaseServer/essbaseserver1
    9: S_IFDIR mode:0750 dev:377,1 ino:6480151 uid:502012187 gid:502012187 size:4096
      O_RDONLY|O_NDELAY|O_LARGEFILE FD_CLOEXEC
      /source/path/Oracle/Middleware/user_projects/epmsystem2/EssbaseServer/essbaseserver1/bin
    10: S_IFIFO mode:0660 dev:377,1 ino:9088704 uid:502012187 gid:502012187 size:0
      O_RDONLY|O_LARGEFILE
      /source/path/Oracle/Middleware/user_projects/epmsystem2/EssbaseServer/essbaseserver1/bin/ESSBASE1_1
    11: S_IFREG mode:0660 dev:326,22000 ino:128325 uid:502012187 gid:502012187 size:0
      O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE
      /target/path/Oracle/Middleware/user_projects/epmsystem2/EssbaseServer/essbaseserver1/bin/ESSBASE1_1
    

UPDATE: Note that the above cp process has been killed. So below observations are trying to reproduce the issue.

  • File that is getting blocked:

    bash-3.2$ ls -l ESSBASE1_1
    prw-rw----   1 kent   kent         0 Nov 22  2011 ESSBASE1_1
    
  • When I try to cp this file individually:

    bash-3.2$ cp ESSBASE1_1 ESSBASE1_1kent  # PID = 08745
    
    ...
    

    it hangs too.

  • Looking at pflags, it seems to be sleeping.

    bash-3.2$ pflags 08745
    8745:   cp ESSBASE1_1 ESSBASE1_1kent
            data model = _ILP32  flags = MSACCT|MSFORK
    /1:    flags = ASLEEP  read(0x3,0xffbf6bc8,0x8000)
    
  • Searching for all other processes using this file

    bash-3.2$ fuser -f /source/path/Oracle/Middleware/user_projects/epmsystem2/EssbaseServer/essbaseserver1/bin/ESSBASE1_1
    /source/path/Oracle/Middleware/user_projects/epmsystem2/EssbaseServer/essbaseserver1/bin/ESSBASE1_1:     8745o   25057o
    

    Note that the 'o' indicates that the process is using the file as an open file.

  • Finding at the processes:

    bash-3.2$ pstree | grep 8745
     | | |       |--- 09695 kent grep 8745
     | |         \--- 08745 kent cp ESSBASE1_1 ESSBASE1_1kent
    bash-3.2$ pstree | grep 25057
     | | |       \--- 09700 kent grep 25057
     | |   |-+- 25057 kent 86:43 /some/path/Oracle/Middlewa
    

    Process with PID 25057 is a server process.

  • The open file is on a mount-point. It's a NAS disk.

    bash-3.2$ cat /etc/mnttab | grep apps
    server1.com:/vol/server1_nec_nosnap0/dev_apps   /source/path/       nfs     rw,xattr,dev=5e40001    1362894854
    

  • Can anyone help me understand what is causing the copy command to freeze up?
  • I can guess that the application is running and that might be in some way preventing any other process from accessing a particular file.
  • But in such a scenario I would expect cp to abort and throw me an error rather than stay blocked indefinitely.
  • As per the update, it seems process PID=25057 seems to be using the file as an open file. But why does this block the cp command?
share|improve this question
    
Few quesetions but the read it's hanging on is for /source/path/Oracle/Middleware/user_projects/epmsystem2/EssbaseServer/essbasese‌​rver1/bin/ESSBASE1_1 Is that a file or directory? Is it on NFS per chance? Do you know if xfn or other types of automount is configured on this server? What happens if you just try to copy that file directly? –  Joel Davis Jun 6 '13 at 4:10
    
Hi @JoelDavis, I updated the post to answer your questions. The file is on NFS. I know that the mounts are automounted whenever I cd to the directory after a reboot, but I do not know how to check if its configured on Solaris 10. –  Kent Pawar Jun 6 '13 at 14:34
2  
it actually looks like Flup hit the nail on the head, it is definitely a FIFO and if you're trying to copy a FIFO you will be waiting there a while, especially if the other side is waiting on data from CP. –  Joel Davis Jun 6 '13 at 15:10
    
Thanks for confirming it @JoelDavis. I read up on FIFOs and it seems that rsync is the best way to go. –  Kent Pawar Jun 6 '13 at 15:16
add comment

1 Answer

up vote 6 down vote accepted

If you look at your pfiles output for file descriptor 10, you'll notice that the file is a FIFO; this is also evidenced by the p type in your ls listing. The nature of a FIFO is that reads will block unless another process is writing data, and this is why your cp got stuck when trying to read its contents.

To get around the problem, you could use rsync to copy the directory tree instead.

rsync -a /source/path/Oracle /target/path

rsync is clever enough to make a duplicate FIFO rather than read from the original one. Note that you don't specify Oracle for the destination, as rsync will create the directory there.

share|improve this answer
    
Thanks for the explanation Flup! Looking into rsync –  Kent Pawar Jun 6 '13 at 14:38
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.