Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a backup server which takes backup of other servers and in few days back the backup server started experiencing error and the not able to take back. I think the error log bellow could describe the problem better than me:

2012-09-10 20:05:40 Aborting backup up after signal PIPE
2012-09-10 20:05:44 Got fatal error during xfer (aborted by signal=PIPE)
2012-09-10 21:00:15 full backup started for directory rootBackup (baseline backup #277)

2012-09-11 10:21:07 Aborting backup up after signal PIPE
2012-09-11 10:21:11 Got fatal error during xfer (aborted by signal=PIPE)
2012-09-11 11:00:24 full backup started for directory rootBackup (baseline backup #277)
2012-09-11 13:22:41 Aborting backup up after signal PIPE
2012-09-11 13:22:52 Got fatal error during xfer (aborted by signal=PIPE)
2012-09-11 14:00:30 full backup started for directory rootBackup (baseline backup #277)for directory rootBackup
2012-09-14 06:38:02 Got fatal error during xfer (Child exited prematurely)
2012-09-14 06:38:08 Backup aborted (Child exited prematurely)
2012-09-14 07:00:27 incr backup started back to 2012-09-11 14:00:30 (backup #278) for directory rootBackup
2012-09-20 14:22:04 Got fatal error during xfer (Child exited prematurely)
2012-09-20 14:22:10 Backup aborted (Child exited prematurely)
2012-09-20 15:00:45 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-20 15:26:29 Got fatal error during xfer (Child exited prematurely)
2012-09-20 15:26:35 Backup aborted (Child exited prematurely)
2012-09-20 16:00:12 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-20 16:27:37 Got fatal error during xfer (Child exited prematurely)
2012-09-20 16:27:43 Backup aborted (Child exited prematurely)
2012-09-20 17:00:09 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-20 17:27:37 Got fatal error during xfer (Child exited prematurely)
2012-09-20 17:27:43 Backup aborted (Child exited prematurely)
2012-09-20 18:00:20 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-20 18:27:49 Got fatal error during xfer (Child exited prematurely)
2012-09-20 18:27:55 Backup aborted (Child exited prematurely)
2012-09-20 19:00:26 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-20 19:28:29 Got fatal error during xfer (Child exited prematurely)
2012-09-20 19:28:36 Backup aborted (Child exited prematurely)
2012-09-20 20:00:32 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-20 20:23:06 Got fatal error during xfer (Child exited prematurely)
2012-09-20 20:23:11 Backup aborted (Child exited prematurely)
2012-09-20 21:00:16 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-20 21:20:30 Got fatal error during xfer (Child exited prematurely)
2012-09-20 21:20:37 Backup aborted (Child exited prematurely)
2012-09-20 22:00:15 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-20 22:15:21 Got fatal error during xfer (Child exited prematurely)
2012-09-20 22:15:26 Backup aborted (Child exited prematurely)
2012-09-20 23:00:21 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-20 23:15:06 Got fatal error during xfer (Child exited prematurely)
2012-09-20 23:15:11 Backup aborted (Child exited prematurely)
2012-09-21 01:00:09 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-21 01:27:47 Got fatal error during xfer (Child exited prematurely)
2012-09-21 01:27:53 Backup aborted (Child exited prematurely)
2012-09-21 02:00:20 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-21 02:34:06 Got fatal error during xfer (Child exited prematurely)
2012-09-21 02:34:13 Backup aborted (Child exited prematurely)
2012-09-21 03:00:21 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-21 03:26:13 Got fatal error during xfer (Child exited prematurely)
2012-09-21 03:26:19 Backup aborted (Child exited prematurely)
2012-09-21 04:00:14 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-21 04:17:05 Got fatal error during xfer (Child exited prematurely)
2012-09-21 04:17:11 Backup aborted (Child exited prematurely)
2012-09-21 05:00:09 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-21 05:27:00 Aborting backup up after signal PIPE
2012-09-21 05:27:02 Got fatal error during xfer (aborted by signal=PIPE)
2012-09-21 06:00:04 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-21 06:19:01 Aborting backup up after signal PIPE
2012-09-21 06:19:02 Got fatal error during xfer (aborted by signal=PIPE)
2012-09-21 07:00:01 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-21 07:11:57 incr backup 283 complete, 138 files, 282332868 bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)
2012-09-22 07:00:09 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 07:11:31 Got fatal error during xfer (Child exited prematurely)
2012-09-22 07:11:36 Backup aborted (Child exited prematurely)
2012-09-22 08:00:07 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 08:16:04 Got fatal error during xfer (Child exited prematurely)
2012-09-22 08:16:10 Backup aborted (Child exited prematurely)
2012-09-22 09:00:05 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 09:25:18 Got fatal error during xfer (Child exited prematurely)
2012-09-22 09:25:26 Backup aborted (Child exited prematurely)
2012-09-22 10:00:29 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 10:26:23 Got fatal error during xfer (Child exited prematurely)
2012-09-22 10:26:28 Backup aborted (Child exited prematurely)
2012-09-22 11:00:13 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 11:23:26 Got fatal error during xfer (Child exited prematurely)
2012-09-22 11:23:32 Backup aborted (Child exited prematurely)
2012-09-22 12:00:13 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 12:16:27 Got fatal error during xfer (Child exited prematurely)
2012-09-22 12:16:33 Backup aborted (Child exited prematurely)
2012-09-22 13:00:11 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 13:25:22 Got fatal error during xfer (Child exited prematurely)
2012-09-22 13:25:27 Backup aborted (Child exited prematurely)
2012-09-22 14:00:33 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 14:26:53 Got fatal error during xfer (Child exited prematurely)
2012-09-22 14:26:58 Backup aborted (Child exited prematurely)
2012-09-22 15:00:10 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 15:22:22 Got fatal error during xfer (Child exited prematurely)
2012-09-22 15:22:30 Backup aborted (Child exited prematurely)
2012-09-22 16:00:38 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 16:25:32 Got fatal error during xfer (Child exited prematurely)
2012-09-22 16:25:38 Backup aborted (Child exited prematurely)
2012-09-22 17:00:11 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 17:24:36 Got fatal error during xfer (Child exited prematurely)
2012-09-22 17:24:41 Backup aborted (Child exited prematurely)
2012-09-22 18:00:21 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 18:24:42 Got fatal error during xfer (Child exited prematurely)
2012-09-22 18:24:50 Backup aborted (Child exited prematurely)
2012-09-22 19:00:20 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 19:24:47 Got fatal error during xfer (Child exited prematurely)
2012-09-22 19:24:53 Backup aborted (Child exited prematurely)
2012-09-22 20:00:20 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 20:23:21 Got fatal error during xfer (Child exited prematurely)
2012-09-22 20:23:26 Backup aborted (Child exited prematurely)
2012-09-22 21:00:14 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 21:43:12 Aborting backup up after signal PIPE
2012-09-22 21:43:13 Got fatal error during xfer (aborted by signal=PIPE)
2012-09-22 22:00:10 incr backup started back to 2012-09-19 05:00:02 (backup #282) for directory rootBackup
2012-09-22 22:14:58 Got fatal error during xfer (Child exited prematurely)
2012-09-22 22:15:04 Backup aborted (Child exited prematurely)
2012-09-22 23:00:15 incr backup started back to 2012-09-19 05:00:02 (backup 

Note: I am using backuppc for backup. Backup server is able to take backup for other servers but for one server its experiencing error. So I believe there must be some problem in that particular client server so that its prematurely exiting .

Update:

Network seams to be OK(netstat -in) on client side.

Kernel Interface table
Iface   MTU Met   RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0       1500 0  1297053361      0 126811 0      2321209324      0      0      0 BMRU
lo        16436 0  35169163      0      0 0      35169163      0      0      0 LRU

The bellow is the contents of: file /var/lib/backuppc/pc/ns381613.ovh.net-daily/XferLOG.bad.z, modified 2012-11-21 17:25:14 (Extracting only Errors)

full backup started for directory www (baseline backup #300)
Connected to ns381613.ovh.net:873, remote version 30
Negotiated protocol version 28
Connected to module www
Sending args: --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive --ignore-times . .
Xfer PIDs are now 4845
[ skipped 120129 lines ]
Remote[2]: file has vanished: "/mywebsite.com/sessions/sess_017p907qdbm3rn5vv8gdin2ab7" (in www)
Remote[2]: file has vanished: "/mywebsite.com/sessions    Remote[2]: file has vanished: "/mywebsite.com/sessions/sess_vkn369qbj5dhoj4no0bei1sen3" (in www)
Remote[2]: file has vanished: "/mywebsite.com/sessions/sess_vkn7demudpv0othe6e98s2v1v4" (in www)
Remote[2]: file has vanished: "/mywebsite.com/sessions/sess_voseo6s018c8tmocgthj87irj1" (in www)
Remote[2]: file has vanished: "/mywebsite.com/sessions/sess_vqpqhv16urbrh99acecmujj8j1" (in www)
Remote[2]: file has vanished: "/mywebsite.com/sessions/sess_vs0j1bsfina3f4913lorb7j681" (in www)
Remote[2]: file has vanished: "/mywebsite.com/sessions/sess_vu3jupeug0qpls9f6ikub544t0" (in www)
Remote[2]: file has vanished: "/mywebsite.com/sessions/sess_vu3pp00ko7uip62jf7vp8o8la1" (in www)
[ skipped 55244 lines ]
Read EOF: Connection reset by peer
Can't write 32780 bytes to socket
Tried again: got 0 bytes
finish: removing in-process file mywebsite.com/www-bk-17-oct-2012/lms/archive/LAWYER_1350329888/document/screencasts/TechtutorTv.flv
Child is aborting
Done: 160949 files, 2244371209 bytes
Got fatal error during xfer (aborted by signal=PIPE)
Backup aborted by user signal
Not saving this as a partial backup since it has fewer files than the prior one (got 160949 and 160949 files versus 5985910)

Then I tried:

sudo -u backuppc /usr/share/backuppc/bin/BackupPC_dump -v -f ns381613.ovh.net

Output

Sending args: --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive --ignore-times . .

Xfer PIDs are now 22310
xferPids 22310
Remote[1]: rsync error: timeout in data send/receive (code 30) at io.c(137) [sender=3.0.7]
Read EOF: 
Tried again: got 0 bytes
Child is aborting
Parent read EOF from child: fatal error!
Done: 0 files, 0 bytes
Got fatal error during xfer (Child exited prematurely)
cmdSystemOrEval: about to system /bin/ping -c 1 ns381613.ovh.net
cmdSystemOrEval: finished: got output PING ns381613.ovh.net (188.165.247.43) 56(84) bytes of data.
64 bytes from ns381613.ovh.net (188.165.247.43): icmp_req=1 ttl=60 time=0.591 ms

--- ns381613.ovh.net ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.591/0.591/0.591/0.000 ms

cmdSystemOrEval: about to system /bin/ping -c 1 ns381613.ovh.net
cmdSystemOrEval: finished: got output PING ns381613.ovh.net (188.165.247.43) 56(84) bytes of data.
64 bytes from ns381613.ovh.net (188.165.247.43): icmp_req=1 ttl=60 time=0.366 ms

--- ns381613.ovh.net ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.366/0.366/0.366/0.000 ms

CheckHostAlive: returning 0.366
Backup aborted (Child exited prematurely)
Not saving this as a partial backup since it has fewer files than the prior one (got 0 and 0 files versus 1)
dump failed: Child exited prematurely

In the above output I see it says timeout, But I have increased the timeout variable in configuration.

share|improve this question
    
Have you checked the client? –  Karlson Sep 27 '12 at 15:14
    
No, But the backup server was able to take backup before of client, so I do not thing there would be any configuration problem in client. –  Subhransu Sep 27 '12 at 18:17
    
The standard disclaimer by every lawyer "Past performance doesn't guarantee future results". Applied here this doesn't mean that other factors didn't change (added client reboot for example), so have you checked what is happening at the client when backup server errors? –  Karlson Sep 27 '12 at 19:39
    
Ok, I am currently checking the clients configuration. But how to check the client at the time of error, is there any error log also available at client ? –  Subhransu Sep 28 '12 at 7:21
    
Have you tried increasing the log level? Have you looked at the usual suspects (/var/log/messages, /var/log/daemon.log). Have you looked at this page? –  Karlson Sep 28 '12 at 12:24
show 3 more comments

1 Answer 1

just a shot in the dark, but did the number of files in a backed up directory on the client end, increased significantly around the time this problem started exhibiting itself ? The message

(aborted by signal=PIPE)

makes me think of a Unix pipe. In the backup nomenclature, this pipe maybe a pipe of file names. And when the number of files exceed a certain buffer size (unable elaborate on that without knowing the system architectures of both the server and the client), you may encounter errors as such. Since this error is coming from a backup application that I am not familiar with, I can not attest to correlation between the message and the number of files, but it doesn't hurt to check.

How you can get around it, if it is the case that is, is totally up to you. Trim the directory by eliminating older files or create sub-directories and push certain number of files into these directories. Or a combination of the two.

Good luck

share|improve this answer
    
Hi! Thanks for your answer, but could you please check the updated question to help me in finding solution ? –  Subhransu Dec 3 '12 at 9:52
    
did you pay attention to "Remote[1]: rsync error: timeout in data send/receive (code 30) at io.c(137) [sender=3.0.7]" line in your output section ? obviously this backup application is using rsync to reach out to the remote machines and this ns381613.ovh.net machine is not connecting ? Again, without knowing your network topology, it is hard to guess why it is not connecting, but a firewall rule change comes to mind. Try to establish an rsync connection, outside this backup program and see if it is working on its own properly. My guess, it is not. –  MelBurslan Dec 5 '12 at 21:40
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.