Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

This question is a problem derived from solving another problem that you can see on this thread.

To make it short, my dedicated server has a RAID1 array with 2x3TB HDD. A week ago, one of them failed. The company that owns the server has replaced it, so now I have a good drive with all the data, and a new one completely empty.

I am a TOTAL NEWBIE in Linux and hardware-related stuff, so forgive me is the question is very obvious but, I have no idea how to rebuild the RAID from what I have.

This information might be useful (i understand there is no RAID right now):

root@rescue /dev # lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   2.7T  0 disk
sdb      8:16   0   2.7T  0 disk
├─sdb1   8:17   0     1M  0 part
├─sdb2   8:18   0   127M  0 part
├─sdb3   8:19   0   200M  0 part
├─sdb4   8:20   0   2.7T  0 part
└─sdb5   8:21   0 455.5K  0 part
loop0    7:0    0   1.5G  1 loop
root@rescue /dev # cat /proc/mdstat
Personalities : [raid1]
unused devices: <none>

UPDATE 1 Quick info:

   CPU1: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz (Cores 8)
   Memory:  15974 MB
   Disk /dev/sda: 3000 GB (=> 2794 GiB) doesn't contain a valid partition table
   Disk /dev/sdb: 3000 GB (=> 2794 GiB)
   Total capacity 5589 GiB with 2 Disks

UPDATE 2:

As suggested by Trinue:

root@rescue ~ # lspci
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
00:1c.5 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 6 (rev b5)
00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5)
00:1c.7 PCI bridge: Intel Corporation 82801 PCI Bridge (rev b5)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation H67 Express Chipset Family LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller (rev 05)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
03:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)
05:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 01)

UPDATE 3:

As suggested by @Koko, i've tried to mount the 4 partitions, but got errors on 3 of them. May this disk also be broken?

root@rescue / # mount -o ro /dev/sdb1 /mnt/disk
mount: you must specify the filesystem type
root@rescue / # mount -o ro /dev/sdb4 /mnt/disk
ntfs_attr_pread_i: ntfs_pread failed: Input/output error
Failed to calculate free MFT records: Input/output error
NTFS is either inconsistent, or there is a hardware fault, or it's a
SoftRAID/FakeRAID hardware. In the first case run chkdsk /f on Windows
then reboot into Windows twice. The usage of the /f parameter is very
important! If the device is a SoftRAID/FakeRAID then first activate
it and mount a different device under the /dev/mapper/ directory, (e.g.
/dev/mapper/nvidia_eahaabcc1). Please see the 'dmraid' documentation
for more details.
root@rescue / # mount -o ro /dev/sdb2 /mnt/disk
mount: you must specify the filesystem type
root@rescue / # mount -o ro /dev/sdb3 /mnt/disk
root@rescue / # cd /mnt/disk
root@rescue /mnt/disk # dir
EFI

UPDATE 4:

As suggested by Michael Martinez and Koko, i've tried to duplicate data from sdb to sda, with the following errors:

root@rescue /mnt/disk # dd if=/dev/sdb of=/dev/sda
dd: reading `/dev/sdb': Input/output error
6619712+0 records in
6619712+0 records out
3389292544 bytes (3.4 GB) copied, 67.7475 s, 50.0 MB/s

UPDATE 5:

These ara the instructions the owner of the server provide: http://wiki.hetzner.de/index.php/Festplattenaustausch_im_Software-RAID/en for replacing an HDD in one of their servers. However, you will notice that i don't have the raid or the partitions as in the examples they provide.

UPDATE 6:

Hetzner already answered me: "Due to the fact that you haven't ordered an hardware RAID controller, it has a software RAID."

UPDATE 7:

root@rescue / # mount /dev/sda1 /mnt/disk
mount: you must specify the filesystem type
root@rescue / # mount /dev/sda2 /mnt/disk
mount: you must specify the filesystem type
root@rescue / # mount /dev/sda3 /mnt/disk
root@rescue / # mount /dev/sda4 /mnt/disk
mount: you must specify the filesystem type
root@rescue / # mount /dev/sda5 /mnt/disk
mount: you must specify the filesystem type
root@rescue / # cd /mnt/disk
root@rescue /mnt/disk # dir
EFI

UPDATE 8:

I should point out that before running the mount command, i dd sdb into sda and started to create a new array using this commands:

# mdadm --create root --level=1 --raid-devices=2 missing /dev/sdb1  
# mdadm --create swap --level=1 --raid-devices=2 missing /dev/sdb2

root@rescue / # mount
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sys on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=8176304k,nr_inodes=2044076,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
213.133.99.101:/nfs on /root/.oldroot/nfs type nfs (ro,noatime,vers=3,rsize=8192,wsize=8192,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=213.133.99.101,mountvers=3,mountproto=tcp,local_lock=all,addr=213.133.99.101)
aufs on / type aufs (rw,relatime,si=1848aabe5590850f)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=1635764k,mode=755)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=3271520k)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)

UPDATE 9:

When server refused to boot the first time, I asked customer service for a manual reboot. The answer they gave me was:

Dear Client, we have restarted your server, but it seems that there is one hard drive faulty. If you want we can repalce them, for that please confirm us the data loss on this drive and the downtime about 15 minutes. Your server is now in the rescue system:

Y immediately when to the robot website, when I can admin the server and searched for information about the rescue system, and here's what I found:

After activating the rescue system a config file will be created on our DHCP-server. On the next reboot your server will boot from the network and a minimal operating system will be loaded from our TFTP-server. Then you will be able to use the rescue system as long as you want. The order for the rescue system will be active for 60 minutes. If you then reboot your server, the usual system will be started from the hard disk. Please visit our Wiki for further information

Rescue system is a 64bit Debian.

UPDATE 10

root@rescue ~/.oldroot/nfs # ls /root/.oldroot/nfs
bash_aliases                rescue32-wheezy-v006.ext2
check                       rescue32-wheezy-v007.ext2
copy-vnode-lvs-to           rescue32-wheezy-v008.ext2
copy-vnode-lvs-to.bak       rescue32-wheezy-v009.ext2
esxi                        rescue64-lenny-v004.ext2
firmware_update             rescue64-squeeze-v011.ext2
freebsd                     rescue64-squeeze-v012.ext2
functions.sh                rescue64-squeeze-v013.ext2
images                      rescue64-squeeze-v014.ext2
images.old                  rescue64-squeeze-v015.ext2
install                     rescue64-squeeze-v016.ext2
ipmi                        rescue64-test.ext2
iso                         rescue64-wheezy-v000.ext2
knoppix                     rescue64-wheezy-v001.ext2
lost+found                  rescue64-wheezy-v002.ext2
opensolaris                 rescue64-wheezy-v003.ext2
raid_ctrl                   rescue64-wheezy-v004.ext2
README                      rescue64-wheezy-v005.ext2
rescue32-lenny-v004.ext2    rescue64-wheezy-v006.ext2
rescue32-squeeze-v011.ext2  rescue64-wheezy-v007.ext2
rescue32-squeeze-v012.ext2  rescue64-wheezy-v008.ext2
rescue32-squeeze-v013.ext2  rescue64-wheezy-v009.ext2
rescue32-squeeze-v014.ext2  shutdown-h
rescue32-squeeze-v015.ext2  shutdown-h-now
rescue32-squeeze-v016.ext2  tightvnc-vkvm.tar.gz
rescue32-test.ext2          vkvm64-squeeze-v001.ext2
rescue32-wheezy-v000.ext2   vkvm64-squeeze-v002.ext2
rescue32-wheezy-v002.ext2   vkvm64-test.ext2
rescue32-wheezy-v003.ext2   vkvm64-v001.ext2
rescue32-wheezy-v004.ext2   vkvm64-wheezy-overlay.ext2
rescue32-wheezy-v005.ext2   vkvm64-wheezy-overlay-v001.ext2

UPDATE 11:

root@rescue ~ # fdisk -l /dev/sdb

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdb: 3000.6 GB, 3000592982016 bytes
256 heads, 63 sectors/track, 363376 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x8ab49420

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1  4294967295  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.

UPDATE 12:

root@rescue ~ # parted -l
Error: The backup GPT table is corrupt, but the primary appears OK, so that will
be used.
OK/Cancel? OK
Model: ATA ST3000DM001-9YN1 (scsi)
Disk /dev/sda: 3001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt

Number  Start   End     Size    File system  Name                          Flags
 1      17.4kB  1066kB  1049kB               LDM metadata partition
 2      1066kB  134MB   133MB                Microsoft reserved partition  msftres
 3      135MB   345MB   210MB   fat16        EFI system partition          boot
 4      345MB   3001GB  3000GB  ntfs         LDM data partition
 5      3001GB  3001GB  466kB                LDM data partition


Model: ATA ST3000DM001-9YN1 (scsi)
Disk /dev/sdb: 3001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt

Number  Start   End     Size    File system  Name                          Flags
 1      17.4kB  1066kB  1049kB               LDM metadata partition
 2      1066kB  134MB   133MB                Microsoft reserved partition  msftres
 3      135MB   345MB   210MB   fat16        EFI system partition          boot
 4      345MB   3001GB  3000GB  ntfs         LDM data partition
 5      3001GB  3001GB  466kB                LDM data partition


Model: Linux Software RAID Array (md)
Disk /dev/md126: 133MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End    Size   File system     Flags
 1      0.00B  133MB  133MB  linux-swap(v1)


Model: Linux Software RAID Array (md)
Disk /dev/md127: 983kB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End    Size   File system  Flags
 1      0.00B  983kB  983kB  ext4
share|improve this question

migrated from serverfault.com May 2 at 14:37

This question came from our site for professional system and network administrators.

    
I would like to add that, if you aren't too constrained by time, and if you have a spare 3TB disk, you might want to "dd" (or image) your one good drive onto the spare as a backup, in case your efforts to add a second drive to the array result in the original being corrupted. This can easily happen if you accidentally specify the new disk as the "master" disk. –  Michael Martinez Apr 30 at 19:06
    
@MichaelMartinez - Answer in UPDATE 4 –  Dídac Punyet May 1 at 8:13
    
The main problem here seems to be that there is no evidence whatsoever that you have any RAID at all, right now. You may once have had it, but if so, someone's gone to some lengths to remove the evidence. Until you know what sort of RAID you used to have, it's going to be very hard indeed to tell you how to put it back. –  MadHatter May 1 at 8:32
    
Thanks @MadHatter - Already asked customer support about that. UPDATE 6 –  Dídac Punyet May 1 at 8:53
    
This earlier post about the same problem has this important part: "Hetzner.... activated the rescue system (Linux)". I read that as the system is now running from a different instalation or even a different OS. The original disks (one of which should be bootable) will still have the RAID configuration. –  Hennes May 1 at 12:18

2 Answers 2

You say you've spoken to Hetzner, who say it has software RAID. The problem is, it doesn't. Your cat /proc/mdstat says something like

Personalities : [raid1]

unused devices: <none>

Which means no software RAID. If you had a broken software RAID, it would look something like

Personalities : [raid1] 
md0 : active raid1 sdb1[1] (F) sda1[0]
      1020032 blocks [2/2] [_U]

md1 : active raid1 sda2[2] sdb2[1] (F)
      975739772 blocks super 1.1 [2/2] [_U]
      bitmap: 3/8 pages [12KB], 65536KB chunk

unused devices: <none>

I got that from my server, so some of the details would be different for you, and I reconstructed that from notes at the time of the last HDD failure, so some of the details might be wrong, but the important bits are

  1. [_U] (that means one of the mirrors is down; if all was well it would say [UU]), and
  2. (F) (which tells you which half has failed, in this case it's the partition on /dev/sdb in both cases).

Their own instructions, to which you link, show something very similar.

If you saw that, inserting the new HDD into the existing arrays would be a simple matter of partitioning the new disc and using mdadm to add the partitions to the mirrors. But since you currently have an un-RAIDed system, we can't do this.

I would ask Hetzner why it shows no sign of software RAID, with reference to their own documentation. It may be that they had to rip the MD RAID references out in order to get the system to boot at all, but if that's the case, then in doing so they've screwed up any ability you might have had to repair the RAID.

You say you're "a TOTAL NEWBIE in Linux and hardware-related stuff". Honestly, if that's so, then you're in over your head quite badly, here. I would seek professional assistance as soon as I could, and be prepared for some downtime while the system is backed up, then restored in its entirety onto a cleanly-rebuilt-and-RAIDed chassis.

share|improve this answer
    
So, I asked why i don't have any raid, and answered: Dear Client, we don't provide any software support for our dedicated servers and vServers so I'm afraid we cannot help you configure your server. We simply setup the hardware and take care of the network connection. You have full root access so you are responsible for setting up the server. Perhaps you can find more information in our wiki at wiki.hetzner.de/index.php/Hauptseite/en or our forum at forum.hetzner.de . So, I'm afraid there's nothing to do.I could hire someone to fix it,but i guess i won't recover my data. –  Dídac Punyet May 1 at 9:51
    
Isn't your data all there right now? Could you paste the output of mount into your question? –  MadHatter May 1 at 9:52
    
Posted in UPDATE 7 –  Dídac Punyet May 1 at 12:16
    
No, Didac, just the output of the mount command; on its own, without any arguments. –  MadHatter May 1 at 12:22
    
Sorry @MadHatter ! - UPDATE 8 –  Dídac Punyet May 1 at 13:12

You seem to be missing a few lines from cat /proc/mdstat. It would be helpful to know what those are. to add a new disk to an existing software raid array:

mdadm --add RAID_DEV NEW_DISK.

ex:mdadm --add /dev/md0 /dev/sdd

You will need to know your raid device which can sometimes be found in dmesg output:

# dmesg | grep md [7563017.386737] md: data-check of RAID array md0 [7563017.386747] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.

If the array is active/degraded, adding the new disk will automatically start the rebuild process (which can be monitored with /proc/mdstat). Depending on your linux distribution, you may also have a /etc/mdadm/mdadm.conf file which may reveal more information about the array.

share|improve this answer
    
Thanks for your answer @Koko . The thing is that no lines are missing: I think the RAID does not exist :S The hole output looks like this: root@rescue /dev # cat /proc/mdstat Personalities : [raid1] unused devices: <none> root@rescue /dev # dmesg | grep md [Wed Apr 30 16:39:25 2014] md: raid1 personality registered for level 1 –  Dídac Punyet Apr 30 at 15:51
1  
Are you sure that you had software raid configured on that server? It seems more a hardware raid issue to me. –  Trinue Apr 30 at 16:13
    
Dídac - Trinue makes a good point. Issuing "cat /proc/mdstat" on a system with no configured software raid comes back with only those lines. what do you get when you issue the "lspci" command? You may see a raid controller listed there. –  Koko Apr 30 at 17:05
    
Good point indeed @Trinue I just assumed it was a software raid but I never checked it (didn't know how). Updated the question to show the output of lspci –  Dídac Punyet Apr 30 at 17:25
    
There is no RAID controller right? Maybe tryinbg differents things I deleted the software raid... is this possible? It's possible to fix? –  Dídac Punyet Apr 30 at 17:38

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.