0

ceph-mon fails to start on rasberry pi (raspbian 8.0)

I recently purchased 3 raspberry pi nodes to create a small storage cluster to test with at my home. I found a couple of procedures on setting this up so it appears folks have successfully done this!

I am running Raspbian GNU/Linux 8.0 (jessie). I'm using ceph-deploy to install the cluster and it appears to install version 10.2.5-7.2+rpi1 of the ceph ARM packages.

When I try to start the ceph-mon service I get the following error from systemd:

Dec 14 19:59:46 ceph-master systemd[1]: Starting Ceph cluster monitor daemon...
Dec 14 19:59:46 ceph-master systemd[1]: Started Ceph cluster monitor daemon.
Dec 14 19:59:47 ceph-master ceph-mon[28237]: *** Caught signal (Segmentation fault) **
Dec 14 19:59:47 ceph-master ceph-mon[28237]: in thread 756a5c30 thread_name:admin_socket
Dec 14 19:59:47 ceph-master systemd[1]: ceph-mon@ceph-master.service: main process exited, code=killed, status=11/SEGV
Dec 14 19:59:47 ceph-master systemd[1]: Unit ceph-mon@ceph-master.service entered failed state.
Dec 14 19:59:47 ceph-master systemd[1]: ceph-mon@ceph-master.service holdoff time over, scheduling restart.
Dec 14 19:59:47 ceph-master systemd[1]: Stopping Ceph cluster monitor daemon...
Dec 14 19:59:47 ceph-master systemd[1]: Starting Ceph cluster monitor daemon...
Dec 14 19:59:47 ceph-master systemd[1]: Started Ceph cluster monitor daemon.
Dec 14 19:59:49 ceph-master ceph-mon[28256]: *** Caught signal (Segmentation fault) **
Dec 14 19:59:49 ceph-master ceph-mon[28256]: in thread 75654c30 thread_name:admin_socket
Dec 14 19:59:49 ceph-master ceph-mon[28256]: ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367)
Dec 14 19:59:49 ceph-master ceph-mon[28256]: 1: (()+0x4b1348) [0x54fae348]
Dec 14 19:59:49 ceph-master ceph-mon[28256]: 2: (__default_sa_restorer()+0) [0x768bb480]
Dec 14 19:59:49 ceph-master ceph-mon[28256]: 3: (AdminSocket::do_accept()+0x28) [0x550ca154]
Dec 14 19:59:49 ceph-master ceph-mon[28256]: 4: (AdminSocket::entry()+0x22c) [0x550cc458]
Dec 14 19:59:49 ceph-master systemd[1]: ceph-mon@ceph-master.service: main process exited, code=killed, status=11/SEGV
Dec 14 19:59:49 ceph-master systemd[1]: Unit ceph-mon@ceph-master.service entered failed state.
Dec 14 19:59:49 ceph-master systemd[1]: ceph-mon@ceph-master.service holdoff time over, scheduling restart.
Dec 14 19:59:49 ceph-master systemd[1]: Stopping Ceph cluster monitor daemon...
Dec 14 19:59:49 ceph-master systemd[1]: Starting Ceph cluster monitor daemon...
Dec 14 19:59:49 ceph-master systemd[1]: Started Ceph cluster monitor daemon.
Dec 14 19:59:50 ceph-master ceph-mon[28271]: *** Caught signal (Segmentation fault) **
Dec 14 19:59:50 ceph-master ceph-mon[28271]: in thread 755fcc30 thread_name:admin_socket
Dec 14 19:59:50 ceph-master systemd[1]: ceph-mon@ceph-master.service: main process exited, code=killed, status=11/SEGV
Dec 14 19:59:50 ceph-master systemd[1]: Unit ceph-mon@ceph-master.service entered failed state.
Dec 14 19:59:50 ceph-master systemd[1]: ceph-mon@ceph-master.service holdoff time over, scheduling restart.
Dec 14 19:59:50 ceph-master systemd[1]: Stopping Ceph cluster monitor daemon...
Dec 14 19:59:50 ceph-master systemd[1]: Starting Ceph cluster monitor daemon...
Dec 14 19:59:50 ceph-master systemd[1]: ceph-mon@ceph-master.service start request repeated too quickly, refusing to start.
Dec 14 19:59:50 ceph-master systemd[1]: Failed to start Ceph cluster monitor daemon.
Dec 14 19:59:50 ceph-master systemd[1]: Unit ceph-mon@ceph-master.service entered failed state.

I'm looking for guidance here as I'm not sure why this doesn't work. I am using the following URLs for my apt repos:

root@ceph-master:~# cat /etc/apt/sources.list
deb http://mirrordirector.raspbian.org/raspbian/ testing main contrib non-free rpi

root@ceph-master:~# cat /etc/apt/sources.list.d/ceph.list
deb https://download.ceph.com/debian-jewel/ jessie main

Has anyone else tried this and had similar problems? Any advice on how to proceed or work around this issue?

3
0

Dec 14 19:59:47 ceph-master ceph-mon[28237]: *** Caught signal (Segmentation fault) **

That's a programming error. ceph is accessing a pointer that is pointing out into nirvana.

As @SYN said - you could be more lucky with a more recent version of ceph, where this bug might be fixed. If not then you should report the bug along with the stack trace visible in your log there upstream to the ceph developpers - everyone should be happy when bugs get caught and fixed especially so in storage software.

0
0

I finally got ceph to work on my Raspberry Pi 3s. The Ceph repos only have ARM packages for arm64 architecture. All the armhf packages are gone. I think the ceph community doesn't care about 32-bit anymore and have been focusing on 64 bit. The trick was to get an arm64 version of Ubuntu installed. The only arm64 packages available at download.ceph.com were for either xenial or trusty. After several failed attempts to install a 64 bit version of ubuntu, I used a 64bit version of debian and just changed the repos to point to:

deb http://ports.ubuntu.com/ubuntu-ports xenial main universe
deb http://ports.ubuntu.com/ubuntu-ports xenial-updates main universe
deb http://ports.ubuntu.com/ubuntu-ports xenial-security main universe

I found the 64-bit rpi debian image here:

https://github.com/bamarni/pi64/releases

I have ceph luminous up and running!

ceph-admin@ceph-node0:~/my-cluster$ sudo ceph -s
  cluster:
    id:     5fa3e5e2-04ea-407e-b6f1-8ef5a3b3b1f2
    health: HEALTH_OK

  services:
    mon: 1 daemons, quorum ceph-node0
    mgr: ceph-node0(active)
    osd: 3 osds: 3 up, 3 in

  data:
    pools:   0 pools, 0 pgs
    objects: 0 objects, 0 bytes
    usage:   3164 MB used, 168 GB / 171 GB avail
    pgs:     
1
  • Thanks @slappyjam. I face the similar problem. I was deploying ceph on RPI 3, running Debian 8.0 Jessie, 32-bit OS. I installed new 64 bit OS as like you mentioned. Now, things are working. Although, there was one issue : ceph user was not allowed to write in /var/lib/ceph so, I changed with the command: $ chown -R ceph:ceph /var/run/ceph . Now things are working fine.
    – Arif A.
    Aug 16 '18 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.