Questions tagged [ceph]

For Ceph, the open-source, distributed storage system, mounted as CephFS using FUSE or NFSv4.

Filter by
Sorted by
Tagged with
0 votes
1 answer
72 views

cephadm - how to separate ssh network from monitor network

In my company, for several years we were using Ceph while using ceph-ansible as the deployer (and for upgrade, scale operations, etc). Recently I was assigned to migrating to 'cephadm' for ...
Itay R's user avatar
  • 1
1 vote
1 answer
34 views

is it possible to run a ceph rbd on erasure coded ceph pool, without a separate replicated metadata pool?

I'm new to ceph so forgive me if this is common knowledge, but I can't find it. This seems like a simple question, but I can't find any solid answer. In 2017 when rbd on ec pools was first implemented,...
stu's user avatar
  • 143
1 vote
1 answer
56 views

Can't remove ceph xattrs on linux

I had set xattrs for quota limits on CephFS $ setfattr -n ceph.quota.max_bytes -v 1100000000 /mnt/cephfs/data/ I can get value of this attribute $ getfattr -n ceph.quota.max_bytes /mnt/cephfs/data/ ...
Dan B's user avatar
  • 11
0 votes
0 answers
238 views

Ceph non-replicated pool (replication 1)

I have a 10 node cluster. I want to create a non-replicated pool (replication 1) and I want to ask some questions about it: Let me tell you my use case: I don't care about losing data, All of my data ...
Morphinz's user avatar
  • 429
0 votes
1 answer
154 views

DRBD on top of Ceph

Would it be possible to have DRBD directly running inside a Ceph Pool? I have a backup machine with files stored directly on disk. The offsite backup machine has Ceph installed and configured on all ...
Mr. Diba's user avatar
  • 390
1 vote
1 answer
530 views

CEPH can't find OSD's

I installed ceph cluster using cephadm and bootstrapping. I can view disks via inventory, but they do not show in the device list. Why is this the case? How can I add a disk to the cluster? root@RX570:...
Trevor K Smith's user avatar
0 votes
1 answer
858 views

How to add a disk to Ceph in Debian

I have a Ceph node with 2 disks (SSD 500G). /dev/sda is used for the OS (Debian 11) I want to install the second SSD as a OSD for Ceph. But Ceph does not recognize my 2nd disk. Calling $ sudo cephadm ...
Ralph's user avatar
  • 153
0 votes
1 answer
907 views

ceph quincy Unable to create a new OSD id

I am running a ceph cluster (quincy) with some nodes and several OSDs. I tried to remove/delete a ODS and now my cluster seems to still know some old meta data. I am unable to add the existing OSD ...
Ralph's user avatar
  • 153
0 votes
0 answers
94 views

Ceph error when checking the status of the cluster

ceph -s error picture is as follows: In addition, I tried to execute the ceph-deploy admin xxNode command in the deployment directory, but the result was not satisfactory. I am not sure whether the ...
Alan smith's user avatar
0 votes
1 answer
151 views

cephadm set cluster_network

I have setup new cluster on single NIC using cephadm but i have added extra nic for replication cluster_network. This is what i did to configure cephadm but it didn't work. $ ceph config set global ...
Satish's user avatar
  • 1,652
0 votes
1 answer
3k views

ceph storage MDSs report slow metadata IOs

I am playing with ceph storage in lab and i have single server so thought installing all service on single box, like MON, OSD, MDS etc. I have create two disk using loopdevice (This server has SSDs ...
Satish's user avatar
  • 1,652
0 votes
1 answer
783 views

5 replica over 3 node ceph cluster possible?

Does that make sense and is it possible on a 3 node setup, how should the crush rule look like? Thats the current one. The idea behind this is to compensate multiple osd outages (for example 3 osds ...
jsterr's user avatar
  • 1,321
1 vote
2 answers
2k views

How do I create a raw (no filesystem) loopback device that passes as a legitimate blockdevice?

The question How do I create and mount a fake block device (using a large file/disk image) that passes as a legitimate unformatted disk? Backstory I am trying to set up rook with ceph (a distributed ...
Hannes Knutsson's user avatar
1 vote
1 answer
295 views

Creating cephfs volume in a test environment is very slow

I've installed Ceph into a single VM for testing purposes (I'm not testing ceph, per se, but I need a ceph endpoint for the tools I'll be running). The install was easy enough, but the cephfs ...
larsks's user avatar
  • 33.9k
0 votes
0 answers
98 views

Newly added node is not available in ceph pool

Absolute newbie in Ceph and down with a problem. Added 3 new nodes to my K8's setup and its available and in Ready state - as per kubectl get nodes. But, cant see the same in my ceph pool, while ...
Chel MS's user avatar
  • 113
0 votes
2 answers
3k views

Purge disks after removing Ceph

I'm trying to remove ceph totally from my servers. I released OSDs from server node and format disks, create new partition with parted but still see ceph partition inside disks. Followed this ...
fth's user avatar
  • 101
0 votes
0 answers
2k views

how can increase ceph osd size, created on os disk

I m new in ceph world, and test ceph cluster on one virtual server. I`m creating osd , according this: https://docs.ceph.com/docs/dumpling/start/quick-ceph-deploy/#add-ceph-osd-daemons i create some ...
Sergey's user avatar
  • 1
0 votes
0 answers
158 views

Ceph RBD kernel module

Within ceph kernel client, I am looking for functions responsible for rbd block open/read/write operations. Since /drivers/block/rbd.c contains function rbd_open(), it looked as a suitable candidate ...
wtdmn's user avatar
  • 1
0 votes
1 answer
305 views

Delete (corrupted?) directory?

I have an Ubuntu 16.04 with a cephfs mount shared with another 2 Ubuntus. There is a folder inside the cephfs mount that won't answer to any command (ls, cd, rm, du...). So I guess it's corrupted or ...
Zumo de Vidrio's user avatar
-1 votes
1 answer
1k views

Can't download file on dropbox 400 Bad Request

Guide https://tracker.ceph.com/projects/ceph/wiki/Ceph_Vagrant_Setup $ wget https://www.dropbox.com/s/hn28qgjn59nud6h/ceph-all-in-one.box --2020-03-14 12:11:09-- https://www.dropbox.com/s/...
rawmain's user avatar
  • 105
0 votes
0 answers
571 views

Mount disk used by rook-ceph

I'm running a Kubernetes cluster with 3 nodes, each having 2 hard drives, one with Linux installed on. Another that I didn't mount to let rook store data on it. I've seen on the documentation that ...
MHogge's user avatar
  • 237
3 votes
1 answer
651 views

ssh won't connect with RSA key pairs iff /home directory is ceph-mounted

I've got the problem that on a server I maintain, after switching to a Ceph filesystem for the homedirectories, suddenly ssh login through RSA keys stopped working. I've tried checking several things ...
Joeytje50's user avatar
  • 165
1 vote
0 answers
104 views

Open-iSCSI Target IQN Naming Case

I'm using LRBD to configure iSCSI targets from a Ceph cluster, LRBD uses Open-iSCSI/targetcli to configure the targets. When I use an uppercase letter in the unique part of the iqn (after the colon),...
rusty shackleford's user avatar
0 votes
1 answer
402 views

Ceph Monitor iptables rules strange behaviour

I'm trying to set up iptables for a Ceph cluster. I'm currently putting rules together for one of the monitor servers. The monitor daemon listens on tcp/6789 (ip address removed as it is a public ...
rusty shackleford's user avatar
0 votes
1 answer
261 views

Ceph filesystem from old pools

I have deleted a Ceph filesystem, and copied the metadata and data pools. Is it possible to recreate the filesystem from these copies?
Cyclic3's user avatar
  • 930
0 votes
1 answer
852 views

Is it possible to increase Ceph throughput with multi-port NICs without LACP?

My Ceph cluster consists of 3 rather old servers storage servers, and a Netgear GS724Tv1, which does not appear to support LACP. I have already set up a separate cluster network and public network, ...
Cyclic3's user avatar
  • 930
1 vote
1 answer
1k views

Ceph create pool / Crush map max_size

I have the default bucket types type 0 osd type 1 host ... type 10 root and three host buckets: host cluster01a { id -2 # do not change unnecessarily # weight 1.607 alg straw hash 0 # ...
FaxMax's user avatar
  • 726
1 vote
1 answer
699 views

Ceph with many small monitors/mds

Ceph documentation (http://docs.ceph.com/docs/jewel/start/hardware-recommendations/) puts an emphasis on the need for more powerful hardware as the cluster scales. For instance hardware specs for ...
Adam's user avatar
  • 11
1 vote
0 answers
1k views

Ceph for small files?

Currently I have 6 dedicated servers in a data center. Two Servers are Mailservers running exim and dovecot (Maildir) and 4 Webservers. Each Server has two 3TB HDDs. My current problem is, that we ...
user39063's user avatar
3 votes
2 answers
4k views

ceph luminous osd memory usage

I am trying to build small CEPH cluster on single box - testing in the lab, before switching bigger probject to it. Run into memory problems and can not find way to control it. We have old Core2Duo ...
Anton Aleksandrov's user avatar
1 vote
2 answers
308 views

Running a program with autologin not working like running on bash

I wrote a program and its automatically installs ceph. This program works fine when I run manually But when I login automatically with getty (my conf below) and run the program as last line in /etc/...
Morphinz's user avatar
  • 429
0 votes
2 answers
230 views

ceph-mon fails to start on rasberry pi (raspbian 8.0)

ceph-mon fails to start on rasberry pi (raspbian 8.0) I recently purchased 3 raspberry pi nodes to create a small storage cluster to test with at my home. I found a couple of procedures on setting ...
slappyjam's user avatar
0 votes
1 answer
2k views

how to check if two servers are replicated properly on ceph cluster

Recently I built ceph with two server configuration. most uneasy thing is that i don't know how to check if two servers are replicated properly on ceph cluster. Does anyone using ceph confirm that ...
jihyun kim's user avatar
1 vote
1 answer
802 views

How to mount using ceph-fuse and specify IP in /etc/fstab?

My server configuration is as follows: A ceph cluster server(10.1.1.138) B ceph cluster server(10.1.1.54) C ceph client (10.1.1.238) I could mount using the following ceph-fuse command: sudo ceph-...
jihyun kim's user avatar
1 vote
1 answer
2k views

mount cephfs failed because of failure to load kernel module

I got a confused issue in docker as below: After install ceph successfully, i want to mount cephfs but failed: [root@dbffa72704e4 ~]$ /bin/mount 172.17.0.4:/ /cephfs -t ceph -o name=admin,secretfile=...
daixiang0's user avatar
1 vote
1 answer
988 views

does CEPH RBD mount on Linux support boot device?

Does CEPH RBD mount on Linux support boot device? for RBD deployment, example would be like this: http://blog.programster.org/ceph-deploy-and-mount-a-block-device
Thomas G. Lau's user avatar
2 votes
1 answer
2k views

Comparison of Ceph vs btrfs for single node storage

Single Node Ceph: Your Next Home Storage Solution makes case for using Ceph over ZFS on a single node. This got me wondering about Ceph vs btrfs: What are the advantages / disadvantages of using ...
Tom Hale's user avatar
  • 30.1k
2 votes
1 answer
7k views

Using ceph storage directly for docker volumes

Does Docker or Kubernetes support Ceph storage interfaces (objects, red, cephfs)?
Ijaz Ahmad's user avatar
  • 7,214
1 vote
1 answer
2k views

[ceph_deploy][ERROR ] RuntimeError: NoSectionError: No section: 'ceph'

Follow the official docs:http://docs.ceph.com/docs/master/start/quick-start-preflight/#rhel-centos I do the below configuration of the ceph.repo: [ceph-noarch] name=Ceph noarch packages baseurl=http:...
lustre's user avatar
  • 317
0 votes
3 answers
3k views

Require liblttng when I install the ceph in CentOS7.2

When I install the ceph and ceph-radosgw in my CentOS 7.2: yum -y install ceph ceph-radosgw I get the below error: ...... --> Finished Dependency Resolution Error: Package: 1:ceph-base-10.2.9-0....
three-blocks's user avatar
1 vote
2 answers
978 views

RuntimeError: NoSectionError: No section: 'ceph-source'

When I use ceph-deploy to install the ceph to every node: ceph-deploy install node0 node1 node2 I get the below error: [node1][WARNIN] check_obsoletes has been enabled for Yum priorities plugin [...
lustre's user avatar
  • 317
4 votes
3 answers
2k views

How to delete a invalid osd in ceph cluster?

[root@dev-master ceph-cluster]# ceph osd tree ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY -1 0.01740 root default -4 0.00580 host osd2 0 0.00580 osd.0 down 0 ...
Inuyasha's user avatar
1 vote
1 answer
2k views

Recover VM Images from dead ceph cluster

A few days ago our Ceph Cluster died - nobody knows why, but it seems that the monitors just got deleted. When trying to re-add them to the ceph.conf we get timeouts when starting ceph and these ...
Kryptur's user avatar
  • 121
3 votes
1 answer
837 views

Ceph - Prepare Shared SSD For Journaling

The Ceph documentation for adding drives shows the use of the prepare statement twice, with the same drive for the journal as shown below (where /dev/ssd is the journal drive): Once the prepare ...
Programster's user avatar
  • 2,247
1 vote
0 answers
2k views

ceph fs apply latency too high resulting in high load in VMs

I have some problems in a ceph cluster. The fs_apply_latency is too high which leads to high load and slow responding qemu VMs (which use ceph images as VHD). The setup is: 5 hosts with 4 HDDs and 1 ...
devnull's user avatar
  • 227
4 votes
3 answers
2k views

Can I enable cephx authentication using a pool in qemu/kvm?

(CentOS 7, Ceph Incunabulus, everything patched up to date) Background I've got a working converged Ceph/qemu/kvm cluster with cephx enabled for authentication. I'm trying to use this to store block ...
Peter Crowther's user avatar
2 votes
1 answer
799 views

Problems with overall ceph performance with QEMU

I am experiencing some performance problems with my QEMU KVMs on my ceph cluster. The cluster has 4 nodes with 4x1TB drives each, 48/64GB RAM, Intel Xeon and AMD Opterons. They are interconnected by ...
devnull's user avatar
  • 227
2 votes
0 answers
242 views

Remove orphaned rados block device

I'm testing ceph at the moment. Today I decided to start over with my cluster and deleted the old one to setup a new one. I forgot to unmap and delete a mapped rbd device from another host. Now I see ...
devnull's user avatar
  • 227
1 vote
1 answer
882 views

Using a rados block device as block device in libvirt qemu

What is the difference between using a rados block device in libvirt qemu with disk type='network' and disk type='block'? The ceph guide tells me to do the following: <disk type='network' device='...
devnull's user avatar
  • 227