### Configuration ## OSD memory Configuration # Changing paramenter of a running service ``` ceph daemon /var/run/ceph/ceph-osd.22.asok config show | grep memory ceph daemon osd.22 config set osd_memory_target 2147483648 ``` or you can inject parameter with ``` ceph tell mon.\* injectargs '--mon-allow-pool-delete=true' ``` # Changing paramenters permanently Change the parameters on the admin `ceph.conf` file ``` ceph-deploy --overwrite-conf config push <node> ``` ## Crush map change the device class of an OSD if needed ``` ceph osd crush rm-device-class osd.1 ceph osd crush set-device-class nvme osd.1 ``` Add bucket type to hierarchy: ``` ceph osd crush add-bucket group1 host ceph osd crush set 0 1.0 host=group1 ``` Define the crush map rule ``` ceph osd crush rule create-replicated nvme_meta default host nvme ceph osd crush rule create-replicated hdd_data default host hdd ``` ## Manual edit crush crush map ``` ceph osd getcrushmap -o crush.map crushtool -d crush.map -o decompiled.map crushtool -c decompiled.map -o manual.map ceph osd setcrushmap -i manual.map ceph osd crush rule create-replicated data_custom default custom_group hdd ``` ## Pools ``` ceph osd pool create ceph_data 1024 replicated hdd_data ceph osd pool set ceph_data size 2 ceph osd pool create ceph_metadata 2 replicated nvme_meta ceph osd pool set ceph_metadata size 2 ceph osd force-create-pg 2.0 --yes-i-really-mean-it ``` To change the number of placement groups in a pool ``` ceph osd pool set <pool name> pg_num <num> ``` ### Pool autotune ``` ceph osd pool autoscale-status ceph osd pool set foo pg_autoscale_mode on ``` # Erasure coding ``` ceph osd erasure-code-profile set ec-31-profile k=3 m=1 crush-failure-domain=osd crush-device-class=hdd ceph osd pool create ec31 2000 erasure ec-31-profile ceph osd pool set ec31 allow_ec_overwrites true ceph osd pool application enable ec31 cephfs ceph osd erasure-code-profile set ec-22-profile k=2 m=2 crush-failure-domain=host crush-device-class=hdd ceph osd pool create ec22 1024 erasure ec-22-profile ceph osd pool set ec22 allow_ec_overwrites true ceph osd pool application enable ec22 cephfs ``` ## Filesystems ``` ceph fs new cephfs ceph_metadata ceph_data ``` ## Authentication List the current authentication keys ``` ceph auth list ``` Adding a new key ``` ceph auth get-or-create client.1 mon 'allow r' mds 'allow rw' osd 'allow rw pool=ceph_data' ``` to change ACL you must enable the client to write also on the metadata (note the `p` on the `mds`) ``` ceph auth get-or-create client.2 mon 'allow rw' mds 'allow rwp' osd 'allow rwx pool=ceph_data, allow rwx pool=ceph_metadata' ``` ## Mount the filesystem Copy the content of the key to client that is going to mount the fs ``` [client.1] key = XXXXXXXXXXXXXXXXXXXXXXXXXXXXx== ``` Create a file `/etc/ceph/client.admin` with content ``` XXXXXXXXXXXXXXXXXXXXXXXXXXXXx== ``` and then issue ``` mount -v -t ceph 131.154.130.69:6789:/ /mnt/ceph -o name=1,secretfile=/etc/ceph/client.admin ``` ### Mount a different folder To specify a diffent folder you have to specify the path as an mds parameter ``` ceph fs authorize cephfs client.<client name> / r <path> rw ``` The folder must be present in the fs before mounting. ``` mount -v -t ceph 131.154.130.166:6789:<path> <mount point> -o name=<client>,secretfile=<secrefile> ``` ## Logging If your OS disk is relatively full, you can accelerate log rotation by modifying the Ceph log rotation file at `/etc/logrotate.d/ceph`. Add a size setting after the rotation frequency to accelerate log rotation (via cronjob) if your logs exceed the size setting. For example, the default setting looks like this: ``` rotate 7 weekly size 500M compress sharedscripts ``` And add a cron job: ``` crontab -e ``` like this ``` 30 * * * * /usr/sbin/logrotate /etc/logrotate.d/ceph >/dev/null 2>&1 ```