It's not pretty but sometimes it has to be done Here is the set of commands I use to install Ceph MONs, MGRs or OSDs manually This came about becuase I was trying to use cephadm but the tool was broken for Ubuntu 22.04 at the time(Wouldn't bootstrap) so I did a package install using apt install ceph which pulled Quincy …

Read More

This script is a work in progress, fell free to reach out to me if you have any improvements. The idea is to leverage Ceph's snap differential export feature and Copy-On-Write snapshots\layers to allow for quick and easy snaps and exports of running VM volumes from one cluster to another. The obvious disclaimer about …

Read More

This is just a basic example of getting RadosGW to run in docker with Keystone auth support. It runs S3 and Swift API's out of the box, so you can use tools like s3cmd or 'openstack container list' using the below configs. Command to run inside docker container 1radosgw -d -f --cluster ceph --name client.rgw.mon-1.rgw0 …

Read More

Edit crushmap in a running cluster then re-inject it - Be careful! 1ceph osd getcrushmap -o crushmap 2crushtool -d crushmap -o crushmap.txt 3vim crushmap.txt <--Make your changes here 4crushtool -c crushmap.txt -o crushmap1 5ceph osd setcrushmap -i crushmap1

Read More

Sometimes it's necessary to rebuild the content of a Ceph monitor's local database, E.G it becomes corrupt or you somehow loose it! This process assumes the cluster is running and the remaining mons have achieved quorum, in which case it's safe to blow away the local DB of a single mon and let it re-create from the …

Read More

I always forget how to mount the subfolder of a CephFS filesystem sudo ceph-fuse -m 192.168.2.9:6789 -r /subfolder /folder/on/my/machine Note: The proceeding / after the -r is important Some great info here on restricting access to specific folders and more notes on mounting …

Read More

This script enables the requisite features on all RBD images in a pool to allow you to run rbd du and have it return a result quickly as opposed to having to calculate the size very time rbd ls -p backup1 | while read line; do echo "$line" rbd feature enable backup1/$line object-map exclusive-lock rbd object-map …

Read More

Dont take my word for it on the WAL sizing - Check http://docs.ceph.com/docs/mimic/rados/configuration/bluestore-config-ref/ This script will create a spare 20G Logical volume to use as the WAL for a second spinner later if you need it export SSD=sdc export SPINNER=sda vgcreate ceph-ssd-0 /dev/$SSD vgcreate ceph-hdd-0 …

Read More

When using the new ceph-volume scan and activate commands on ceph Nautilus after an upgrade from Luminous I was getting the following message 1root@ceph2 ~]# ceph-volume simple activate --all 2--> activating OSD specified in /etc/ceph/osd/37-11af5440-dadf-40e3-8924-2bbad3ee5b58.json 3Running command: /bin/ln -snf …

Read More

Original article here Ceph’s default IO priority and class for behind the scene disk operations should be considered required vs best efforts. For those of us who actually utilize our storage for services that require performance will quickly find that deep scrub grinds even the most powerful systems to a halt. Below …

Read More