tag:blogger.com,1999:blog-6167470039524068257.post4046975307918258133..comments2023-10-28T04:25:11.315-07:00Comments on Champion of Cyrodiil: Recover Openstack Ceph data with missing/no monitor(s)Champion Of Cyrodiilhttp://www.blogger.com/profile/06792737627887393792noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-6167470039524068257.post-90323142229243817962017-01-30T10:19:01.466-08:002017-01-30T10:19:01.466-08:00The "wildcard" is the key, "13f2a30...The "wildcard" is the key, "13f2a30976b17" which is defined as replicated header file names for each rbd image on your ceph cluster.<br /><br />It has been a while, but I believe this is the RBD Image associated with your VM image. I believe it is in the Openstack database for cinder, the volume management schema.<br /><br />The general idea here is that you use the image-id with a wildcard to find all the obs, strip the replicas (duplicates), scp them from the nodes to the master server, and dd them together.<br /><br />This script, https://github.com/charlescva/ceph-recovery/blob/master/recover-osd-data.sh on line 4, defines your nodes. For you since you have 12, 20 and 35... just use those three nodes for line 4 and enter the image ID as the wildcard. Line 7 through 17 basically create a list of files. consolidate-stuff.sh makes them into a single list, scp-obs.sh will scp them from the nodes to the master server in a single folder called 'test', and then dd will chain them together. Your result should be the recovered image which in my case were qcow files.<br /><br />Hope this helps, sorry for the late response. Also my bash scripting was weak when i did this two years ago, sorry ;pChampion Of Cyrodiilhttps://www.blogger.com/profile/06792737627887393792noreply@blogger.comtag:blogger.com,1999:blog-6167470039524068257.post-89189494372218826442016-12-14T16:26:23.614-08:002016-12-14T16:26:23.614-08:00hello!
I lost my openstack ceph monitor and now ...hello! <br /><br />I lost my openstack ceph monitor and now I need do recover the vm images in ceph. I have 3 osd nodes: node-12 (osd.0); node-20 (osd.1); node-35 (osd.2 and osd.3). I would like to use your scripts for recovering the images. Could you help me? I have some questions:<br /><br />* How can I find all "wildcards"?<br />* Is each different "wildcard" in ceph osd related with a different vm image in openstack?<br />* There is 6 scripts in your repository. How can I use them (execution order) to automate the recovery process of all images in my 3 nodes (4 OSDs).<br /><br />Thank you very much in advance. I am looking for a solution since a long time.Unknownhttps://www.blogger.com/profile/15388914163780949368noreply@blogger.comtag:blogger.com,1999:blog-6167470039524068257.post-88438293199129462732015-09-30T11:16:44.976-07:002015-09-30T11:16:44.976-07:00It has been a while since I reviewed this script, ...It has been a while since I reviewed this script, but it looks like DIR_4 is being included in the result.csv, which means the 'find' command is catching it. Perhaps try to exclude DIR_4 from the result.csv, since it does not look like an object file.Champion Of Cyrodiilhttps://www.blogger.com/profile/06792737627887393792noreply@blogger.comtag:blogger.com,1999:blog-6167470039524068257.post-42841893804223544152015-08-31T16:07:05.562-07:002015-08-31T16:07:05.562-07:00hello!
i use you script and get scp: /var/lib/ceph...hello!<br />i use you script and get scp: /var/lib/ceph/osd/ceph-3/current/1.34_head/DIR_4: not a regular file<br /><br />#!/bin/bash<br />set -e<br />read -p "HeaderWildcard: " WILDCARD<br />for i in "2";<br /> do<br /> #echo ****NODE$i****<br /> for x in $(ssh -q proxmox$i ls /var/lib/ceph/osd);<br /> do<br /> #echo ****$x****<br /> for y in $(ssh -q proxmox$i find /var/lib/ceph/osd/$x/current -type d -size +1b | grep _head);<br /> do<br /> ssh -q proxmox$i find $y -type f -name *$WILDCARD* >> proxmox$i.$x.data.files.tmp<br /> done<br /> cat proxmox$i.$x.data.files.tmp | cut -d "/" -f 8,9 >> proxmox$i.$x.data.files<br /> rm proxmox$i.$x.data.files.tmp<br /> done<br /> done<br /><br />for x in $(for i in $(ls *.files); do echo $i; done); do ./consolidate-stuff.sh $x; done<br />rm *.files<br />cat result.csv | sort -t "," -k2 -u > sorted.results<br />rm result.csv<br />./scp-obs.sh<br />cd test<br />for i in $(ls); do dd if=$i of=$WILDCARD.qcow2 bs=1024 conv=notrunc oflag=append; done<br />mv $WILDCARD.qcow2 ..<br />cd ...<br />rm -Rf testAnonymousnoreply@blogger.com