[PYTHON] Exploring the possibility of recovering deleted VMDK files on VMFS
About deleting a VM
When operating the vSphere Client, the right-click menu is
Delete from inventory and Delete from disk are next to each other.
Both give warnings, but I inadvertently intend to remove them from my inventory. If you delete it from the disk, the VM data will be buried and you will not be able to use it.
Since the deletion itself ends immediately, it is presumed that 0 has not been overwritten. I think the data itself remains I would like to find out if the deleted files can be recovered.
Survey flow
Since the specification of VMFS is not published, Try to investigate by referring to the source of vmfs-tools. It doesn't seem to be perfect for handling VMFS5, but it's a challenge for the time being.
- Install vSphere ESXi as a guest on the host vSphere ESXi
(The VMFS area of the vmdk file with this guest ESXi installed will be investigated) - Install CentOS as a Nested VM
(Add CentOS as a guest to guest ESXi) - Shut down Netsed VM and guest ESXi and copy vmdk of guest ESXi
- Check the situation in advance with the Nested VM file existing.
- Find out what changed after removing the Nested VM from disk
(Also copy the deleted vmdk file and check the VMFS area)
In the Nested environment, the VMFS-formatted partition under investigation is in the vmdk file, so Easy to take to other places for work.
Survey started
The <vmname> .vmdk file is called a virtual disk descriptor file and is not the body of the disk data, so
Copy the <vmname> -flat.vmdk file.
The target here is named __backup_vsphere-flat.vmdk.
Well, how should I find it? First, with the Nested VM not deleted Let's see if we can find the data for the Nested VM.
The search target is the MBR of the Nested VM. Transfer the VMDK of guest ESXi to another Linux and start investigating there.
However, I don't know much about MBR, so let's check the structure first. Roughly understand the following based on the reference materials.
- 446 bytes from the beginning is the boot loader
- 64 bytes behind the bootloader is a partition table (4 records of 16 bytes each)
- The last 2 bytes are the magic number
If you read the MBR of CentOS with GRUB installed at hand, you will find the following:
- A string that GRUB is likely to use
- Partition table
- Magic number
Use these as clues to find the MBR. For Nested VM, I created a 10GB VMDK and installed two of them. I cut each partition differently. The guest ESXi vmdk is made with 40GB, and I search from end to end for the time being.
searchmbr.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys, os, re, struct
#target = './mbr.dd'
target = './__backup_vsphere-flat.vmdk'
MAGIC_OF_MBR = '\x55\xaa'
MARK_OF_GRUB = 'GRUB \x00Geom\x00Hard Disk\x00Read\x00 Error'
try:
if sys.argv[1] == '-d':
DEBUG = 1
except:
pass
def D(mesg):
try:
if DEBUG:
import inspect
print "[DEBUG:%d] %s" % (inspect.currentframe().f_back.f_lineno, mesg)
except NameError:
pass
def convert_num(num):
if len(str(int(num))) > 12:
return '%.1fT' % (num/(1024.0**4))
elif len(str(int(num))) > 9:
return '%.1fG' % (num/(1024.0**3))
elif len(str(int(num))) > 6:
return '%.1fM' % (num/(1024.0**2))
elif len(str(int(num))) > 3:
return '%.1fk' % (num/(1024.0**1))
else:
return str(num)
def check(data):
bootloader = data[0:446]
part_table = data[446:510]
magic = data[510:512]
if magic != MAGIC_OF_MBR:
return False
if re.search(MARK_OF_GRUB, bootloader):
print 'This is maybe MBR !!'
else:
return False
splitter = lambda s, n: [s[i:i+n] for i in range(0, len(s), n)]
print "Partition Boot %s %s Id Size" % ('Start'.rjust(10), 'End'.rjust(10))
i=1
for record in splitter(part_table, 16):
boot = record[0]
D('flag: %s' % hex(ord(boot)))
if boot == '\x80':
boot = '*'
elif boot == '\x00':
boot = ''
else:
print 'boot flag is wrong'
boot = hex(ord(boot))
id = record[4:5]
#Check only LBA data(Don't look at CHS data)
start_sector, num_sector = struct.unpack('II', record[8:16])
print "%s %s %10d %10d %s %sB" % (str(i).rjust(9), boot.rjust(4), start_sector, start_sector + num_sector, hex(ord(id)).rjust(4), convert_num(num_sector * 512))
i+=1
return True
with open(target , 'r') as f:
try:
fd = f.fileno()
except Exception as e:
print e
raise Exception('cannot get file size')
else:
size = os.fstat(fd).st_size
i=1
data = f.read(512)
while data:
if check(data):
print "position of above data: %d - %d" % (f.tell() - 512, f.tell())
data = f.read(512)
if f.tell() > size / 100.0 * i:
D("searching... %d / %d" % (f.tell(), size))
i+=1
MBR search
# python searchmbr.py
This is maybe MBR !!
Partition Boot Start End Id Size
1 * 2048 1026048 0x83 500.0MB
2 1026048 2074624 0x82 512.0MB
3 2074624 20971520 0x83 9.0GB
4 0 0 0x0 0B
position of above data: 38196477952 - 38196478464
This is maybe MBR !!
Partition Boot Start End Id Size
boot flag is wrong
1 0x24 332251185 2480556663 0x0 1.0TB
boot flag is wrong
2 0xf9 2548826057 4435176463 0xbe 899.5GB
boot flag is wrong
3 0x70 3053499650 4967399938 0x0 912.6GB
boot flag is wrong
4 0xd7 0 0 0x4f 0B
position of above data: 38792327168 - 38792327680
This is maybe MBR !!
Partition Boot Start End Id Size
boot flag is wrong
1 0x24 332251185 2480556663 0x0 1.0TB
boot flag is wrong
2 0xf9 2548826057 4435176463 0xbe 899.5GB
boot flag is wrong
3 0x70 3053499650 4967399938 0x0 912.6GB
boot flag is wrong
4 0xd7 0 0 0x4f 0B
position of above data: 39116325888 - 39116326400
This is maybe MBR !!
Partition Boot Start End Id Size
1 * 2048 1026048 0x83 500.0MB
2 1026048 20971520 0x8e 9.5GB
3 0 0 0x0 0B
4 0 0 0x0 0B
position of above data: 39419117568 - 39419118080
This is maybe MBR !!
Partition Boot Start End Id Size
boot flag is wrong
1 0x24 332251185 2480556663 0x0 1.0TB
boot flag is wrong
2 0xf9 2548826057 4435176463 0xbe 899.5GB
boot flag is wrong
3 0x70 3053499650 4967399938 0x0 912.6GB
boot flag is wrong
4 0xd7 0 0 0x4f 0B
position of above data: 40218379264 - 40218379776
This is maybe MBR !!
Partition Boot Start End Id Size
boot flag is wrong
1 0x24 332251185 2480556663 0x0 1.0TB
boot flag is wrong
2 0xf9 2548826057 4435176463 0xbe 899.5GB
boot flag is wrong
3 0x70 3053499650 4967399938 0x0 912.6GB
boot flag is wrong
4 0xd7 0 0 0x4f 0B
position of above data: 40242606080 - 40242606592
I was able to find it safely. Some strange flags are found, but the Nested VM I made it with 10GB vmdk, so I'm sure the two flags are correct.
VMFS exploration
Now let's read VMFS. From here is the production.
# parted __backup_vsphere-flat.vmdk unit s print
model: (file)
disk/root/recover/__backup_vsphere-flat.vmdk: 83886080s
Sector size(logic/Physics): 512B/512B
Partition table: gpt
Number Start End Size File System Name Flag
1 64s 8191s 8128s fat16 boot
5 8224s 520191s 511968s fat16
6 520224s 1032191s 511968s fat16
7 1032224s 1257471s 225248s
8 1257504s 1843199s 585696s fat16
2 1843200s 10229759s 8386560s fat16
3 10229760s 83886046s 73656287s
Partition table is GPT The largest area should be VMFS, but I'm not sure with parted.
In GPT, the GUID of VMFS is ʻAA31E02A400F11DB9590000C2911D1B8. If you change the notation, ʻAA31E02A-400F-11DB-9590-000C2911D1B8
Looking at the location corresponding to the large partition No. 3,
The first three items are little endian. Certainly the GUIDs matched.
GPT survey
# hexdump -C -s `expr 512 \* 2 + 128 \* 2` -n 128 vsphere-flat.vmdk | less
00000500 2a e0 31 aa 0f 40 db 11 95 90 00 0c 29 11 d1 b8 |*.1..@......)...|
00000510 45 01 35 2b 41 d6 59 45 aa 77 33 40 c0 94 f4 0d |E.5+A.YE.w3@....|
00000520 00 18 9c 00 00 00 00 00 de ff ff 04 00 00 00 00 |................|
00000530 00 00 00 00 00 00 00 00 00 00 ff ff ff ff ff ff |................|
00000540 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
I want to continue reading information about VMFS.
vmfs_fs.h
#define VMFS_FSINFO_MAGIC 0x2fabf15e
struct vmfs_fsinfo_raw {
uint32_t magic;
uint32_t volver;
u_char ver;
uuid_t uuid;
uint32_t mode;
char label[128];
uint32_t dev_blocksize;
uint64_t blocksize;
uint32_t ctime; /* ctime? in seconds */
uint32_t _unknown3;
uuid_t lvm_uuid;
u_char _unknown4[16];
uint32_t fdc_header_size;
uint32_t fdc_bitmap_count;
uint32_t subblock_size;
} __attribute__((packed));
Looking at vmfs-tools, it seems that there is a magic number, so try to find it in the same way as for MBR.
searchvmfsinfo.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys, os, re, struct
target = './__backup_vsphere-flat.vmdk'
MAGIC_OF_VMFSINFO = '\x5e\xf1\xab\x2f'
try:
if sys.argv[1] == '-d':
DEBUG = 1
except:
pass
def D(mesg):
try:
if DEBUG:
import inspect
print "[DEBUG:%d] %s" % (inspect.currentframe().f_back.f_lineno, mesg)
except NameError:
pass
def check(data):
if MAGIC_OF_VMFSINFO in data:
print 'Magic Number found. This is maybe VMFS !!'
return True
else:
return False
with open(target , 'r') as f:
try:
fd = f.fileno()
except Exception as e:
print e
raise Exception('cannot get file size')
else:
size = os.fstat(fd).st_size
i=1
data = f.read(512)
while data:
if check(data):
print "position of above data: %d - %d" % (f.tell() - 512, f.tell())
data = f.read(512)
if f.tell() > size / 100.0 * i:
D("searching... %d / %d" % (f.tell(), size))
i+=1
vmfsinfo search
# python searchvmfs.py
Magic Number found. This is maybe VMFS !!
position of above data: 5257560064 - 5257560576
Magic Number found. This is maybe VMFS !!
position of above data: 37435702784 - 37435703296
Magic Number found. This is maybe VMFS !!
position of above data: 38707637760 - 38707638272
Magic Number found. This is maybe VMFS !!
position of above data: 39859326464 - 39859326976
Magic Number found. This is maybe VMFS !!
position of above data: 40373243392 - 40373243904
I found some. The beginning of the VMFS partition is 10229760 * 512 = 5237637120, so it doesn't seem to start right from the beginning.
I'm not sure if there's a gap, but maybe the first place I found it was VMFS information.
Did the others just happen to match?
You can see 128 bytes of label, so let's check it out.
Check label data
# hexdump -C -s `expr 5257560064 + 29` -n 128 __backup_vsphere-flat.vmdk
13960001d 64 61 74 61 73 74 6f 72 65 31 00 00 00 00 00 00 |datastore1......|
13960002d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
# hexdump -C -s `expr 37435702784 + 29` -n 128 __backup_vsphere-flat.vmdk
8b7577e1d 00 73 71 75 61 73 68 66 73 00 68 73 71 73 00 73 |.squashfs.hsqs.s|
8b7577e2d 68 73 71 00 25 75 2e 25 30 32 75 00 6e 73 73 00 |hsq.%u.%02u.nss.|
8b7577e3d 53 50 42 35 00 78 65 6e 69 78 00 2b 55 44 00 44 |SPB5.xenix.+UD.D|
8b7577e4d 55 2b 00 73 79 73 76 00 62 74 72 66 73 00 5f 42 |U+.sysv.btrfs._B|
8b7577e5d 48 52 66 53 5f 4d 00 4c 41 42 45 4c 4f 4e 45 00 |HRfS_M.LABELONE.|
8b7577e6d 4c 56 4d 32 5f 6d 65 6d 62 65 72 00 4c 56 4d 32 |LVM2_member.LVM2|
8b7577e7d 20 30 30 31 00 4c 56 4d 31 5f 6d 65 6d 62 65 72 | 001.LVM1_member|
8b7577e8d 00 48 4d 00 44 4d 5f 73 6e 61 70 73 68 6f 74 5f |.HM.DM_snapshot_|
8b7577e9d
# hexdump -C -s `expr 38707637760 + 29` -n 128 __backup_vsphere-flat.vmdk
90327ae1d 00 73 71 75 61 73 68 66 73 00 68 73 71 73 00 73 |.squashfs.hsqs.s|
90327ae2d 68 73 71 00 25 75 2e 25 30 32 75 00 6e 73 73 00 |hsq.%u.%02u.nss.|
90327ae3d 53 50 42 35 00 78 65 6e 69 78 00 2b 55 44 00 44 |SPB5.xenix.+UD.D|
90327ae4d 55 2b 00 73 79 73 76 00 62 74 72 66 73 00 5f 42 |U+.sysv.btrfs._B|
90327ae5d 48 52 66 53 5f 4d 00 4c 41 42 45 4c 4f 4e 45 00 |HRfS_M.LABELONE.|
90327ae6d 4c 56 4d 32 5f 6d 65 6d 62 65 72 00 4c 56 4d 32 |LVM2_member.LVM2|
90327ae7d 20 30 30 31 00 4c 56 4d 31 5f 6d 65 6d 62 65 72 | 001.LVM1_member|
90327ae8d 00 48 4d 00 44 4d 5f 73 6e 61 70 73 68 6f 74 5f |.HM.DM_snapshot_|
90327ae9d
# hexdump -C -s `expr 39859326464 + 29` -n 128 __backup_vsphere-flat.vmdk
947cd0e1d 00 73 71 75 61 73 68 66 73 00 68 73 71 73 00 73 |.squashfs.hsqs.s|
947cd0e2d 68 73 71 00 25 75 2e 25 30 32 75 00 6e 73 73 00 |hsq.%u.%02u.nss.|
947cd0e3d 53 50 42 35 00 78 65 6e 69 78 00 2b 55 44 00 44 |SPB5.xenix.+UD.D|
947cd0e4d 55 2b 00 73 79 73 76 00 62 74 72 66 73 00 5f 42 |U+.sysv.btrfs._B|
947cd0e5d 48 52 66 53 5f 4d 00 4c 41 42 45 4c 4f 4e 45 00 |HRfS_M.LABELONE.|
947cd0e6d 4c 56 4d 32 5f 6d 65 6d 62 65 72 00 4c 56 4d 32 |LVM2_member.LVM2|
947cd0e7d 20 30 30 31 00 4c 56 4d 31 5f 6d 65 6d 62 65 72 | 001.LVM1_member|
947cd0e8d 00 48 4d 00 44 4d 5f 73 6e 61 70 73 68 6f 74 5f |.HM.DM_snapshot_|
947cd0e9d
# hexdump -C -s `expr 40373243392 + 29` -n 128 __backup_vsphere-flat.vmdk
9666ece1d 00 73 71 75 61 73 68 66 73 00 68 73 71 73 00 73 |.squashfs.hsqs.s|
9666ece2d 68 73 71 00 25 75 2e 25 30 32 75 00 6e 73 73 00 |hsq.%u.%02u.nss.|
9666ece3d 53 50 42 35 00 78 65 6e 69 78 00 2b 55 44 00 44 |SPB5.xenix.+UD.D|
9666ece4d 55 2b 00 73 79 73 76 00 62 74 72 66 73 00 5f 42 |U+.sysv.btrfs._B|
9666ece5d 48 52 66 53 5f 4d 00 4c 41 42 45 4c 4f 4e 45 00 |HRfS_M.LABELONE.|
9666ece6d 4c 56 4d 32 5f 6d 65 6d 62 65 72 00 4c 56 4d 32 |LVM2_member.LVM2|
9666ece7d 20 30 30 31 00 4c 56 4d 31 5f 6d 65 6d 62 65 72 | 001.LVM1_member|
9666ece8d 00 48 4d 00 44 4d 5f 73 6e 61 70 73 68 6f 74 5f |.HM.DM_snapshot_|
9666ece9d
You can see the label datastore1 created by default from the first place you find it.
I don't know what else, but for now it looks like it can be ignored.
To be honest, I don't know what to do from here, but there seems to be an inode.
vmfs_inode.h
struct vmfs_inode_raw {
struct vmfs_metadata_hdr_raw mdh;
uint32_t id;
uint32_t id2; /* seems to be VMFS_BLK_FD_ITEM(id) + 1 */
uint32_t nlink;
uint32_t type;
uint32_t flags;
uint64_t size;
uint64_t blk_size;
uint64_t blk_count;
uint32_t mtime;
uint32_t ctime;
uint32_t atime;
uint32_t uid;
uint32_t gid;
uint32_t mode;
uint32_t zla;
uint32_t tbz;
uint32_t cow;
u_char _unknown2[432];
union {
uint32_t blocks[VMFS_INODE_BLK_COUNT];
uint32_t rdm_id;
char content[VMFS_INODE_BLK_COUNT * sizeof(uint32_t)];
};
} __attribute__((packed));
vmfs_metadata.h
struct vmfs_metadata_hdr_raw {
uint32_t magic; /* Magic number */
uint64_t pos; /* Position in the volume */
uint64_t hb_pos; /* Heartbeat position */
uint64_t hb_seq; /* Heartbeat sequence */
uint64_t obj_seq; /* Object sequence */
uint32_t hb_lock; /* Heartbeat lock flag */
uuid_t hb_uuid; /* UUID of locking server */
uint64_t mtime;
u_char pad1[0x1c0]; /* Padding/unknown */
} __attribute__((packed));
Anyway, if I can find an inode, I feel like I can do something about it.
vmfs_inode.h
#define VMFS_INODE_MAGIC 0x10c00001
The inode also seems to have a magic number, so look for this. It seems to be a clue if you display id and size by the way.
searchinode.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys, os, re, struct
target = './__backup_vsphere-flat.vmdk'
POS_OF_VMFSINFOMAGIC = 5257560064
MAGIC_OF_VMFSINODE = '\x01\x00\xc0\x10'
try:
if sys.argv[1] == '-d':
DEBUG = 1
except:
pass
def D(mesg):
try:
if DEBUG:
import inspect
print "[DEBUG:%d] %s" % (inspect.currentframe().f_back.f_lineno, mesg)
except NameError:
pass
def convert_num(num):
if len(str(int(num))) > 12:
return '%.1fT' % (num/(1024.0**4))
elif len(str(int(num))) > 9:
return '%.1fG' % (num/(1024.0**3))
elif len(str(int(num))) > 6:
return '%.1fM' % (num/(1024.0**2))
elif len(str(int(num))) > 3:
return '%.1fk' % (num/(1024.0**1))
else:
return str(num)
with open(target , 'r') as f:
def check(data):
if MAGIC_OF_VMFSINODE in data:
print 'Magic Number found. This is maybe inode !!'
data = f.read(512)
id, id2, size = struct.unpack('IIQ', data[0:8] + data[20:28])
print 'id = %d, id2 = %d, filesize = %sB' % (id, id2, convert_num(size))
return True
else:
return False
try:
fd = f.fileno()
except Exception as e:
print e
raise Exception('cannot get file size')
else:
size = os.fstat(fd).st_size
f.seek(POS_OF_VMFSINFOMAGIC)
i=1
data = f.read(512)
while data:
if check(data):
print "position of above data: %d - %d" % (f.tell() - 1024, f.tell() - 512)
data = f.read(512)
if f.tell() > size / 100.0 * i:
D("searching... %d / %d" % (f.tell(), size))
i+=1
inode search
# python searchinode.py
Magic Number found. This is maybe inode !!
id = 4, id2 = 1, filesize = 1.4kB
position of above data: 5259788288 - 5259788800
Magic Number found. This is maybe inode !!
id = 4194308, id2 = 1, filesize = 256.0kB
position of above data: 5259790336 - 5259790848
Magic Number found. This is maybe inode !!
id = 8388612, id2 = 1, filesize = 254.7MB
position of above data: 5259792384 - 5259792896
Magic Number found. This is maybe inode !!
id = 12582916, id2 = 1, filesize = 256.0MB
position of above data: 5259794432 - 5259794944
Magic Number found. This is maybe inode !!
id = 16777220, id2 = 1, filesize = 250.6MB
position of above data: 5259796480 - 5259796992
Magic Number found. This is maybe inode !!
id = 20971524, id2 = 1, filesize = 4.0MB
position of above data: 5259798528 - 5259799040
Magic Number found. This is maybe inode !!
id = 25165828, id2 = 1, filesize = 1.1MB
position of above data: 5259800576 - 5259801088
Magic Number found. This is maybe inode !!
id = 32772, id2 = 1, filesize = 0B
position of above data: 5472124928 - 5472125440
Magic Number found. This is maybe inode !!
id = 4227076, id2 = 2, filesize = 0B
position of above data: 5472126976 - 5472127488
Magic Number found. This is maybe inode !!
id = 8421380, id2 = 3, filesize = 0B
position of above data: 5472129024 - 5472129536
Magic Number found. This is maybe inode !!
id = 12615684, id2 = 4, filesize = 0B
position of above data: 5472131072 - 5472131584
Magic Number found. This is maybe inode !!
id = 16809988, id2 = 5, filesize = 0B
position of above data: 5472133120 - 5472133632
Magic Number found. This is maybe inode !!
id = 21004292, id2 = 6, filesize = 0B
position of above data: 5472135168 - 5472135680
Magic Number found. This is maybe inode !!
id = 25198596, id2 = 7, filesize = 0B
position of above data: 5472137216 - 5472137728
Magic Number found. This is maybe inode !!
id = 29392900, id2 = 8, filesize = 0B
position of above data: 5472139264 - 5472139776
Magic Number found. This is maybe inode !!
id = 33587204, id2 = 9, filesize = 0B
position of above data: 5472141312 - 5472141824
Magic Number found. This is maybe inode !!
id = 37781508, id2 = 10, filesize = 0B
position of above data: 5472143360 - 5472143872
Magic Number found. This is maybe inode !!
id = 41975812, id2 = 11, filesize = 0B
position of above data: 5472145408 - 5472145920
Magic Number found. This is maybe inode !!
id = 46170116, id2 = 12, filesize = 0B
position of above data: 5472147456 - 5472147968
Magic Number found. This is maybe inode !!
id = 50364420, id2 = 14, filesize = 0B
position of above data: 5472149504 - 5472150016
Magic Number found. This is maybe inode !!
id = 54558724, id2 = 15, filesize = 0B
position of above data: 5472151552 - 5472152064
Magic Number found. This is maybe inode !!
id = 58753028, id2 = 17, filesize = 0B
position of above data: 5472153600 - 5472154112
Magic Number found. This is maybe inode !!
id = 62947332, id2 = 18, filesize = 0B
position of above data: 5472155648 - 5472156160
Magic Number found. This is maybe inode !!
id = 67141636, id2 = 20, filesize = 1.4kB
position of above data: 5472157696 - 5472158208
Magic Number found. This is maybe inode !!
id = 71335940, id2 = 21, filesize = 2.6kB
position of above data: 5472159744 - 5472160256
Magic Number found. This is maybe inode !!
id = 75530244, id2 = 22, filesize = 261B
position of above data: 5472161792 - 5472162304
Magic Number found. This is maybe inode !!
id = 79724548, id2 = 23, filesize = 0B
position of above data: 5472163840 - 5472164352
Magic Number found. This is maybe inode !!
id = 83918852, id2 = 24, filesize = 10.0GB
position of above data: 5472165888 - 5472166400
Magic Number found. This is maybe inode !!
id = 88113156, id2 = 25, filesize = 493B
position of above data: 5472167936 - 5472168448
Magic Number found. This is maybe inode !!
id = 92307460, id2 = 26, filesize = 0B
position of above data: 5472169984 - 5472170496
Magic Number found. This is maybe inode !!
id = 96501764, id2 = 27, filesize = 64.8kB
position of above data: 5472172032 - 5472172544
Magic Number found. This is maybe inode !!
id = 100696068, id2 = 28, filesize = 0B
position of above data: 5472174080 - 5472174592
Magic Number found. This is maybe inode !!
id = 104890372, id2 = 29, filesize = 8.5kB
position of above data: 5472176128 - 5472176640
Magic Number found. This is maybe inode !!
id = 109084676, id2 = 30, filesize = 0B
position of above data: 5472178176 - 5472178688
Magic Number found. This is maybe inode !!
id = 113278980, id2 = 31, filesize = 123.1kB
position of above data: 5472180224 - 5472180736
Magic Number found. This is maybe inode !!
id = 117473284, id2 = 33, filesize = 1.2kB
position of above data: 5472182272 - 5472182784
Magic Number found. This is maybe inode !!
id = 121667588, id2 = 34, filesize = 2.5kB
position of above data: 5472184320 - 5472184832
Magic Number found. This is maybe inode !!
id = 125861892, id2 = 35, filesize = 261B
position of above data: 5472186368 - 5472186880
Magic Number found. This is maybe inode !!
id = 130056196, id2 = 36, filesize = 0B
position of above data: 5472188416 - 5472188928
Magic Number found. This is maybe inode !!
id = 134250500, id2 = 37, filesize = 10.0GB
position of above data: 5472190464 - 5472190976
Magic Number found. This is maybe inode !!
id = 138444804, id2 = 38, filesize = 493B
position of above data: 5472192512 - 5472193024
Magic Number found. This is maybe inode !!
id = 142639108, id2 = 39, filesize = 0B
position of above data: 5472194560 - 5472195072
Magic Number found. This is maybe inode !!
id = 146833412, id2 = 40, filesize = 119.8kB
position of above data: 5472196608 - 5472197120
Magic Number found. This is maybe inode !!
id = 151027716, id2 = 41, filesize = 0B
position of above data: 5472198656 - 5472199168
Magic Number found. This is maybe inode !!
id = 155222020, id2 = 42, filesize = 8.5kB
position of above data: 5472200704 - 5472201216
^CTraceback (most recent call last):
File "searchinode.py", line 66, in <module>
data = f.read(512)
KeyboardInterrupt
I found something like that, so I stopped.
If you look at the size, there is something that shows 10.0GB, so this is
It looks like an inode in the <vmname> -flat.vmdk file.
I can't keep up with how it is mapped to the actual block,
This \ <vmname >-flat.vmdk inode at 0x400 is
For pointer block id (in vmfs-tools), read the pointer block for that id
As a result of various processing, it seems that the actual block can be found.
Anyway, it seems to be mapping using this area, so It seems that it can be recovered if such information remains after it is deleted.
Let's check this inode from the ESXi shell once.
If 83918852 and 134250500 are <vmname> -flat.vmdk, that's what you expected.
inode display
# pwd
/vmfs/volumes/datastore1
# ls -li */*vmdk
83918852 -rw------- 1 root root 10737418240 Jun 13 13:31 test01/test01-flat.vmdk
88113156 -rw------- 1 root root 493 Jun 13 13:03 test01/test01.vmdk
134250500 -rw------- 1 root root 10737418240 Jun 13 13:57 test02/test02-flat.vmdk
138444804 -rw------- 1 root root 493 Jun 13 13:38 test02/test02.vmdk
There seems to be no fatal misunderstanding so far. Delete it immediately and check the data.
Connect with the vSphere Client and delete with Delete from Disk.
It seems that vmdk cannot be accessed when guest ESXi is running, so shut it down and check.
Deleted from disk
# pwd
/vmfs/volumes/datastore1
# ls -a
. .. .fbb.sf .fdc.sf .pb2.sf .pbc.sf .sbc.sf .vh.sf
Compare with the data before deletion.
Inode information before deletion
## <vmname>-flat.inode indicating vmdk
# hexdump -C -s 5472165888 -n 2048 __backup_vsphere-flat.vmdk
1462aa000 01 00 c0 10 00 a0 ea 0c 00 00 00 00 00 00 3e 00 |..............>.|
1462aa010 00 00 00 00 1f 00 00 00 00 00 00 00 20 00 00 00 |............ ...|
1462aa020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
1462aa030 00 00 00 00 00 00 00 00 91 05 00 00 00 00 00 00 |................|
1462aa040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
1462aa200 04 80 00 05 18 00 00 00 01 00 00 00 03 00 00 00 |................|
1462aa210 00 00 00 00 00 00 00 80 02 00 00 00 00 00 10 00 |................|
1462aa220 00 00 00 00 52 04 00 00 00 00 00 00 4d fd 9a 53 |....R.......M..S|
1462aa230 f6 f8 9a 53 4d fd 9a 53 00 00 00 00 00 00 00 00 |...SM..S........|
1462aa240 80 01 00 00 03 00 00 00 00 00 00 00 00 00 00 00 |................|
1462aa250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
1462aa400 03 ed 02 10 03 ed 02 30 03 ed 02 40 03 ed 02 50 |[email protected]|
1462aa410 03 ed 02 60 03 ed 02 70 03 ed 02 80 03 ed 02 90 |...`...p........|
1462aa420 03 ed 02 a0 03 ed 02 20 00 00 00 00 00 00 00 00 |....... ........|
1462aa430 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
1462aa800
Inode information after deletion
##I couldn't skip directly with hexdump on the vSphere shell, so skip it with dd and then pass it
## <vmname>-flat.inode indicating vmdk
# dd if=vsphere-flat.vmdk bs=512 skip=10687824 count=4
2> /dev/null | hexdump -C
00000000 01 00 c0 10 00 a0 ea 0c 00 00 00 00 00 f0 3d 00 |..............=.|
00000010 00 00 00 00 11 00 00 00 00 00 00 00 24 00 00 00 |............$...|
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000030 00 00 00 00 00 00 00 00 d3 00 00 00 00 00 00 00 |................|
00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000200 04 80 00 05 18 00 00 00 00 00 00 00 03 00 00 00 |................|
00000210 00 00 00 00 00 00 00 00 00 00 00 00 00 00 10 00 |................|
00000220 00 00 00 00 00 00 00 00 00 00 00 00 4d fd 9a 53 |............M..S|
00000230 e8 a9 9d 53 e1 7b 9d 53 00 00 00 00 00 00 00 00 |...S.{.S........|
00000240 80 01 00 00 03 00 00 00 00 00 00 00 00 00 00 00 |................|
00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000800
The location that is supposed to be used for mapping has been overwritten with 0s. ... I can't handle this, let's give up. And let's go to bed.
The small files seemed to fit directly in the inode I also checked that, but the data was found as it was in this file.
Check the small file inode from the deleted vmdk
## <vmname>.vmdk (descriptor)Indicates an inode
# dd if=vsphere-flat.vmdk bs=512 skip=10687828 count=4 2> /dev/null | hexdump -C
00000000 01 00 c0 10 00 a8 ea 0c 00 00 00 00 00 f0 3d 00 |..............=.|
00000010 00 00 00 00 11 00 00 00 00 00 00 00 2b 00 00 00 |............+...|
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000030 00 00 00 00 00 00 00 00 d9 00 00 00 00 00 00 00 |................|
00000040 00 00 00 00 00 00 3e 00 00 00 00 00 1f 00 00 00 |......>.........|
00000050 00 00 00 00 ff e6 9a 53 cf 54 c0 67 9d 62 00 0c |.......S.T.g.b..|
00000060 29 d5 43 a8 00 00 00 00 00 00 00 00 00 00 00 00 |).C.............|
00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000200 04 80 40 05 19 00 00 00 00 00 00 00 03 00 00 00 |..@.............|
00000210 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 00 |............. ..|
00000220 00 00 00 00 00 00 00 00 00 00 00 00 a9 f6 9a 53 |...............S|
00000230 8e f5 9a 53 a9 f6 9a 53 00 00 00 00 00 00 00 00 |...S...S........|
00000240 80 01 00 00 d1 10 00 00 00 00 00 00 00 00 00 00 |................|
00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000400 23 20 44 69 73 6b 20 44 65 73 63 72 69 70 74 6f |# Disk Descripto|
00000410 72 46 69 6c 65 0a 76 65 72 73 69 6f 6e 3d 31 0a |rFile.version=1.|
00000420 65 6e 63 6f 64 69 6e 67 3d 22 55 54 46 2d 38 22 |encoding="UTF-8"|
00000430 0a 43 49 44 3d 35 62 61 64 65 30 63 31 0a 70 61 |.CID=5bade0c1.pa|
00000440 72 65 6e 74 43 49 44 3d 66 66 66 66 66 66 66 66 |rentCID=ffffffff|
00000450 0a 69 73 4e 61 74 69 76 65 53 6e 61 70 73 68 6f |.isNativeSnapsho|
00000460 74 3d 22 6e 6f 22 0a 63 72 65 61 74 65 54 79 70 |t="no".createTyp|
00000470 65 3d 22 76 6d 66 73 22 0a 0a 23 20 45 78 74 65 |e="vmfs"..# Exte|
00000480 6e 74 20 64 65 73 63 72 69 70 74 69 6f 6e 0a 52 |nt description.R|
00000490 57 20 32 30 39 37 31 35 32 30 20 56 4d 46 53 20 |W 20971520 VMFS |
000004a0 22 74 65 73 74 30 31 2d 66 6c 61 74 2e 76 6d 64 |"test01-flat.vmd|
000004b0 6b 22 0a 0a 23 20 54 68 65 20 44 69 73 6b 20 44 |k"..# The Disk D|
000004c0 61 74 61 20 42 61 73 65 20 0a 23 44 44 42 0a 0a |ata Base .#DDB..|
000004d0 64 64 62 2e 61 64 61 70 74 65 72 54 79 70 65 20 |ddb.adapterType |
000004e0 3d 20 22 6c 73 69 6c 6f 67 69 63 22 0a 64 64 62 |= "lsilogic".ddb|
000004f0 2e 74 68 69 6e 50 72 6f 76 69 73 69 6f 6e 65 64 |.thinProvisioned|
00000500 20 3d 20 22 31 22 0a 64 64 62 2e 67 65 6f 6d 65 | = "1".ddb.geome|
00000510 74 72 79 2e 73 65 63 74 6f 72 73 20 3d 20 22 36 |try.sectors = "6|
00000520 33 22 0a 64 64 62 2e 67 65 6f 6d 65 74 72 79 2e |3".ddb.geometry.|
00000530 68 65 61 64 73 20 3d 20 22 32 35 35 22 0a 64 64 |heads = "255".dd|
00000540 62 2e 67 65 6f 6d 65 74 72 79 2e 63 79 6c 69 6e |b.geometry.cylin|
00000550 64 65 72 73 20 3d 20 22 31 33 30 35 22 0a 64 64 |ders = "1305".dd|
00000560 62 2e 75 75 69 64 20 3d 20 22 36 30 20 30 30 20 |b.uuid = "60 00 |
00000570 43 32 20 39 64 20 33 35 20 63 37 20 30 39 20 63 |C2 9d 35 c7 09 c|
00000580 66 2d 63 66 20 64 34 20 30 65 20 61 33 20 32 37 |f-cf d4 0e a3 27|
00000590 20 36 35 20 39 38 20 62 62 22 0a 64 64 62 2e 6c | 65 98 bb".ddb.l|
000005a0 6f 6e 67 43 6f 6e 74 65 6e 74 49 44 20 3d 20 22 |ongContentID = "|
000005b0 37 64 35 30 66 36 36 32 33 30 37 32 30 38 37 61 |7d50f6623072087a|
000005c0 37 31 64 36 61 63 63 39 35 62 61 64 65 30 63 31 |71d6acc95bade0c1|
000005d0 22 0a 64 64 62 2e 76 69 72 74 75 61 6c 48 57 56 |".ddb.virtualHWV|
000005e0 65 72 73 69 6f 6e 20 3d 20 22 38 22 0a 00 00 00 |ersion = "8"....|
000005f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000800
Copy and investigate vmdk of guest ESXi after deletion The magic number of the inode can be found, so although the size is 0 bytes It seems that small files can be recovered by searching from one end.
Search for inodes after removal from disk
# python searchinode.py
Magic Number found. This is maybe inode !!
id = 4, id2 = 1, filesize = 1.1kB
position of above data: 5259788288 - 5259788800
Magic Number found. This is maybe inode !!
id = 4194308, id2 = 1, filesize = 256.0kB
position of above data: 5259790336 - 5259790848
Magic Number found. This is maybe inode !!
id = 8388612, id2 = 1, filesize = 254.7MB
position of above data: 5259792384 - 5259792896
Magic Number found. This is maybe inode !!
id = 12582916, id2 = 1, filesize = 256.0MB
position of above data: 5259794432 - 5259794944
Magic Number found. This is maybe inode !!
id = 16777220, id2 = 1, filesize = 250.6MB
position of above data: 5259796480 - 5259796992
Magic Number found. This is maybe inode !!
id = 20971524, id2 = 1, filesize = 4.0MB
position of above data: 5259798528 - 5259799040
Magic Number found. This is maybe inode !!
id = 25165828, id2 = 1, filesize = 1.1MB
position of above data: 5259800576 - 5259801088
Magic Number found. This is maybe inode !!
id = 32772, id2 = 1, filesize = 0B
position of above data: 5472124928 - 5472125440
Magic Number found. This is maybe inode !!
id = 4227076, id2 = 2, filesize = 0B
position of above data: 5472126976 - 5472127488
Magic Number found. This is maybe inode !!
id = 8421380, id2 = 3, filesize = 0B
position of above data: 5472129024 - 5472129536
Magic Number found. This is maybe inode !!
id = 12615684, id2 = 4, filesize = 0B
position of above data: 5472131072 - 5472131584
Magic Number found. This is maybe inode !!
id = 16809988, id2 = 5, filesize = 0B
position of above data: 5472133120 - 5472133632
Magic Number found. This is maybe inode !!
id = 21004292, id2 = 6, filesize = 0B
position of above data: 5472135168 - 5472135680
Magic Number found. This is maybe inode !!
id = 25198596, id2 = 7, filesize = 0B
position of above data: 5472137216 - 5472137728
Magic Number found. This is maybe inode !!
id = 29392900, id2 = 8, filesize = 0B
position of above data: 5472139264 - 5472139776
Magic Number found. This is maybe inode !!
id = 33587204, id2 = 9, filesize = 0B
position of above data: 5472141312 - 5472141824
Magic Number found. This is maybe inode !!
id = 37781508, id2 = 10, filesize = 0B
position of above data: 5472143360 - 5472143872
Magic Number found. This is maybe inode !!
id = 41975812, id2 = 11, filesize = 0B
position of above data: 5472145408 - 5472145920
Magic Number found. This is maybe inode !!
id = 46170116, id2 = 12, filesize = 0B
position of above data: 5472147456 - 5472147968
Magic Number found. This is maybe inode !!
id = 50364420, id2 = 14, filesize = 0B
position of above data: 5472149504 - 5472150016
Magic Number found. This is maybe inode !!
id = 54558724, id2 = 15, filesize = 0B
position of above data: 5472151552 - 5472152064
Magic Number found. This is maybe inode !!
id = 58753028, id2 = 17, filesize = 0B
position of above data: 5472153600 - 5472154112
Magic Number found. This is maybe inode !!
id = 62947332, id2 = 18, filesize = 0B
position of above data: 5472155648 - 5472156160
Magic Number found. This is maybe inode !!
id = 67141636, id2 = 20, filesize = 0B
position of above data: 5472157696 - 5472158208
Magic Number found. This is maybe inode !!
id = 71335940, id2 = 21, filesize = 0B
position of above data: 5472159744 - 5472160256
Magic Number found. This is maybe inode !!
id = 75530244, id2 = 22, filesize = 0B
position of above data: 5472161792 - 5472162304
Magic Number found. This is maybe inode !!
id = 79724548, id2 = 23, filesize = 0B
position of above data: 5472163840 - 5472164352
Magic Number found. This is maybe inode !!
id = 83918852, id2 = 24, filesize = 0B
position of above data: 5472165888 - 5472166400
Magic Number found. This is maybe inode !!
id = 88113156, id2 = 25, filesize = 0B
position of above data: 5472167936 - 5472168448
Magic Number found. This is maybe inode !!
id = 92307460, id2 = 26, filesize = 0B
position of above data: 5472169984 - 5472170496
Magic Number found. This is maybe inode !!
id = 96501764, id2 = 27, filesize = 0B
position of above data: 5472172032 - 5472172544
Magic Number found. This is maybe inode !!
id = 100696068, id2 = 28, filesize = 0B
position of above data: 5472174080 - 5472174592
Magic Number found. This is maybe inode !!
id = 104890372, id2 = 29, filesize = 0B
position of above data: 5472176128 - 5472176640
Magic Number found. This is maybe inode !!
id = 109084676, id2 = 30, filesize = 0B
position of above data: 5472178176 - 5472178688
Magic Number found. This is maybe inode !!
id = 113278980, id2 = 31, filesize = 0B
position of above data: 5472180224 - 5472180736
Magic Number found. This is maybe inode !!
id = 117473284, id2 = 33, filesize = 0B
position of above data: 5472182272 - 5472182784
Magic Number found. This is maybe inode !!
id = 121667588, id2 = 34, filesize = 0B
position of above data: 5472184320 - 5472184832
Magic Number found. This is maybe inode !!
id = 125861892, id2 = 35, filesize = 0B
position of above data: 5472186368 - 5472186880
Magic Number found. This is maybe inode !!
id = 130056196, id2 = 36, filesize = 0B
position of above data: 5472188416 - 5472188928
Magic Number found. This is maybe inode !!
id = 134250500, id2 = 37, filesize = 0B
position of above data: 5472190464 - 5472190976
Magic Number found. This is maybe inode !!
id = 138444804, id2 = 38, filesize = 0B
position of above data: 5472192512 - 5472193024
Magic Number found. This is maybe inode !!
id = 142639108, id2 = 39, filesize = 0B
position of above data: 5472194560 - 5472195072
Magic Number found. This is maybe inode !!
id = 146833412, id2 = 40, filesize = 0B
position of above data: 5472196608 - 5472197120
Magic Number found. This is maybe inode !!
id = 151027716, id2 = 41, filesize = 0B
position of above data: 5472198656 - 5472199168
Magic Number found. This is maybe inode !!
id = 155222020, id2 = 42, filesize = 0B
position of above data: 5472200704 - 5472201216
Magic Number found. This is maybe inode !!
id = 159416324, id2 = 43, filesize = 0B
position of above data: 5472202752 - 5472203264
Magic Number found. This is maybe inode !!
id = 163610628, id2 = 44, filesize = 0B
position of above data: 5472204800 - 5472205312
Magic Number found. This is maybe inode !!
id = 167804932, id2 = 45, filesize = 0B
position of above data: 5472206848 - 5472207360
Magic Number found. This is maybe inode !!
id = 171999236, id2 = 46, filesize = 0B
position of above data: 5472208896 - 5472209408
Magic Number found. This is maybe inode !!
id = 176193540, id2 = 47, filesize = 0B
position of above data: 5472210944 - 5472211456
Magic Number found. This is maybe inode !!
id = 180387844, id2 = 49, filesize = 0B
position of above data: 5472212992 - 5472213504
Magic Number found. This is maybe inode !!
id = 184582148, id2 = 50, filesize = 0B
position of above data: 5472215040 - 5472215552
Magic Number found. This is maybe inode !!
id = 188776452, id2 = 51, filesize = 0B
position of above data: 5472217088 - 5472217600
Magic Number found. This is maybe inode !!
id = 192970756, id2 = 52, filesize = 0B
position of above data: 5472219136 - 5472219648
Magic Number found. This is maybe inode !!
id = 197165060, id2 = 53, filesize = 0B
position of above data: 5472221184 - 5472221696
Magic Number found. This is maybe inode !!
id = 201359364, id2 = 54, filesize = 0B
position of above data: 5472223232 - 5472223744
^CTraceback (most recent call last):
File "searchinode.py", line 64, in <module>
data = f.read(512)
KeyboardInterrupt
Summary
- The data itself is probably not overwritten (when deleted)
- Inode information is partially erased
- You can retrieve files that are small in size (1kB or less?) And that fit the data in the inode.
(About .vmxf or .vmdk for virtual disk descriptor file) - The VMDK file for
<vmname> -flat.vmdkis almost impossible because the information of the corresponding block of inode is deleted. - If it is about MBR, it can be found and recovered directly regardless of the inode.
- Make a backup of your important data
Reference material
VMFS related
- http://glandium.org/projects/vmfs-tools/
- http://buildvirtual.net/using-vmkfstools-to-manage-vmfs-datastores/
MBR related
- http://ja.wikipedia.org/wiki/ Master boot record
- http://thestarman.narod.ru/asm/mbr/index.html
GPT related
- http://ja.wikipedia.org/wiki/GUID Partition Table