Search This Blog

Wednesday, August 20, 2014

Using USB Devices on Solaris (ZFS)

UPDATE:
Solaris 11 does not support the ZFS filesystem provided by 'native' ZFS for linux, and vice-versa.  Thus, you should export your data as NTFS so that everyone is happy, as NTFS is the most supported filesystem for 'read-only'.  Otherwise it's Solaris->Solaris only.  I have not tested native ZFS with import/export, but i suspect the version of zpool (5000 or something like that) will work fine between fedora type distros and ubuntu.


Recently I needed to export a large amount of data on our Solaris NFS Server.  However, getting this information straight off this robust server is not as intuitive or straight forward as you might first think.  Many filesystems are not natively supported by solaris, thus can cause a lot of headache trying to figure out how to use fdisk and format.  Additionally, with block sizes of 4096 (like on a Seagate 3TB hard drive) it may not even be compatible with UFS.

Before you begin, take a look at this chart on wikipedia which breaks down the various versions of zpool and zfs.  This will only work if you are using the proper version of zpool/zfs between hosts.  Currently the Native Linux ZFS project uses a zpool version not support by Solaris 11 and vice versa.  So ZFS cannot be imported between hosts: http://en.wikipedia.org/wiki/ZFS#List_of_operating_systems_supporting_ZFS

ZFS is a good solution and can be imported on other linux distributions such as CentOS or Ubuntu.  Here is a synopsis in which I exported a large amount of data with the label "TwitterFeeds"

List available drives
$ format -e
...    other disks likely shown here...
54. c9t0d0 <Seagate-Expansion Desk-0604 cyl 45597 alt 2 hd 255 sec 63> /pci@0,0/pci108e,cb84@2,1/hub@6/storage@2/disk@0,0
          /pci@0,0/pci108e,cb84@2,1/hub@6/storage@2/disk@0,0
Created a ZFS pool on the USB drive, added a ZFS file system, chowned it with my default user and started rsync with nohup to log (rsync.out)
$ sudo zpool create TwitterFeeds c9t0d0
$ sudo zfs create TwitterFeeds/export 
$ sudo chown -R user:group /TwitterFeeds/export
$ nohup rsync -r --progress /sasdata/TwitterFeeds /TwitterFeeds/export > rsync.out 2>&1&
Example entry of output in rsync log:
bytesize Percent% xferRate Time(file#, to-check=filesremain/estimatedtotal)Absolute/File/Path/Filename.ext
805313211 100%   29.23MB/s    0:00:26 (xfer#364, to-check=1008/1417)TwitterFeeds/08/19/2014/05/17/22/10FEB25021318-S3DM_R5C4-053771096010_01_P001.DAT
You can use tail -f on the rsync log to periodically view progress.
$ tail -f /TwitterFeeds/export/rsync.out
Once rsync has completed, the ZFS File System and Zpool are unmounted and removed from your available zpools.
$ sudo zpool export TwitterFeeds
The USB Drive can now be physically removed and plugged into another computer.  Plug USB Drive in to target system.  ZFS detects any moved or renamed devices, and adjusts the configuration appropriately. To discover available pools, run the zpool import command with no options. To import a pool, specify the name as an argument to the import command (TwitterFeeds).  By default, the zpool import command only searches devices within the /dev/dsk directory. If devices exist in another directory, or you are using pools backed by files, you must use the -d option to search alternate directories.  This may be required when using CentOS/Ubuntu with ZFS.
$ zpool import TwitterFeeds
$ zpool import -d /dev/rdsk/c9t0d0 TwitterFeeds #specifying disk device path manually
Once Imported, "$ sudo zfs get all" should show a PROPERTY mount point for the zpool to access the data. From this point you should be able to use native OS filesystem utils like cp, rm, chmod, and others.

Reference Links:
Managing ZFS Storage Pools
Managing ZFS File Systems (Not the Physical Devices!)
Using ZFS on Linux

No comments:

Post a Comment