I was playing with my Oracle VM setup and needed some shared storage that was block based. I did not have a storage array available but I did have a solaris box, that I use for Oracle VDI, available.

I set up a few iscsi targets on this solaris server and exported them to my 2 Oracle VM servers. Here's how I did this :

(1) On the solaris side :

# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
rpool 544G 129G 415G 23% ONLINE -

I just have a simple zpool, called rpool, on this box. It has plenty of space available for my needs. So I will use rpool and I will create 5 50gb vols :

zfs create -V 50G rpool/ovm1
zfs create -V 50G rpool/ovm2
zfs create -V 50G rpool/ovm3
zfs create -V 50G rpool/ovm4
zfs create -V 50G rpool/ovm5

I want to use these volumes for iscsi so I have to enable them as shared iscsi devices :

zfs set shareiscsi=on rpool/ovm1
zfs set shareiscsi=on rpool/ovm2
zfs set shareiscsi=on rpool/ovm3
zfs set shareiscsi=on rpool/ovm4
zfs set shareiscsi=on rpool/ovm5

The command iscsitadm list target should list these devices so make sure they show up.

# iscsitadm list target
Target: rpool/ovm1
iSCSI Name: iqn.1986-03.com.sun:02:896c766c-0943-4da5-d47e-9575b5a0be36
Connections: 2
Target: rpool/ovm2
iSCSI Name: iqn.1986-03.com.sun:02:a3116b46-73e0-e8c2-e80c-9a4f71aff069
Connections: 2
Target: rpool/ovm3
iSCSI Name: iqn.1986-03.com.sun:02:a838c400-2730-c0d6-f2c2-ee186a0261c1
Connections: 2
Target: rpool/ovm4
iSCSI Name: iqn.1986-03.com.sun:02:2e046afb-d66d-4f3f-c5de-8115e0ddd931
Connections: 2
Target: rpool/ovm5
iSCSI Name: iqn.1986-03.com.sun:02:66109fbe-81ac-ef05-f85e-ab8c1f34cb43
Connections: 2

At this point I want to make sure that I have some access control on these devices. To make it easier, I will create an alias for my 2 servers and use the alias for the ACL.

get the iqn from the 2 servers
on my 2 ovm servers (wcoekaer-srv1, wcoekaer-srv2) get the content of
/etc/iscsi/initiatorname.iscsi (for each server)

InitiatorName=iqn.1986-03.com.sun:01:2a7526f0ffff

On the solaris side create the aliases :

iscsitadm create initiator -n iqn.1986-03.com.sun:01:2a7526f0ffff wcoekaer-srv1
iscsitadm create initiator -n iqn.1986-03.com.sun:01:e31b08110f1 wcoekaer-srv5

Add the ACL to the targets :

iscsitadm modify target -l wcoekaer-srv1 rpool/ovm1
iscsitadm modify target -l wcoekaer-srv1 rpool/ovm2
iscsitadm modify target -l wcoekaer-srv1 rpool/ovm3
iscsitadm modify target -l wcoekaer-srv1 rpool/ovm4
iscsitadm modify target -l wcoekaer-srv1 rpool/ovm5
iscsitadm modify target -l wcoekaer-srv5 rpool/ovm1
iscsitadm modify target -l wcoekaer-srv5 rpool/ovm2
iscsitadm modify target -l wcoekaer-srv5 rpool/ovm3
iscsitadm modify target -l wcoekaer-srv5 rpool/ovm4
iscsitadm modify target -l wcoekaer-srv5 rpool/ovm5

(2) the Oracle VM side

On each server just do 2 simple things :

# iscsiadm -m discovery -t sendtargets -p ca-vdi1 where ca-vdi1 is my solaris server name

# service iscsi restart

When I do cat /proc/partitions on my servers I will see the devices show up

# cat /proc/partitions
major minor #blocks name

8 0 160836480 sda
8 1 104391 sda1
8 2 3148740 sda2
8 3 1052257 sda3
253 0 6377804 dm-0
253 1 6377804 dm-1
253 2 6377804 dm-2
8 16 52428800 sdb
8 32 52428800 sdc
8 48 52428800 sdd
8 80 52428800 sdf
8 64 52428800 sde

These 5 new devices sd[b..f] are shared storage for Oracle VM and can be used to pass through to the VM's as phy: devices or put ocfs2 on it and use as shared filesystem storage for dom0 repositories.

I am setting up an 11gR2 rac template (the cool stuff Saar did) so I am using my devices to create a 2 node RAC cluster with phy: devices.