When I first started using zones on Solaris, I ran into networking difficulty. I didn’t want private zone traffic polluting the network at the ISP, and I didn’t need any of the zones to be directly exposed (all traffic could either be proxied or go through ipnat). There was no nice solution for doing this, so I had to do something that involved turning on routing and doing weird things with the arp cache. I forget the specifics, and really, the only reason to remember them would be to do something similar on a different platform if forced to.

About 2 years ago, Solaris added Crossbow network virtualization system to the feature set. This is very nice, and extremely simple. For a bit machine, it would be possible to create virtual networks that only some zones can use and not others. For instance, a customer with 10 zones could have their zones talk among themselves but not to another customers zones. It also makes it possible to control network profile of zones, for instance rate limiting and applying a quota to them.

For a small installation like mine, it just makes it easy to do the right thing. In the root zone:

dladmin create-etherstub etherstub0

Now, etherstub0 is your new private network. To attach devices to that network, do:

dladmin create-vnic -l etherstub0 vnic0

I use vnic0 in the root zone, then configure vnic1-N for the other containers. If you do this before creating the zone, then in the zone config you just do this:

set ip-type=exclusive
add net
set physical=vnic6
end

For a zone that is already setup, you have to alter the config while it is stopped. But you will also need to plumb that zones vnic interface and create /etc/hostname.vnicN. Like wise, in the root zone, if you have a vnic into the virtual network there, you will also need to plumb and create the hostname file for that vnic.

For my server, I have a single IP address. So, what I do is have ipnat on the root zone forward ports to specific zones on the virtual network. If you have multiple IPs, you could use routing from the root zone to other zones. However, it would probably be more efficient to use a virtual private network for interzone traffic, and also give each zone a vnic attached directly to the main interface with a command like:

dladm create-vnic -l bge0 vnic22

I’m sure there are many other possibilities that I’m not aware of.