Virtual Nerd'ing

[ non-geeks need to look away - notes about virtualisation on Linux ]



Years before I became a photo nerd, I was already a computer nerd with C/C++ and UNIX being favourite no-girlfriend past-times. Roll forward the years and now I can both.

After years of hijacking other people's cast off PCs to build various NFS/SMB file servers, I finally got a new machine that could serve my current requirements to run a virtualised Windows/Solaris environment. The hardware chosen was a 4x core 64bit Intel i5-760 2.8Ghz Dell Vostro 430 shipped with Windows 7 x64 on a 160GB Samsung HD161GJ disk.

The task list:

• Linux install/post install
• Virtualisation: VMware
• Windows 7
• SunOS/Solaris

Hardware

The receipt of the machine was a bit of a pain (noisy CPU and graphic card fans which had replacement parts sent to me by Dell) but was eventually resolved. However, I was a little surprised to see the internal frame was the 'old' screw/slots arrangement. Most modern PCs (certainly the Dells I've used before) have rails for additional harddisks etc but the Vostro case didn't.

I added another 1.5TB Seagate Barracuda 7200.11 ST31500341AS drive (~57 GBP delivered) and a single DIMM 4GB of 240pin unbuffered non-ECC DDR3-1333mhz PC3 RAM (~49 GBP delivered). I seem to recall that for same expansions through Dell would have cost in the region of 400 GBP.

The additional disk became /dev/sdb and part of it would be home to the 64bit Linux/Fedora 14 installation. The grub boot loader was manually installed on /dev/sdb via the rescue mode of the Fedora 14 installation DVD. To boot Linux, I would force the BIOS boot sequence (F12) to display all attached devices for selection and the second disk was selected: this was done to ensure that original /dev/sda device was untouched.

LVM

One note for installation is that you should almost certainly use LVM - LVM will allow for non-destructive resizing of logical volumes (or 'partitions' in old money) after the initial installation: it is a huge pain for the selection of packages to be complete only for the installer to baulk on lack of space for the target partition.

Reducing logical volumes can be done only with the LV unmounted, whilst extensions can be done online. The practicalities of reducing your LVs include booting the Fedora bootable DVD selecting the rescue option and skipping auto mounting of local partitions. At the shell, activate and verify the volume group (including free extents) and also the logical volumes:

lvm vgchange -a y
lvm vgdisplay
lvm lvdisplay

The output will also show then know volume names, /dev/vg/lv_tmp, but it is the /dev/mapper equivalents that the LVM operations will be applied. With the current iteration of Fedora/LVM, the reduction of space in the LV (as reported from lvdisplay) /dev/vg/lv_tmprequires:

umount /dev/mapper/vg-lv_tmp
e2fsck -f /dev/mapper/vg-lv_tmp
lvm lvreduce -r -L 1GB /dev/mapper/vg-lv_tmp

The example reduces the LV to a total of 1GB in size - it is worth noting that the operations are now simplified in that no separate resize2fs call is required, with the LVM tools (the -r flag) taking care of the necessary.

Linux: Post Installation

BIOS Boot Problems

The grub boot loader was configured with the root partition hd1,7 (/dev/sdb8) but as grub tried to access this device, it claimed that no such partition existed (Error 22: No such partition). After a little digging around, it was discovered that the BIOS determined that the harddisk selected to boot (in the case /dev/sdb) was actually treated as the first device: to grub, that meant /dev/sdb was actually hd0.

Fixing this in grub.conf was simple (root(hd0,7)). Note however, once the boot process begins, the BIOS and Linux will recognise that the real location of the root disk on /dev/sdb.

nVidia Binary Drivers

Fedora will install the nouveau opensource driver to run the graphics card. However, there are a couple of issues when using the opensource driver as apposed to nVidia's own drivers (v304.43):

• no 3D h/w acceleration
• no GPU fan control

Whilst the first point is not a problem, the latter is desirable since the GPU fan, by default spins up to full speed all the time which is noisy and likely to shorten the lifespan.

Installation of nVidia's drivers are relatively simple:

• disable the nouveau driver from loading at boot time/blacklist in modprobe
• as root, stop X and the NVIDIA-Linux-x86_64-xxx.yy.run

The second step will automatically build and install the new drivers. Details of these items are documented in the nVidia binary driver package. With the older drivers (v260.xx), once installed there was a noticable performance degradation when switching between windows in X. This was resolved by adding the line

/usr/bin/nvidia-settings \
-a InitialPixmapPlacement=2 \
-a GlyphCache=1

to /etc/gdm/PostLogin/Default which will ensure the required setting is applied on each successful X login, AND by updating the cairo runtime libraries (cairo-1.10.0-2 worked for me). With the additional update, switching between X appls (noticably firefox) was smoother.

Dual Displays

The easiest way to configure this is via nvidia-settings although make sure you use TwinView instead of the "Use xinerama" tickbox. For monitors that do not share the same resolution, the nvidia TwinView MetaMode options allow you to specify panning for the smaller resolution via the "@" notion:

Option "metamodes" "DFP: 1920x1200 +0+0, CRT: 1280x1024 @1280x1200 +1920+88"

Misc Software

Once installed, Fedora 14 still maintains it's no mp3 support but this was easily resolved.

The initial step was to update the rpm src repositories' db to include rpmfusion:

rpm -ivh \
http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm

and then the installation of the relevant backends could begin:

yum install \
gstreamer-plugins-bad gstreamer-ffmpeg gstreamer-plugins-ugly \
mplayer \
-y

Other indespensible tools installed include:

• further Fedora RPMs: autofs, xinetd, telnetd (and adjusting SElinux)
• VLC - video/dvd player, libdvdcss, libdvdread
• handbrake - video trans/encoding util
• dvd region - tool for setting dvd drives' firmware region code
• rpm2tgz - tool to magically transfrom rpm packages to raw tar.gz files
• Sun Studio/dbx - the only debugger worth considering (at least on Sun Solaris)
• libmad, libid3tag, flickcurl, taglib, sox, urxvt
• Adobe FlashPlayer 64bit 11.1.102.63 - 11.2.202.235 causes rendering problems for me on youtube

The final item that was installed on the 32bit runtime linker, ld-linux.so.2 which I grabbed from a local mirror. The 32bit runtime linker is in the glibc package which was extracted by rpm2tgz and files copied manually. This has no impact on the 64bit linker/glibc as they named differently/reside in different locations (/lib for 32bit, /lib64 for 64bit). With this last change, the system can run 32bit Linux ELF binaries too.

Finally, two post install configuration items: replacing OpenJDK with Sun JRE and tweaking the GDM login prompt.

The OpenJDK installation worked for most part, but for some keyboard mapping issues in applets meant the switch to Sun's JRE. Once downloaded and unpacked to /usr/jre, switching firefox's JRE required:

cd /usr/lib64/mozilla/plugins && \
rm libjavaplugin.so && \
ln -s ../../../jre/lib/amd64/libnpjp2.so libjavaplugin.so

Firefox can also be upgraded direct from mozilla's release site - x64 binaries are now being provided as part of their builds and these seem to provide better support fro flickr/facebook picture uploading (no random crashing) but ensure that JRE 1.6u24 or later is used.

The GDM login prompt by default lists all users in a drop-down list which is unsightly if you have a large number of users; to disable:

gconftool-2 --config-source \
xml:readwrite:/etc/gconf/gconf.xml.defaults \
--direct --type bool --set \
/apps/gdm/simple-greeter/disable_user_list true

Virtualisation: VMware on Linux

Now, the real work.

One of the major items that I wanted to achieve was virtualisation under Linux, and being able to run Capture NX2 without having to reboot. This was now possible with the (relatively) bigger/faster host, VMware and Fedora 14/X11/GNOME supporting ICC monitor/display colour profiles.

The intention to run VMware meant that the CPU had to support hardware virtualisation: in this case, the Intel VT-x extensions.

There is one (VMware) concept that need to be stated upfront:

VMware needs to run the virtualised guest OS from it's own installation - you can NOT use an existing OS installation from another partition

In normal operation, the virtual machine is created and then the guest OS is installed within that virtual machine. However, VMware provides vCenter Converter 4.3 which can clone your already installed physical host and generate a virtual machine image (p2v). Note however, this transforms the existing OS including all of its files etc into virtual image.

The initial VMware virtual machine consists of the .vmx cfg file and the actual .vmdk disk image and this is loaded and booted by VMware. There are a number of different VMware products and for my use vmplayer was sufficent (and freely available). I had installed 3.1.2 which threw up a host of problems on installation (unable to compile the necessary vmmon/vmsock kernel modules without manual patching) but the 3.1.3 release fixes this issue.

A Window to Caputure NX

Once I had installed/configured Capture NX2 on the Windows 7 x64 OS, I installed vCenter Converter and generated the virtual machine image. This was done whilst Windows 7 was running and took about 1hr 20 for a disk that occupied ~50GB (the final image was ~35GB).

The virtual machine can then be booted by vmplayer.

As expected the converted physical image didn't boot succesfully the first time round - whilst the vmplayer process would start, Windows 7 would crash/blue screen at the splash page and subsequently report an error status: 0xc0000225. After more digging, I found references to the solution noted on the VMware community boards but reproduced below since I frequently find VMware's community boards down/inaccessible.

# Mount the Win7 DVD in VMware Player and boot to it.
1. At the first screen (Language Selection), hit Shift-F10 for a command prompt.
2. Run regedit
3. Load the system hive from the VM's disk:
1. Highlight HKEY_LOCAL_MACHINE
2. File > Load Hive
3. Select < c: > \Windows\System32\config\system (name it something like "asdf")
4. Expand HKEY_LOCAL_MACHINE\asdf\ControlSet1\Services\intelide
5. Change the data for value Start from 3 to 0
6. File > Unload Hive
7. Exit regedit
8. Reboot the VM

It would appear that the cloned physical image's Windows registry needed fixing such that all IDE 'devices' could be initialised correctly. This in my opinion is a bug within the VMware coverter.

Once this fix was applied, vmplayer was able to boot Windows 7. The Linux host was then configured with a SMB/CIFS server, exporting an area that was shared between the Linux host and the VMware guest OS and I was able to get files in/out of the VMware process.



If the Windows installation you have cloned doesn't provide boot disks (as per many OEM/pre-installed OS) then the Windows registry can be edited with the Offline NT passwd/Registry editor. VMware allows for virtual images to be mounted on the Linux host (menu: Virtual Machine settings -> hardware -> harddisk and through the utilities drop-down menu) and the Windows registry can be edited through this route.

Upon successful registry configuration and boot, I was advised to run, as administrator, from a cmd prompt:

takeown /F %WINDIR%\System32\sppcomapi.dll
icacls %WINDIR%\System32\sppcomapi.dll /deny *S-1-1-0:F

to complete the VMware/Windows setup.

One further note: I had initially tried running VMware with only 2GB of RAM and this killed the host even though I had limited the virtual machine to 1GB; with 6GB installed, the guest Windows 7 OS performance is indistinguisable.

More Geek-dom: Heading Towards the Sun

Around 2005, Sun's enterprise UNIX operating system, SunOS/Solaris, was made itself freely available for SPARC and x86/x64 based processors as Open Solaris. For a number of years I had wanted to get the SunOS development environment at home, but my legacy PC hardware (previous machine was a PII with 128MB RAM serving NFS/SMB) wasn't up to the task.

As of writing, OpenSolaris appears set to be discontinued, cancelled by Oracle (new owners of Sun), and is replaced by Sun Solaris Express which is strangely accessible via this url. The Solaris Express product will continue to be free but the open source development model has been retired replaced with a binary only distribution, which has long been the case with Sun Studio (the C/C++/dbx suite).

Whilst being a long time Linux user (starting back in my university days) SunOS is where I have spent the majority of my professional career: having dbx running on Linux (and crashing a lot even on startup for simple programs) just isn't the same.

With the 1.5TB disk capacity and the processing power, SunOS at home (even if it is initially via a VMware guest hosting Solaris) finally became a reality.

The Solaris VMware image was created with a max 16GB non pre-alloc'd image and the Solaris installation ISO (no need to burn the DVD) was booted through vmplayer. The installer provides different configurations for installation: the Entire Solaris Software Group installation option is most relevent, with a set of UFS (why ZFS??) filesystems were created: /, /usr and /var - note that there is no /tmp partition required as Solaris creates this a swap partition. The entire installation was under 9GB although the VMware image was ~3.5GB.

By default, the Solaris installation will include the X11 xdm greeter. As the Solaris host will be used as a server, this is not necessary and running X only serves to consume resources. Therefore, X can be disabled from automatically starting by issuing: dtconfig -d

Solaris to Linux

Connectivty to the Solaris host will be via telnet sessions with NFS serving filesystems between the hosts.

It is worth noting that Linux supports a broken version of NFSv4 - non-Linux hosts which mount the export v4 filesystems will find no access to files with ACL errors returned by the tools. This is a problem with the Linux implementation and the Linux host needs to disable exporting NFSv4 filesystems (/etc/sysconfig/nfs requires RPCNFSDARGS="-N 4"): NFSv3 works between Linux and Solaris.

Authentication between the host can be acheived via NIS, with the Linux host as an NIS master to the Solaris slave. There are noted problems with creating passwd/shadow entries as Solaris by default uses DES for passwd encrpytion and Fedora 14 uses SHA512: both systems can be updated to match - Solaris via /etc/security/policy.conf and Linux via authconfig

As the Linux host will always be available to Solaris host, the NIS master was hosted on the Linux box (this box will also be a NIS client to itself)

NIS: Linux Master

The first stage was to move all NIS authenticated users out of the /etc/passwd and /etc/shadow files into NIS only equivalents in /etc/yp. Whilst this isn't strictly necessary, I didn't want anything to screw up my shadow file. The etc/passwd had +:::::: added as the last line to instruct NIS entries to be checked.

/etc/yp.conf
domain nisdomain.com server elise.nisdomain.com

/etc/passwd
...
+::::::

/etc/yp/passwd
ray:x:500:100:Ray:/home/ray:/bin/bash

/etc/yp/group
users:x:100

/var/yp/Makefile
MINUID=500 # this has to match, some systms force higher userids
MINGID=100
...
YPSRCDIR = /etc
YPPWDDIR = /etc/yp
...
HOSTS = $(YPPWDDIR)/hosts

As the default NIS passwd and shadow files have been moved, /etc/sysconfig/yppasswdd (ETCDIR) and /var/yp/Makefile (YPPWDIR) need to be updated to reflect the NIS passwd/shadow file locations.

The /etc/nsswitch.conf requires to be updated such that:

passwd: compat files
shadow: compat files
group: compat files

hosts: files dns

ethers: nis [NOTFOUND=return] files
netmasks: nis [NOTFOUND=return] files
networks: nis [NOTFOUND=return] files
protocols: nis [NOTFOUND=return] files
rpc: nis [NOTFOUND=return] files
services: nis [NOTFOUND=return] files

bootparams: nis [NOTFOUND=return] files
netgroup: nis
publickey: nis

automount: files nis
aliases: files nis

The NIS domain name needed to be configured on the Linux master via adding NISDOMAIN=nisdomain.com in /etc/sysconfig/network and at this point, the initial NIS maps can be created:

cd /var/yp && mkdir nisdomain.com && make all

/etc/yp.conf was created with broadcast as the only line - forcing clients to search for master via the legacy broadcast requests.

The NIS services were started: ypserv, yppasswdd and ypbind.

To configure other linux NIS master-slaves, this guide provides some good background.

NIS: Solaris Client

The steps for the client were a lot more straight foward and required the following steps as root:

echo "nisdomain.com" > /etc/defaultdomain
domainname $(cat /etc/defaultdomain)
cp /etc/nsswitch.nis /etc/nsswitch.conf
cp /etc/passwd /etc/.passwd
echo "+::::::" >> /etc/passwd
cd /var/yp && ypinit -c
svcadm enable network/nis/client
svcadm restart network/nis/client

To verify the configuration, ensure ypbind is running and ypwhich can be run to determine the current NIS master that the client is bound: the NIS maps can be queried by ypcat (eg ypcat passwd or ypcat hosts)


With both hosts configured, NIS will be authenticating the users as well as providing the local net hosts resolution. One final note relates to updating NIS passwds - the Solaris passwd code will automatically determine whether the entry being changed is local/NIS and will accordingly call yppasswd: The Linux passwd is less intellegent and explicit invocation of yppasswd is required.

I never really ever stopped being a geek.