VM'ing: Copy music to/from IOS from Linux via VMs

Recently I've needed to spend more time using the Fedora installation on my aging 2010 Dell Vostro however I've found various occasions where I wanted to copy music files to my iPhone/IOS 9.x/10.x devices from Linux.

In years gone by Linux's recommended options for IOS music management included gtkpod, banshee and rhythmbox but none of these options work with my iPhone and Fedora. With my current Fedora 24 installation and IOS 9.x/10.x the only option would be to run a virtualised OS that supported iTunes via VMware or VirtualBox.

My past experience of creating a VM image from my running Windows 7 installation works but has a few drawbacks:

• it takes several hours to perform physical-to-virttual (p2v) conversion of your Windows OS to a VM disk image (using VMware converter)
• the p2v is a replica of your Windows OS drive, with no options to selectively choose components/exclude directories

The latter point is a potential killer for me: even with bare Windows 7 installation, my final VM disk image was > 30GB which might be too large (disk) penalty to simply copy music files to your iPhone. Obviously if you are already running with a VM image for other tasks (MS Office tools for instance) then this will not be a concern.

Note: VMware is only available for a 64bit host OS whereas VirtualBox supports both 32bit and 64bit host OSes.

Using a virtual disk image from Windows XP mode

One alternative for Windows 7 and 8 users (may be applicable to other new Windows versions) is to create your VM disk image from Windows 7's XP mode. Windows XP mode is provided by Microsoft as a way to install/run a 32bit WinXP Pro SP3 environment on your Windows 7 system via Windows virtualisation.

WinXP mode is provided as a ~470GB exe which includes a virtual hard disk image (.vhd) that can be extracted using Winzip (Windows) or 7zip /usr/bin/7za (Linux) and the disk image can be used by VMware and VirtualBox. Installation of WinXP mode is not needed; the exe file is an archive file that includes a file called VirtualXPVHD which is the .vhd image.

$ 7za l WindowsXPMode_en-us.exe
...
Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------ ------------------------
2009-09-15 21:57:46 ....A 100112 xpmmsilauncher.exe
2009-09-15 21:20:00 ....A 100640 xpmmsilauncher64.exe
2009-09-15 22:17:56 ....A 496218100 sources/xpm
2009-09-15 22:17:30 ....A 392704 sources/xpminstl32.msi
2009-09-15 21:45:56 ....A 393728 sources/xpminstl64.msi
------------------- ----- ------------ ------------ ------------------------
2009-09-15 22:17:56 497205284 492597008 5 files

$ 7za e WindowsXPMode_en-us.exe sources/xpm
$ 7za x xpm VirtualXPVHD

>

The advantage of using WinXP mode as the basis of your own VM is that the size of the resulting VM image after VMware/VirtualBox setup is ~4.5GB which is significantly favourable to the >30GB image from (my) Windows 7 p2v.

As with other 32bit Windows XP Pro installations however, there are some points to note:

• its 32bit which means max memory available to the OS will be 4GB
• CPU allocated by virtualisation software may be limited due to WinXP Pro limiting to 2x CPUs but ulimited number of cores within the CPUs
• can run (albeit a little slowly) with as little as 256MB of RAM

More recently, I've gone with a WinXP mode based VM image that I've installed my licensed copies of Nikon Capture NX2, MS Office 2007 and iTunes 12.x: the resulting disk image is ~5.4GB which is still very manageable.

VMTools and Guest Additions

Both VMware and VritualBox offer guest OS tools/drivers that make the Windows guest play nicer with the VM engine providing fairly common functionality such as ability to copy/paste from/to the guest/host OS as well as video and USB drivers.

VWWare has a set of VMTools that are supposed to auto-install. With VWware Workstation (aka WMPlayer) 12.5 I found this never to work - a popup would advise that VMware wanted to install/upgrade VMTools but it would fail. No other logs would advise of why it failed.

You can workaround this by manually pulling the VMTools directly from VMware's website, place on SMB or in the image and install from the exe. The latest Windows guest VMTools is available here

For the Windows 7 VM, the VMTools weren't absolutely necessary but this was a different story for WinXP Pro. Using both images on the same Fedora installation, the WinXP Pro VM was horrendously slow (moving explorer windows around the screen when there were no applications running took several seconds to draw/refresh) even when I had allocated 6 'cores' (of the 4x hyper-threaded cores of my i7 870). With the installation of VMTools and, perhaps a VMware friendly video driver, the latency problems were resolved and the VM instance was fine even when reduced to 2x 'cores'.

VirtualBox provides an all-in-one ISO CD image that appears bundled with its installation that can be installed via the VirtualBox menus.

Getting your Files into the VM

With the basic VM setup completed, we still need to get files in and out of the VM. Historically I have run a SMB/CIFS server on the host Linux box and mapped drives as needed.

One feature with the default samba 4.x/Fedora 24 configuration is executable files are not visible on the share until they have the execute bit set. However, even with this bit set, the Windows VM can NOT run the executable off the share so we have to copy the file(s) inside the local VM image which is obviously pointless.

To fix one of these issues, the /etc/samba/smb.conf can be updated with:

acl allow execute always = True

Note on WinXP, you will run into issues if you runas administrator off the network share in elevated mode but this by design.

Once the SMB share is available, this can be used for the incoming files.

Another way to get files into the VM is via the Shared Folders feature of VMware and
VirtualBox.

For VMware, the main settings on the host: VM->Settings->Options->Shared Folders -> Add. On the guest Windows OS, you map the network drive using the special target \\.host\sharename.

Similarly on VirtualBox, on the host: Devices->Install Guest Additions followed by Devices->Shared Folders. On the guest Windows OS, you map the network drive using the special target \\vboxsvr\sharename.

On various forums, there are many complains about VM shared folder performance but one reason you may wan use VM shared folders instead of SMB is to totally isolate the VM guest from outside world.

With my limited testing on VirtualBox using robocopy, extracted using 7za from Windows 2003 Resource kit's exe and embedded msi file (there's no .dll dependancies), copying data into the the VM was faster using Shared Folders compared to SMB shares: the same file from the same localhost was pulled.

Copying Music from/to iPhone from Linux

As noted at the top, earlier versions of IOS played nicely with a set of tools on Linux. This is no longer the case and whilst you can read back your music files and ringtones/voice memos its not always straight forward.

Pulling files from iPhone

With recent Fedoras and the work of libimobile and ifuse, you can traverse your IOS device. At the toplevel exists set a dirs:

• Recordings - ringtones and voice memos (a PITA to extract from Windows)
• iTunes_Control/Ringtones - install m4r ringtones
• iTunes_Control/Music - directory full of subdirectories called Fxx that hold your mp3s but with filesnames that do not match your originals

Knowing this, you can 'backup' your files off your iPhone - obviously there is some work to rename your mp3s (if you wish) but if the original included intact/correct ID3 tags then this is not such a problem since you can import the randomy-named files onto another IOS device and that device will use the ID3 tags to identify the music. Note however, if you import your music files and then retag/update the file information through iTunes, then this information is NOT reflected back in the mp3 files.

Putting files onto iPhone

And now, back to the original requirement and problem: putting music files onto my iPhone/iPod whilst in my Fedora environment.

This section will concentrate on using VMware or VirtualBox along with the WinXPmode vhd or a p2v OS Windows disk image and iTunes installed on each of those.

Note: iTunes for WinXP only supports up to IOS 9.x; the multiple platform iTunes download page is here

At this point unfortunately, this is another one of those occasions where Apples has restrictions on what we do with our own devices. All of my iPhones and ipods were initially setup such that I manually manage music and videos: this is tickbox that if changed will ERASE all your existing music/video files from your IOS device. Therefore, all my devices are setup like this during their original setups as I work on multiple machines, each connected to my NAS where my master music is kept. Even though each machine is pulling music files from the same source, I want to be able to add music from various machines.

Well, this is situation is fine for ipods/ipads but this is NOT how Apple designed it for iPhones. With your iPhone, you are forced to manage your iPhone from one master iTunes library - for me, that's my original Dell Windows 7 installation that I still use for photo/video editing. On that iTunes library, I have selected "manually manage music and videos". However if I attach my iPhone to any other iTunes, this option is deselected - selecting it will give the joyful message about removing all existing content and sync'ing with this (empty) library.

Great job Apple.

This dumb AF Apple design decision causes me a little bit of a problem since my neither my WinXPpro or Windows 7 VM images are my master iTunes library... so it looked like it was back to square one. However, one of the other utilities I bought many years ago was a tool called CopyTransContacts which I used to backup and reinstate my contact lists to my iPhone. Again another dumb AF Apple design decision preventing people to extracting their own data.

CopyTrans provide a free tool called CopyTrans Manager which is designed to manage your files on your IOS devices including copy files to and from the device! This is exactly the tool we needed: using CopyTrans Manager I can copy files to any iPhone without restriction.

Linux kernel updates

Both VMware and VirtualBox relies on kernel modules that are tied to the running kernel. This means that when a new kernel is installed/used, the same kernel modules need to be compiled and installed. Fortunately, VMware and VirtualBox make this painless:

# build kernel modules for newly installed kernel
$ vmware-modconfig --console --install-all

# build kernel modules for newly installed/non running kernel ahead of reboot
$ KERN_DIR=/usr/src/kernels/foobar /usr/lib/virtualbox/vboxdrv.sh setup

It remains to be seen whether these pain points can be ever resolved in Linux alone; however, given the pace and feature sets and backward incompatible updates that Apples includes in its IOS and iTunes revisions it would seem unlikely.

However, thanks to virtualisation we at least have a workaround for those wishing to copy our music files to and from our iPhones from Linux.