OpenFiler performance problems with VMWare Server [solved]

Translation of the original in Catalan. Read it in: CatalàCastellano

After many months with a serious performance problem on the home SAN/NAS server that I have set up in a virtual machine of VMWare Server on Ubuntu 8.04 LTS, I've finally decided to fix the problem. In some other post I had mentioned the problem, the thing is that the performance of the 1.5Tb I have on the server from the Host operating system is about 70MB/s but the same tests from the disks managed by the Guest machine, that is OpenFiler, gave a pathetic 3, 4 or at most 5MB/s. That is, when accessing the shared resources over NFS or SMB the performance was insufficient and other servers or machines that use these resources were affected.

Finally, after searching a bit I saw that the problem came from system interrupts that were being lost because the clock rate of the Host server wasn't able to process so many interrupts. So, investigating the matter I saw that what has to be done is to support HPET, in my case for this to work I've had to enable it too in the BIOS of the HP ML110 I use as physical server. With these changes the speed tests of the virtual machine with OpenFiler have improved substantially and right now they give results around 50MB/s. Obviously, I'm still far from an optimal performance but I no longer know whether this is a configuration matter or whether the problem is deeper and already depends on VMWare's own internal management with the modules that act as interfaces between the real hardware and the virtual one.

If you want to know whether you have HPET enabled or not in the kernel it's as simple as looking at dmesg:

root@vm0:~# dmesg |grep hpe
[  120.210408] hpet clockevent registered
[  120.210412] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[  120.210416] hpet0: 3 64-bit timers, 14318180 Hz

If you don't have it enabled, you need to enable the following in the kernel's .config file:

CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_RTC=y

Besides that there are other options to narrow down this kind of problem, some that allow stabilizing virtual machines even if we don't squeeze them to the maximum can let us work better if performance isn't the biggest of our problems and we don't saturate the Host. For example, the following option can be added to the virtual machine's .vmx file:

host.useFastClock = FALSE

By the way, if you want to identify the problem the Host machine's kernel gives errors like:

Aug 21 12:56:11 dey kernel: rtc: lost some interrupts at 2048Hz.

or also things like this:

select() to /dev/rtc to wait for clock tick timed out.

As always on google you can find a lot of information on how to tune your Linux so that it performs at maximum with VMWare. For my part, now I still have to get going a thousand and one things that depended on this file server which little by little was forcing me to stop services for lack of performance. I hope that soon I'll be able to get going features like the photo album and other similar ones.

2 comments

Comments from the old blog, as they were written. Comments are closed.

  1. Marcos Martinez (frommelmak)

    Fa uns dies mentre buscava l'ultima versió del VMware Server em van explicar que la gent de VMware a tret una versió gratuïta (una mica capada) del seu hypervisor, el conegut VMWare Infrastructure ESX, al que anomenen ESXi.

    Al tractar-se d'un hypervisor natiu i no d'un de tipus hosted, Imagino que en qüestions de performance aquesta solució ha de ser millor que una basada en VMware Server.

    http://www.vmware.com/products/esxi

    De tota manera i per curiositat, demà a la feina miro a veure si el kernel del ESXi te el HPET activat o no.

    La veritat es que de moment funciona perfectament i no trobo a faltar res molt important de la versió ESX que jo coneixia.

    off topic: ara ja tens el meu correu personal a més del de la feina ;-)

  2. Oriol Rius

    ostres, doncs no ho sabia això del ESXi ja tinc ganes de provar-lo. Segur que donarà més bon rendiment, o això espero.

    Malgrat sóc Partner de VMWare certificat la veritat és que no faig gaire cas de les newsletters que envien perquè són uns pesats i no paren d'enviar spam.

    Moltes gràcies per la informació.

    off topic: ja tinc guardats els teus dos correus a la meva agenda.