Posts Tagged ‘linux’

FASTER Unit Test

Thursday, January 19th, 2012

I’m developing with linux(Vmware guest) on Windows7(host OS).
My laptop has 5400rpm HDD and 8G mem.
My project has many test cases and fixtures, all test cases finished in about 15 minutes :(

I wanted to finish group test faster. I guessed large amount of fixture data is bottleneck, since it deletes and restores data many times. Disk I/O of Virtual Machine is slower than Host OS’s one.
At first I changed from HDD to SSD, all test cases finished in under 10 minutes, Wow.

Unfortunately, SSD has a limit on the number of write(update) operation. When group test ran, I worried that it might shorten the life of SSD.
So I moved MySQL data files to Memory(tmpfs) to reduce SSD I/O.
After doing it, group test finished in about 5 minutes :)

Redhat clone Linux like CentOS has tmpfs(on memory filesystem) and mounts it /dev/shm by default.

I made the following scripts,
1.  onMemoryDeploy.sh: To move files to memory and make softlink to it
2. onMemoryMySQL: init script to run onMemoryDeploy.sh start/stop

Before executing the scripts, I recommend to backup mysql data files.

/etc/rc.d/init.d/mysql stop
cp -rp /var/lib/mysql /var/lib/mysql-orig

Here is the script to move data files to memory and move them back.
/root/onMemoryDeploy.sh script on gist


#!/bin/sh

MYSQL=/etc/rc.d/init.d/mysql

MEMDIR=/dev/shm
MEM_MYSQL=$MEMDIR/mysql

LIBDIR=/var/lib
ORIGDIR=$LIBDIR/mysql

start()
{
  if [ ! -L $MEM_MYSQL ] && [ -d $ORIGDIR ]; then
    $MYSQL stop
    mv $ORIGDIR $MEMDIR/
    ln -s $MEM_MYSQL $ORIGDIR
    $MYSQL start
  fi
}

stop()
{
  if [ -L $ORIGDIR ] && [ -d $MEM_MYSQL ]; then
    $MYSQL stop
    rm -f $ORIGDIR
    mv $MEM_MYSQL $LIBDIR/
  fi
}

case "$1" in
  start)
    start
  ;;
  stop)
    stop
  ;;
  *)
    echo "Usage: onMemoryDeply.sh start or stop"
    exit 1
  ;;
esac

exit 0

Here is init script(for CentOS) that runs onMemoryDeploy.sh (start/stop) at the OS boot and shutdown.
/etc/rc.d/init.d/onMemoryMySQL script on gist

#!/bin/bash
#
# Startup script for onMemoryMySQL
# chkconfig: 345 85 15
#

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/usr/sbin:/usr/bin:/bin

test -f /root/onMemoryDeploy.sh || exit 0

start()
{
  /root/onMemoryDeploy.sh start
  touch /var/lock/subsys/onMemoryMySQL
}

stop()
{
  /root/onMemoryDeploy.sh stop
  rm -f /var/lock/subsys/onMemoryMySQL
}

case "$1" in
  start)
    start
  ;;
  stop)
    stop
  ;;
  *)
    echo "Usage: onMemoryMySQL start or stop"
    exit 1
  ;;
esac

exit 0

Then register init script with chkconfig tool.

/sbin/chkconfig --add onMemoryMySQL
/sbin/chkconfig onMemoryMySQL on

Have a good testing time!