Asus G50vt

Ubuntu Jaunty Jackalope and Karmic Koala runs perfect on this. Come and see:

Asus did well with this product. Personally I have no problem with its performance or quality. The chassis is firm. It looks and feels endurable and artistic. The price is totally acceptable for this (1600$ at Nov 2008).

The specs are:

Of course I do not forget lspci


00:00.0 Host bridge: Intel Corporation Mobile 4 Series Chipset Memory Controller Hub (rev 07)
00:01.0 PCI bridge: Intel Corporation Mobile 4 Series Chipset PCI Express Graphics Port (rev 07)
00:1a.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 03)
00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 03)
00:1a.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 03)
00:1a.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 03)
00:1b.0 Audio device: Intel Corporation 82801I (ICH9 Family) HD Audio Controller (rev 03)
00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 (rev 03)
00:1c.1 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 2 (rev 03)
00:1c.2 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 3 (rev 03)
00:1c.5 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 6 (rev 03)
00:1d.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03)
00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03)
00:1d.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03)
00:1d.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev 93)
00:1f.0 ISA bridge: Intel Corporation ICH9M-E LPC Interface Controller (rev 03)
00:1f.2 RAID bus controller: Intel Corporation Mobile 82801 SATA RAID Controller (rev 03)
01:00.0 VGA compatible controller: nVidia Corporation G94 [GeForce 9800M GS] (rev a1)
03:00.0 Network controller: Intel Corporation Wireless WiFi Link 5100
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
07:01.0 FireWire (IEEE 1394): Ricoh Co Ltd R5C832 IEEE 1394 Controller (rev 05)
07:01.1 SD Host controller: Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter (rev 22)
07:01.2 System peripheral: Ricoh Co Ltd R5C843 MMC Host Controller (rev 12)
07:01.3 System peripheral: Ricoh Co Ltd R5C592 Memory Stick Bus Host Adapter (rev 12)
07:01.4 System peripheral: Ricoh Co Ltd xD-Picture Card Controller (rev ff)


I used Jaunty Jackalope and now it is running Karmic Koala for months. Here are the notes:

INSTALLATION

• If you have 2 disk configuration, you can use FakeRAID for RAID1 or RAID0 configuration without putting much effort. It will definitely boost performance with RAID0 but you have take extra care since loss of a disk will mean complete data loss. If you also want to use bundled Windows Vista with better performance, you should really consider this. In this case, you will need to use alternative installer cd instead of regular Ubuntu installer because regular installer does not include support for booting from dmraid devices
• If you want to use FakeRAID, note that GRUB2 does not support it yet. You will need to use GRUB instead.
• Regular installation media works ok if you do not want to use FakeRAID

HARDWARE DETECTION

• Ubuntu automatically discovers every single hardware component, function keys and uses them. You do not need to make any Xorg configuration at all. (Thanks XRandR) I recommend enabling restricted drivers when Ubuntu asks for it.
• NVidia’s original binary driver is unable to process sound through HDMI interface. Since we also have SPDIF, it is not a huge problem.
• OLED driver is also included in the staging drivers area and it is available with Ubuntu. You can check here about what can do with this toy.
• You may hear speakers still running when headphone is plugged. You can add following option to /etc/modprobe.d/alsa-base.conf|options snd-hda-intel power_save=10 power_save_controller=N model=g71v
• Traditionally, hibernating is not stable but standby works ok.

This is a screenshot from my desktop, running KDE 4.3

Here is the full size

I would like to conclude by telling that this is a good Asus product with components working with at least Ubuntu 9.04 and 9.10.

I wanted to test Windows 7 RC with my old Vaio. The installation did not take more than 35 minutes, which good for such an hardware. However, Windows failed to use following devices:

• Wireless network adapter (Inter ProWireless)
• ATI Display adapter
• Internal USB camera
• Bluetooth
• Fingerprint reader
• SD card reader

This was the first disappointment. Windows traditionally fails to use some hardware automatically but I expected more from Windows 7. We are in 2009 and we deserve more.

I browsed to http://esupport.sony.com to find drivers. Unfortunately there is only a few Windows Vista Business drivers and Intel ProWireless is not one of them.

Since Windows XP drivers do not work with Windows 7, my adventure with it ended shortly.

For over ten years, I keep hearing “Linux does not support much hardware” from dinosaurs.

Linux -believed to run fewer hardware- is able to use all of these hardware (still except fingerprint reader) I am able to use this laptop with the latest advanced kernels and software, without any problem.

On the other hand, billions of dollars Microsoft + Sony brotherhood can not provide me a install-and-go operating system or new drivers for new advanced operating systems. They force me to use 8 year old and ugly Windows XP.

Consider a typical zealot and his/her friends. (Are any female Linux zealot anyway ?) I will write some cases and they do happen somewhere in the world.

Our command prompt obsessive zealot comes to a friend’s home and asks for wireless network parameters. Once he gets, he runs series of iwconfig, wpa_supplicant and ifconfig commands to connect. He writes a profile for this network by hand and tests it. It takes about 10-15 minutes. He does not use simple network connection applets of Gnome/Kde. He does this because he is obsessed with running minimal applications on a 4GB laptop. Do you know what his friends think about now ? “It takes running some cryptic commands and 15 minute just to connect to a network using Linux”. This is the only experience they have ever seen. They will judge Linux by this from now on..

A friend of our zealot is curious about Linux and wants to try. As a trusted and renown friend, our zealot is happy to get a chance to win another user. They met at home at night, order some pizza and our zealot starts installing Gentoo or Slackware. (Or some other enduser disoriented distribution)

- Hey, what’s that Slackware/Gentoo thing ? I always hear about Ubuntu, why do not we just install it ?
- This is the one I use. By using this, you will have the absolute control on your system. You will make sure that it runs only what you want and they will be compiled for the best performance
- Cool

Now our zealot runs cfdisk and some other text based tools to organize disk. He carefully setups partitions and language configurations. He selects every package individually and 3 hours has already passed. Now the new user thinks that he can never do this by himself. Our lovely zealot has prevented his friend to experience nice and short gui installers of Ubuntu or Fedora based distributions.

Now the system has been installed and our zealot carefully checks for hardware components. He grabs the latest vanilla kernel and select modules one by one. It takes another 3 hour to make a fitting kernel image for that computer. He assures that this will give the best performance and he does not tell what will happen if the user tries to connect a different hardware. He gains may be 3-5% in performance but he loses flexibility of having thousands of hardware drivers which are ready to plug and play.

When it is 3 A.M, the system is ready to run, at last. Thanks to our zealot. Now the users asks for some applications. Our zealot grabs source files and compiles by setting configure flags according to his own taste. He quickly mentions about the compilers, source files etc to his friend, who has absolutely no idea what a programming language is..Our zealot thinks he can install better software although there are tens of thousands of testers and developers who create one-click-install packages which are tuned and tested.

Our unlucky user tries to use Linux for a week. He can not change display properties because most of them were handwritten to xorg.conf although it is not required in most cases. He barely remembers how to compile a program and fails trying to install something. At last, he does not access his external disks easily because our zealot does not let hal and udev run because he thinks they are not necessary..

This is completely harmful to free software society. These kind of people should stop and think about what they are doing. If you want to help, do not try to consider end users like yourself. Consider them as babies and give what they want. Give them a system which can use most of the connected hardware, easily play media, and watch videos on YouTube. Ordinary users do not need to run tiny window managers. They do not need to know about source code or build system.

Just show them installing the system by clicking next. Then explain that backing up is as easy as archiving home directory. Lastly, instruct to use package manager to search, install and uninstall applications by clicking their name.

If you will not do this, just let those users use other operating systems. Creating a negative image of desktop Linux experience is the last thing we want. Right ?

I’ve collected the list of the top sites from Alexa. They provide a csv file ordered by ranking. Then I used a tiny script to collect Server header from top 15000 sites. After that, I used libgeo-ipfree-perl package to guess geographic location of sites. The numbers may not be exact but enough to give very strong idea.

There are some points that I should say:

• Geographical location has been found based on ip/country match. It may not be 100% correct
• Some sites does not include Server header and some of them just give incorrect information. (about 900 in total) It does not make the site more secure, and I do not think it will make any difference in percentages. It is possible to modify Server header in all important web server software
• Only for the Top 100, I used Netcraft for the information about missing 8 servers.
• Of course there are cache engines, other web servers and proxies but their numbers are just neglectable
• Apache-Coyote has also some usage but I omit to make graphs simpler. It does not have affect on the idea of this work
• The numbers in the charts are not percentages but the actual server numbers. For the country pie charts, I also omit unknown servers if there are many (not much indeed)
• For the countries, I only checked those who has at least 5 sites in Top 15000

Top 100 Sites

We see a clean win for Apache. This is the most important and marginal output. If you look at only Netcraft’s deployed server counts, Microsoft IIS seems only 15% behind Apache. However when we check the best of the best, the busiest web sites in the Internet, we see that Microsoft IIS is far behind Apache. Besides, nginx is only a few sites behind of IIS. The names of the sites running IIS are also notable:

www.live.com (Microsoft)
www.msn.com (Microsoft)
www.myspace.com
www.microsoft.com (Microsoft ..ehh)
www.doubleclick.com
www.go.com
www.conduit.com
espn.go.com

In the previous year, Google has changed their Apache server headers to GWS. GWS is a modified version of Apache and if we combine this number with plain Apache deployments, the difference increases.

Top 1000 Sites

We still have same aggressive percentage of Apache and its derivatives

Top 10000 Sites

As the number increases to 10000, Microsoft IIS gains some more points.

Top 15000 Sites

The scheme does not change very much. Even we include rather low traffic sites, the winner is Apache

Choices based on countries

It is also interesting to check which countries prefer which software. Although open source software dominates in the world, we can notice some local differences.

This is one of the most important graph since most of the high traffic web sites are originated from the US. Although IIS seems closing the gap in the world scale, Apache is certainly preferred much more than IIS in here.

The picture changes when we move to Europe. Finland, Norway, Ukraine, Slovakia, Austria, Bulgaria, Romaina, Hungary, Ireland, Luxembourg, Portugal seem to use only open source web server software in Top 15000. Some other notable countries are here:

Japan has also notable Apache usage:

Also, there are some countries which prefer Microsoft IIS over Apache and other open source solutions
Argentina, Chile, Bermuda, Egypt have only Microsoft based sites in Top 15000.

China is also an important Microsoft user although it has the highest rate for software piracy along with Russia and Thailand.

Here is Iran, where everything from US is considered to be demonic, except Microsoft. Business is business anyway..

And lastly, my own country Turkey

Conclusion

After all these, there is not much need for a conclusion but I will write anyway

• Open source platforms (mainly Linux) and Apache (with derivatives) are the most preferred combination for high traffic web sites.period.
• nginx performed relatively well and it is a rising star. In some countries, it is used more than Microsoft IIS
• Usage of Microsoft IIS increases as we move away from Top 100
• Free software in web services has well adopted in most European countries and United States
• Some countries develop technology and some consume it. It is noticable that, countries which are well-known for their technology leadership also prefer using Free Software in their busiest web services. Consumers lack vision, knowledge, qualified engineers and in my opinion, this affects the platform decision.

A month ago, I registered for 1z0-047 , Oracle SQL exam for a step towards OCA (Oracle Certified Associate). I choosed the nearest test center in Istanbul. For those who may not know, Istanbul city area is longer 100km..So it is very important to choose the nearest test center. (It was still 20km away from my work location anyway)

Me and a friend of mine were registered for the same exam at same date/time. When we get to the test center (Basarisoft), we were told there there is only “one” computer and we must wait for each other. Since there was no other option, we accepted.

“Testing Center” was more like a prison cell. Imagine a room with a small desk, uncomfortable chair, tiny window on top. Two people can barely stand in the room. This was the testing center indeed. Prometric’s testing server was under the desk with cover open! So called “window” was towards a wall. Nobody asked for an id card, cell phones or laptop. They left us alone and never come back.

I filled out a form in Prometric’s site but nothing happens.

Next time for 1z0-042 (OCA), I decided to change my test center. There was another center, Komdatasoft, which I used 5 years ago. It was very professional but far from me. I noticed that the company moved according to Prometric’s list of testing center and I choosed it.

When I got to the location, people at the reception desk told me that there is no such company there. I checked by myself to make sure and called Prometric for help.

The guy on the helpdesk was quite surprised when I explained the situation and I’m afraid he did not believe me in the beginning. He told me that he will call the testing center to verify.

A few minutes later, helpdesk said that no one was actually answering the phone. They compensated this by granting a free exam ticket. I managed to have my exam at last.

What I do not understand about all these, what kind of testing company have absolutely no idea about its testing centers. Prometric does not even know if their test center is alive. When I check the site list, I can still see this ghost testing center.

As a candidate, I expect to see a certain level of standard in testing centers.

This is totally unacceptable and irrespectful. Thanks Prometric! I could not imagine worse..

I am using an Asus G50vt. It is a wonderful laptop with many features. It also includes Intel’s fake raid and I use it to stripe two internal disks.

I upgraded my Intrepid Ibex using “update-manager -d” command, the recommended way from Ubuntu. It oversimplifies the whole distribution upgrade. It activates the propoer repositories, downloads and installs packages, but with a single dirty problem.

The version of dmraid in the Jaunty Jackalope Beta is now rc15-4 and it does not recognize Intel RAIDs. Therefore, initrd is unable to activate the volume. The solution is to download the latest version (currently rc15-6) from Jaunty repositories along with dependencies and install it. Installation will update the initrd image so you do not have to do it. To boot and access the system, you can use the alternate install cd Strangely, it will recognize those volumes but not your own dmraid.

Now you have your volume again..

It is totally unusable.

Right. I am not exaggerating. I am not trying to smash it. I wish I could say something better but it is really hard.

Just like everybody complains, this version does not have a management console client. This would not be a problem if the proposed method was working ok. Now, vmware server users have to use web browsers to manage their guests.

During first connection attempt, VMWare tries to install a plugin to your favourite web browser to get the console. The problem is, sometimes it takes a minute to get in to the console. If you are lucky enough, you can click a couple ofthings. If you press ctrl+alt to exit, you may again wait some time.

This is totally unacceptable. I installed the software on 3 different systems and all of them behaved similar. Then I decided to return to 1.0.8.
My systems are running 64-bit Ubuntu Intrepid Ibex 8.10.

Due to the changes in 2.6.27, VMWare Server 1.0.8 does not install by itself. In order to do, followings steps should be taken:

• Install g++ build-essentials linux-source-2.6.27 linux-headers-2.6.27-11-server ia32-libs inetutils-inetd kernel-package packages
• Get the patch:
  wget http://www.insecure.ws/warehouse/vmware-update-2.6.27-5.5.7-2.tar.gz
• Untar the vmware server archive and start the installation with vmware-install.pl. Answer the questions as you like but when it asks to continue with vmware-config.pl, say no
• Untar the update. Change directory to it, and run runme.pl. It will patch kernel modules and continue with vmware-config.pl
• If the installation fails after printing “Generating SSL Certificates”, touch /etc/vmware/ssl/rui.key and /etc/vmware/ssl/rui.crt files
• Installation should finish without an error now. However you will be surprised when you see that you can not login via console, even if you enter correct password
  Intrepid Ibex changed the default password hashing algorithm from md5 to sha512 and PAM clients used by VMWare Server does not use sha512. Open /etc/pam.d/common-passwd and change sha512 in the password line to md5. Do not forget to reset your password to make sure that shadow contains md5, not sha512 hashes.

This should be enough, normall. If the system is 32 bit, it is also possible to use the systems own pam libraries instead of VMWare’s to be able to use sha512 passwords.

I hear some of you saying “Why don’t you just use KVM and say goodbye to all those ?” If you have to share the administration of the system, it is better to use a software that everyone can easily use. It should be admitted that, even there are GUIs for KVM, it is not as easy as VMWare and VMWare makes more people happy using non-critical/non-production systems. (I do not want to be called at Sunday just because that I am the only one who can manage KVM. Yes, I am bad )

• 4x HP Compaq DL380 G5
• Intel Xeon quad core
• 16 GB RAM
• 2 x 146 GB SCSI internal disk
• 2 x Qlogic HBA
• 2 x Gigabit Ethernet
• MSA 2000 with 2x controllers
• 2x Fibre Switch

Our environment will end up having following features:

• All servers, running Ubuntu Hardy Heron 8.04 LTS 64-bit edition (Openvz enabled kernel)
• MSA2000 configured as single RAID6 drive
• GFS cluster setup with simple config, including quorum device and simple fencing
• RAID6 drive formatted as LVM volume and shared between 4 nodes using GFS
• LVS+Heartbeat to setup ip failover for web and mysql services
• Network interfaces are bonded to provide redundancy
• Two nodes will run web server and the other two nodes will use mysql

(db1, db2, web1, web2)

• Use OpenVZ for virtualization

Fun, right ? Ok, I will summarize MSA 2000.

MSA 2000 can be configured using serial console or web management gui. I recommend using serial console just for setting up ip address and continue with its nice web gui. I also recommend reading basic information about volumes (physical-logical) and RAID systems. I preferred using RAID6 because I need to maximize disk space while maintaining safety. RAID5 is not the best option because it can not deal with 2 broken disks and its performance is awful when single disk fails. Using RAID6 we can still stand with 2 disk failues and we will still have acceptable performance. If performance is more important than the storage size, RAID 0+1 is more suitable.

HP delivers servers with hardware raid capability and you will most probably have two mirrored disks. Leave those, it will not have a negative effect on the performance and you will gain important redundancy against our old, crappy hard disk technology.

1. Download Ubuntu Hardy Heron 8.04 64-bit server edition and burn the image. Boot the system with the cd, verify the content on the cdrom and start the installation. Make a base installation with minimal packages. We can continue using updated packages in the repositories.
   
   
   If you plan adding new local disks to hosts, you can choose to have an LVM managed disk during partitioning step. If you do not, you can just select ext3 as local partition.

   Here, some might say that you should make separate partitions for /boot, /var, /home or /usr. Here are some basic rules:

   • Do not make /boot a separate partition unless you have a real good reason
   • Do not make /var a separate partition. Some applications (like mysql) put database files under /var/lib. (Or you can have mail queue under /var/spool/) In that case, put that specific directory on another partition, not whole /var.
   • You should make /home a different partition if you intend to use home directories a lot.
     
     The point is to be able to use one single backup file restore all the necessary files in case of a disaster. After more than 10 years, I can say that I see no advantage of having separate /var or /usr but I faced many problems
2. Setup network parameters during the installation. Provide http proxy if necessary

   After the installation. Open /etc/apt/sources.list

   vi /etc/apt/sources.list

   Change contents of the file like this:

   deb http://us.archive.ubuntu.com/ubuntu/ hardy main restricted universe multiverse
   deb-src http://us.archive.ubuntu.com/ubuntu/ hardy main restricted universe multiverse
   deb http://security.ubuntu.com/ubuntu/ hardy-security restricted main multiverse universe
   deb-src http://security.ubuntu.com/ubuntu/ hardy-security restricted main multiverse universe
   deb http://us.archive.ubuntu.com/ubuntu/ hardy-updates restricted main multiverse universe
   
   deb-src http://us.archive.ubuntu.com/ubuntu/ hardy-updates restricted main multiverse universe

   This will make sure that the system will fetch the updated and security packages. You can also add hardy-proposed and hardy-backports to have more recent packages. However, once you have the system up and running, do not install updates other than security, unless necessary

   Update the package cache and make an upgrade:

   apt-get update
   
   apt-get upgrade
   
3. Now run tasksel and select OpenSSH server. If you want to have a desktop environment, also select Ubuntu Desktop. Tasksel will install all the required packages. I will not focus on installation of web server or database packages. It quite depends on needs and taste. So I’m stepping in to other stuff.
4. Set up serial console for remote access. ilo2 requires additional license in order to provide remote console functionality via browser. However, you can still have a serial console using virtual serial port.
   
   First we need to create a serial port listener. Since Ubuntu uses upstart as the init replacement, we will not use old inittab. Open /etc/event.d/ttyS0 and write:
   ttyS0 – getty

   start on runlevel 2
   start on runlevel 3
   start on runlevel 4
   start on runlevel 5

   stop on runlevel 0
   stop on runlevel 1
   stop on runlevel 6

   respawn
   exec /sbin/getty -L 115200 ttyS0 vt102

   Let root login from the serial console. Open /etc/securetty and enter ttyS0

   Now edit /boot/grub/menu.lst to tell grub to use ttyS0 during boot up phase

   serial –unit=0 –speed=115200 –word=8 –parity=no –stop=1
   terminal –timeout=10 serial console

   And lastly, append following to the end of the each kernel line in /boot/grub/menu.lst

   console=ttyS0 console=tty0
5. Install multipath-tools. Ubuntu kernels already include support for Multipathed IO. This allows us to connect to the storage array using two different physical paths. We can have redundancy for failover or we can distribute I/O load between the paths

   My /etc/multipath.conf looks like:

   defaults {
   udev_dir /dev
   user_friendly_names yes
   }
   blacklist {
   devnode "cciss"
   devnode "fd"
   devnode "hd"
   devnode "md"
   devnode "sr"
   devnode "scd"
   devnode "st"
   devnode "ram"
   devnode "raw"
   devnode "loop"
   }
   multipaths {
   multipath {
   wwid 3600c0ff000d56f978446734801000000
   alias san1
   }
   }
   device {
   vendor "HP"
   model "MSA2*"
   path_grouping_policy multibus
   getuid_callout "/lib/udev/scsi_id -g -u -s /block/%n"
   selector "round-robin 0"
   rr_weight uniform
   prio_callout "/bin/true"
   path_checker tur
   hardware_handler "0"
   failback immediate
   no_path_retry 12
   rr_min_io 100
   }
   

   If we had other devices in the path which we do not want to include, we would have a blacklist section. Examples for this section is included in the default config file. My path_grouping_policy is failover. You can also use other policies to distribute load, such as multibus

   Start multipathing daemon by running /etc/init.d/multipath-tools start . Then run multipath -ll (Double L) . You should see like this:

   root@node1:~# multipath -ll
   san1 (3600c0ff000d56f978446734801000000) dm-0 HP      ,MSA2212fc
   [size=820G][features=0][hwhandler=0]
   \_ round-robin 0 [prio=0][active]
     \_ 0:0:0:0 sda        8:0   [active][ready]
   \_ round-robin 0 [prio=0][enabled]
     \_ 1:0:0:0 sdb        8:16  [active][ready]
   

   If you see only one path, then your fibre switches may need configuration or you may have problems in cabling. All nodes should be able to see the device. Now we can mount the disk on one of the nodes but we must not use it on the other nodes concurrently, for now.

6. You should be able to see the disk on the storage array by now. You have to see as many as physical disks as you created in the management GUI of the MSA 2000.
   
   Note: For kernel versions < 2.6.19, qla2xxx drivers include failover capability. However, the driver does not work with newer kernels due to serious changes in kernel structs. Newer versions of the driver omits this capability and recommends using multipath-tools. I recommend sticking with the newest kernels.
   • Run cfdisk against the shared disk
   • Create an LVM partition. The size is up to you, I will continue with one gigantic partition
   • Create another partition for cluster quorum disk. ~10MB is enough and the type can be Linux
   • Save and exit. (You may need to reboot or run partprobe to let the kernel see the changes directly)
   • First create a physical volume. We will put our volume groups in it.
     pvcreate /dev/mapper/san1-part1
     
   • Create and activate a volume group. We will put our logical volumes in it
     vgcreate vgname /dev/mapper/san1-part1
     
     vgchange -a y vgname
     
   • Create logical volume using all available space
     vgdisplay vgname|grep “Total PE”
     
     lvcreate -l usetotalpehere vgname -n lvname
     
   • Create your favourite file system using the new logical volume. The path will look like:
     mkfs /dev/mapper/lvname
     
7. Now we will configure GFS2 cluster. First, we have to install required packages on all the nodes.
   apt-get install gfs2-tools gfs-tools cman clvm

   Add following modules to /etc/modules

   loop
   lp
   rtc
   fuse
   gfs
   lock_dlm
   ip_vs_ftp
   ip_vs_lblc
   ip_vs_lc
   ip_vs_wlc
   ip_vs_lblcr
   ip_vs_nq
   ip_vs_wrr
   ip_vs_sh
   ip_vs_dh
   ip_vs_sed
   ip_vs_rr
   bonding
   

   Run modprobe against all these modules to load them in to the kernel
   
   
   Create GFS2 (or GFS, whichever you want. GFS2 is still experimental) volume. Parameter j defines the number of journals. Normally, every node must have one. We have to use Distributed Lock Manager for lock management.

   mkfs -t gfs2 -p lock_dlm -j 4 /dev/mapper/lvname

   Setup /etc/lvm2/lvm.conf by defining following values

   locking_type = 2
   fallback_to_clustered_locking = 0
   fallback_to_local_locking = 0
   locking_library = “liblvm2clusterlock.so”

   We should never fallback to another locking mechanism otherwise we can harm the data integrity.

   Now, we have to create our quorum disk. Basically, quorum disk is accessed by all nodes frequently to write timestamped “I am here” messages. If an alive message is not seen for specified time, that node is considered to be dead and fenced from the storage. Quorum disk is extremely important for data consistency. It must reside on a physical partition and all nodes musth have access to it. Quorum can not be on a LVM volume because accessing to cluster LVM requires cluster membership. Initialize the quorum disk:

   mkqdisk /dev/mapper/san1-part2

   Open /etc/cluster/cluster.conf and define the GFS cluster. Ideally, all nodes should have at least one working fencing method. Otherwise, it might not be possible to prevent data corruption. The best fencing methods are power related ones. Since I only have a ilo2 port, I send poweroff signal through it. It is also possible to run scripts which login to fiber switch and shutdown a specific port.

   <br /> <?xml version="1.0"?><br /> <cluster name="mycluster" config_version="1"><br /> <cman expected_votes="5" two_nodes="0" cluster_id="777"><br /> </cman><br /> <clusternodes><br /> <clusternode name="node1" nodeid="1" votes="1"><br /> <fence><br /> <method name="ilo"><br /> <device name="ilo_node1"/><br /> </method><br /> </fence><br /> </clusternode> </p> <p> <clusternode name="node2" nodeid="2" votes="1"><br /> <fence><br /> <method name="ilo"><br /> <device name="ilo_node2"/><br /> </method><br /> </fence><br /> </clusternode><br /> <clusternode name="node3" nodeid="3" votes="1"><br /> <fence><br /> <method name="ilo"><br /> <device name="ilo_node3"/><br /> </method><br /> </fence><br /> </clusternode><br /> <clusternode name="node4" nodeid="4" votes="1"><br /> <fence><br /> <method name="ilo"><br /> <device name="ilo_node4"/><br /> </method><br /> </fence><br /> </clusternode><br /> </clusternodes><br /> <quorumd device="/dev/mapper/san1-part1" votes="4" stop_cman="1"><br /> <heuristic program="/bin/true" score="1" interval="2" tko="3"/><br /> </quorumd> </p> <p><fence_daemon post_join_delay="5" clean_start="1"/><br /> <fencedevices><br /> <fencedevice name="ilo_node1" agent="fence_ilo" ipaddr="192.168.25.223" login="Administrator" passwd="1234" action="off"/><br /> <fencedevice name="ilo_node2" agent="fence_ilo" ipaddr="192.168.25.224" login="Administrator" passwd="1234" action="off"/><br /> <fencedevice name="ilo_node3" agent="fence_ilo" ipaddr="192.168.25.221" login="Administrator" passwd="1234" action="off"/><br /> <fencedevice name="ilo_node4" agent="fence_ilo" ipaddr="192.168.25.222" login="Administrator" passwd="1234" action="off"/><br /> </fencedevices><br /> </cluster><br />
8. Setup bonding for ethernet interface.
   
   Normally, I recommend having 4 ethernet interfaces for maximum reliability. Since we have 4 nodes, we can not just use cross cables or serial cables for cluster interconnect. We have to connect to a switch and switches can fail anytime. Therefore, two of the four interfaces must be bonded in failover mode to in order to make sure that even if one link/switch fails cluster traffic can still survive. This extremely vital. Cluster interconnect will not only carry heartbeat messages, but also provide transport for distributed lock manager. If distributed lock manager losts connectivity, the node will lost its access to global storage.
   
   Bonding setup is extremely simple. Just configure /etc/network/interfaces file like this:

   auto bond0
   iface bond0 inet static
           address x.y.z.t
           netmask a.b.c.d
           gateway q.w.e.r
           slaves eth0 eth1
   

   Of course you will need another similar section for bond1

   Later, it is possible to check the status from /proc/net/bonding/bond0

   Ethernet Channel Bonding Driver: v3.2.3 (December 6, 2007)
   
   Bonding Mode: fault-tolerance (active-backup) (fail_over_mac)
   Primary Slave: None
   Currently Active Slave: eth0
   MII Status: up
   MII Polling Interval (ms): 100
   Up Delay (ms): 100
   Down Delay (ms): 100                                         
   
   Slave Interface: eth0
   MII Status: up
   Link Failure Count: 0
   Permanent HW addr: 00:1e:0b:d3:f2:34
   
   Slave Interface: eth1
   MII Status: down
   Link Failure Count: 0
   Permanent HW addr: 00:1e:0b:d3:f2:32
   

9. Reboot all nodes. If everything is correct and there is no network issue, all nodes should join to the cluster. Cluster memberships can be checked by running cman_tool nodes.

Now we can continue with setting up heartbeat for web and sql services. Before this, all ip address should be planned and entered to all /etc/hosts files on all nodes.

10. Setup Linux IP cluster
    • Install required packages for ip cluster. apt-get install heartbeat2 ipvsadm ldirectord
    • Tune kernel parameters for cluster operation and reboot all nodes. Note that if any of the interfaces is missing, you will probably have issues in ipvs cluster.

      net.ipv4.conf.default.rp_filter=1
      net.ipv4.conf.all.rp_filter=1
      net.ipv4.ip_forward=1
      net.ipv4.conf.all.arp_ignore = 1
      net.ipv4.conf.eth0.arp_ignore = 1
      net.ipv4.conf.eth1.arp_ignore = 1
      net.ipv4.conf.all.arp_announce = 2
      net.ipv4.conf.eth0.arp_announce = 2
      net.ipv4.conf.eth1.arp_announce = 2
      net.ipv4.conf.bond0.proxy_arp = 1
      

    • Edit /etc/ha.d/ha.cf
      Three lines are most important in this file.

      ucast bond0 x.y.z.t
      node web1
      node web2
      

      ucast line defines interface for heartbeat checks and destination ip address. node lines define the hostnames of the heartbeat nodes.

    • Edit /etc/ha.d/haresources. Define ldirectord module for cluster ip. The line must be same in both nodes of the cluster. For example, following line must be used on both db1 and db2.

      web1    ldirectord::ldirectord.cf LVSSyncDaemonSwap::master IPaddr2::clusteripaddress/netmaskslashnotation/bond0/broadcastaddress
      

    • Edit /etc/ha.d/ldirectord.conf. Here we define two real servers with different weights. manual of ipvsadm explains all load balancing algorithms in detail. In this setup, we create an pure active/passive balancing using sed scheduler. Checktype is only connect but it is possible to send http requests and check for return values.

      checktimeout=10
      checkinterval=1
      autoreload=yes
      logfile="local0"
      
      virtual = tomcat:8080
              service = http
              real = web1:8080 gate 65535
              real = web2:8080 gate 1
              checktype = connect
              scheduler = sed
              protocol = tcp
      
11. Create a new openvz based virtual server. You can check here for a template creation howto if you do not already have one