We have a web application that run on two servers of GoDaddy. We experince slow preformance on our production server, although it has stronger hardware then the testing one, and it is dedicated.

I'll start with the configurations.

Testing:

• CentOS Linux 5.8, Linux 2.6.18-028stab101.1 on i686
• Intel(R) Xeon(R) CPU L5609 @ 1.87GHz, 8 cores

• 60 GB total, 6.03 GB used

• Apache/2.2.3 (CentOS)

• MySQL 5.5.21-log
• PHP Version 5.3.15

Production:

• CentOS Linux 6.2, Linux 2.6.18-028stab101.1 on x86_64
• Intel(R) Xeon(R) CPU L5410 @ 2.33GHz, 8 cores

• 120 GB total, 2.12 GB used

• Apache/2.2.15 (CentOS)

• MySQL 5.5.27-log - MySQL Community Server (GPL) by Remi
• PHP Version 5.3.15

We are running the same code on both servers.

The Problem

We have some function that executes ~30000 PDO-exec commands. On our testing server it takes about 1.5-2 minutes to complete and our production server it can take more then 15 minutes to complete. As you can see here, from qcachegrind:

Researching the problem, we've checked the live graphs on phpMyAdmin and discovered that the MySQL server on our testing server was preforming at steady level of 1000 execution statements per 2 seconds, while the slow production MySQL server was only 250 executions statements per 2 seconds and not steady at all, jumping from 0 to 250 every seconds.

You can clearly see it in the graphs:

Testing server:

Production server:

You can see here the comparison between both of the configuration of the MySQL servers.Left is the fast testing and right is the slow production. The differences are highlighted, but I cant find anything that can cause such a behavior difference, as the configs are mostly the same.

Maybe you can see something that I cant see.

Note that our tables are all InnoDB, so the MyISAM difference is (probably) not relevant.

Maybe it is the MySQL Community Server (GPL) that is installed on the production server that can cause the slow performance? Or maybe it needs to be configured differently for 64bit ?

I'm currently out of ideas...