Parsing raw apache logs

Posted by MB34 on Stack Overflow See other posts from Stack Overflow or by MB34
Published on 2010-05-11T15:48:28Z Indexed on 2010/05/11 15:54 UTC
Read the original article Hit count: 284

Filed under:

php

|

parse

|

apache

|

logs

I need some php code for parsing raw apache logs. In particular, I want the number of times mode=search and the term used for searching. Here is an example:

207.46.195.228 - - [30/Apr/2010:03:24:26 -0700] "GET /index.php?mode=search&term=AE1008787E0174 HTTP/1.1" 200 13047 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
212.81.200.167 - - [30/Apr/2010:04:21:43 -0700] "GET /index.php?mode=search&term=WH2002D-YYH HTTP/1.1" 200 12079 "http://www.mysite.com/SearchGBY.php?page=81" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; GTB6.4; .NET CLR 1.1.4322; .NET CLR 2.0.50727; WinuE v6; InfoPath.2; WinuE v6)"
212.81.200.167 - - [30/Apr/2010:04:21:44 -0700] "GET /file_uploads/banners/banner.swf HTTP/1.1" 200 50487 "-" "contype"
66.249.68.168 - - [30/Apr/2010:04:21:45 -0700] "GET /index.php?mode=search&term=WH2002D-YYH HTTP/1.1" 200 12079 "-" "Mediapartners-Google"

© Stack Overflow or respective owner

Related posts about php

Magento, NGINX, PHP-FPM, APC, MEMCACHED, 16gb Ram CentOS, Spiking PHP-FPM to 100% CPU

as seen on Server Fault - Search for 'Server Fault'
I have been trying to resolve my issue of spiking cpu caused by php-fpm processes. I've reduced the php-fpm config settings to: pm = ondemand pm.max_children = 12 pm.start_servers = 2 pm.min_spare_servers = 2 pm.max_spare_servers = 10 pm.max_requests = 500 php_admin_value[memory_limit] = 128M Problem… >>> More
PHP Pear Installation on CentOS

as seen on Server Fault - Search for 'Server Fault'
[root@ip ~]# yum install php-pear* Reducing CentOS-5 Testing to included packages only Finished Setting up Install Process Package 1:php-pear-1.8.1-2.el5.centos.noarch already installed and latest versio … >>> More
Apache configurations for php "AddType text/html php" or "AddType application/x-httpd-php php .php"

as seen on Server Fault - Search for 'Server Fault'
I am taking over an application server and discover that it contain the following settings: AddType text/html php Although it works, but my understanding is that it should set as following: AddType application/x-httpd-php php .php What are the key differences between the two settings?… >>> More
mod_rewrite settings causes server to throw HTTP 500 errors instead of 404

as seen on Server Fault - Search for 'Server Fault'
Hello. I have a server with VBulletin forum (working under Apache 2.2, CentOS). The default settings for it in .htaccess are as follows: RewriteEngine on RewriteCond %{HTTP_HOST} ^gsmforum\.ru RewriteRule (.*) http://www.gsmforum.ru/$1 [R=301,L] # If you are having problems or are using VirtualDocumentRoot… >>> More
Problems installing Memcache (PECL extension)

as seen on Server Fault - Search for 'Server Fault'
I have installed memcached fine, and now I will need to install PECL extension memcache. Im running RedHat x86_64 es5. The installation gives me this: downloading memcache-2.2.6.tgz ... Starting to download memcache-2.2.6.tgz (35,957 bytes) ..........done: 35,957 bytes 11 source files, building running:… >>> More

Related posts about parse

System.Uri can't parse when password field contains a "#"

as seen on Stack Overflow - Search for 'Stack Overflow'
The following code throws System.UriFormatException: var uri = new UriBuilder("ftp://user:pass#[email protected]:21/fu/bar.zip"); System.UriFormatException: Invalid URI: A port was expected because of there is a colon (':') present but the port could not be parsed. Removing the # symbol from… >>> More
Method extension for safely type convert

as seen on ASP.net Weblogs - Search for 'ASP.net Weblogs'
Recently I read good Russian post with many interesting extensions methods after then I remembered that I too have one good extension method “Safely type convert”. Idea of this method I got at last job. We often write code like this: int intValue; if (obj == null || !int.TryParse(obj… >>> More
LINQ 4 XML - What is the proper way to query deep in the tree structure?

as seen on Stack Overflow - Search for 'Stack Overflow'
I have an XML structure that is 4 deep: <?xml version="1.0" encoding="utf-8"?> <EmailRuleList xmlns:xsd="EmailRules.xsd"> <TargetPST name="Tech Communities"> <Parse emailAsList="true" useJustDomain="false" fromAddress="false" toAddress="true"> <EmailRule address="@aspadvice… >>> More
Real-time log parsing and reporting

as seen on Server Fault - Search for 'Server Fault'
We have a small project we are working on part-time that runs on Nginx/MongoDB on Ubuntu 10.04 LTS Server. We'd like to be able to see reports on things like server load, requests/sec, response time, DB load, DB response time, etc. Is there an open source or free (as in beer) tool that can parse such… >>> More
php file downloads instead of being processed with ajax on apache

as seen on Server Fault - Search for 'Server Fault'
I have a small website where some content is displayed within a HTML tag using AJAX. The content is simply taken from another page on the same web site. However, sometimes instead of loading the parsed PHP file, the browser displays a download box instead. I downloaded the file and this is what it… >>> More