Split big Apache log to folder structure

Posted by Dough on Server Fault See other posts from Server Fault or by Dough
Published on 2012-12-01T08:50:20Z Indexed on 2012/12/01 11:08 UTC
Read the original article Hit count: 326

Filed under:
|
|

I just changed my Apache log behavior because it was making me having very BIG files...

So I now use cronolog to split my logs to log/httpd/2012/11/access_2012.11.30.log for exemple, pattern : %Y/%m/access_%Y.%m.%d.log

I now want to split my old 42GB file to the same structure but really don't know how to do that efficiently.

I tried some simple commands with cat, egrep, awk... but really don't know how to handle all that in a more powerful script.

Here is how the log looks like :

x.x.237.134 - - [08/Apr/2011:14:43:09 +0200] "GET...
x.x.50.15 - - [08/Apr/2011:14:43:09 +0200] "GET...
[...]
x.x.254.19 - - [28/Feb/2012:15:24:48 +0100] "GET...

So I need for yeah line to get :

  • year %Y (ex. 2012)
  • month %m (ex. 11)
  • day %d

And to push out the entire line to : %Y/%m/access_%Y.%m.%d.log

Can someone give me clues to get that working ?

Thanks a lot for your interest.

© Server Fault or respective owner

Related posts about apache2

Related posts about log-files