Description of the problem
Regularly, cron php processes crash on our production server, which result in mails with the following body :
PHP Fatal error: PHP Startup: apc_mmap: mmap failed: in Unknown on line 0
Segmentation fault (core dumped)
I think the Segmentation fault (core dumped) should result in core files being handled by apport and then written in /var/crashes, but the files I can see there are there since yesterday, although the last crash occured today :
-rw-r----- 1 root whoopsie 1138528 mai 22 04:09 _usr_bin_php5.0.crash
-rw-r----- 1 frontoffice whoopsie 1166373 mai 20 18:00 _usr_bin_php5.1005.crash
-rw-r----- 1 frontoffice whoopsie 81622658 mai 22 00:05 _usr_sbin_php5-fpm.1005.crash
I tried to download the last one anyway, and ran gdb /usr/sbin/php5-fpm /tmp/_usr_sbin_php5-fpm.1005.crash, only to be told that the file is not a core file (its format was not recognized).
Here is the server's apc configuration :
cat /etc/php5/cli/conf.d/20-apc.ini
extension=apc.so
apc.shm_size=512M
apc.ttl=3600
apc.user_ttl=3600
apc.enable_cli=1
I'm mostly worried about the apc.shm_size… isn't it too high or too low ? I understand it has to do with the size of memory segments.
Question(s)
What could be the problem ?
How can I troubleshoot it (how can I get
a valid core file ?) ?
System information
free
total used free shared buffers cached
Mem: 5081296 4354684 726612 0 374744 959968
-/+ buffers/cache: 3019972 2061324
Swap: 522236 516888 5348
cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.2 LTS"
php -v
PHP 5.4.17-1~precise+1 (cli) (built: Jul 17 2013 18:14:06)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies
php -i excerpt :
Configuration
apc
APC Support => enabled
Version => 3.1.13
APC Debugging => Disabled
MMAP Support => Enabled
MMAP File Mask =>
Locking type => pthread mutex Locks
Serialization Support => php
Revision => $Revision: 327136 $
Build Date => Nov 20 2012 18:41:36
Directive => Local Value => Master Value
apc.cache_by_default => On => On
apc.canonicalize => On => On
apc.coredump_unmap => Off => Off
apc.enable_cli => On => On
apc.enabled => On => On
apc.file_md5 => Off => Off
apc.file_update_protection => 2 => 2
apc.filters => no value => no value
apc.gc_ttl => 3600 => 3600
apc.include_once_override => Off => Off
apc.lazy_classes => Off => Off
apc.lazy_functions => Off => Off
apc.max_file_size => 1M => 1M
apc.mmap_file_mask => no value => no value
apc.num_files_hint => 1000 => 1000
apc.preload_path => no value => no value
apc.report_autofilter => Off => Off
apc.rfc1867 => Off => Off
apc.rfc1867_freq => 0 => 0
apc.rfc1867_name => APC_UPLOAD_PROGRESS => APC_UPLOAD_PROGRESS
apc.rfc1867_prefix => upload_ => upload_
apc.rfc1867_ttl => 3600 => 3600
apc.serializer => default => default
apc.shm_segments => 1 => 1
apc.shm_size => 512M => 512M
apc.shm_strings_buffer => 4M => 4M
apc.slam_defense => On => On
apc.stat => On => On
apc.stat_ctime => Off => Off
apc.ttl => 3600 => 3600
apc.use_request_time => On => On
apc.user_entries_hint => 4096 => 4096
apc.user_ttl => 3600 => 3600
apc.write_lock => On => On
php -m
[PHP Modules]
apc
bcmath
bz2
calendar
Core
ctype
curl
date
dba
dom
ereg
exif
fileinfo
filter
ftp
gd
gettext
hash
iconv
imagick
intl
json
ldap
libxml
mbstring
memcache
memcached
mhash
mysql
mysqli
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_pgsql
pdo_sqlite
pgsql
Phar
posix
Reflection
session
shmop
SimpleXML
soap
sockets
SPL
sqlite3
standard
sysvmsg
sysvsem
sysvshm
tidy
tokenizer
wddx
xml
xmlreader
xmlwriter
zip
zlib
[Zend Modules]
ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 39531
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 39531
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
