Search Results

Search found 73 results on 3 pages for 'rsyslogd'.

Page 1/3 | 1 2 3  | Next Page >

• rsyslogd not monitoring all files

  - by Tom O'Connor

  So.. I've installed Logstash, and instead of using the logstash shipper (because it needs the JVM and is generally massive), I'm using rsyslogd with the following configuration. # Use traditional timestamp format $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat $IncludeConfig /etc/rsyslog.d/*.conf # Provides kernel logging support (previously done by rklogd) $ModLoad imklog # Provides support for local system logging (e.g. via logger command) $ModLoad imuxsock # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none;local6.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* /var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit /var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log In /etc/rsyslog.d/logstash.conf there are 28 file monitor blocks using imfile $ModLoad imfile # Load the imfile input module $ModLoad imklog # for reading kernel log messages $ModLoad imuxsock # for reading local syslog messages $InputFileName /var/log/rabbitmq/startup_err $InputFileTag rmq-err: $InputFileStateFile state-rmq-err $InputFileFacility local6 $InputRunFileMonitor .... $InputFileName /var/log/some.other.custom.log $InputFileTag cust-log: $InputFileStateFile state-cust-log $InputFileFacility local6 $InputRunFileMonitor .... *.* @@10.90.0.110:5514 There are 28 InputFileMonitor blocks, each monitoring a different custom application logfile.. If I run [root@secret-gm02 ~]# lsof|grep rsyslog rsyslogd 5380 root cwd DIR 253,0 4096 2 / rsyslogd 5380 root rtd DIR 253,0 4096 2 / rsyslogd 5380 root txt REG 253,0 278976 1015955 /sbin/rsyslogd rsyslogd 5380 root mem REG 253,0 58400 1868123 /lib64/libgcc_s-4.1.2-20080825.so.1 rsyslogd 5380 root mem REG 253,0 144776 1867778 /lib64/ld-2.5.so rsyslogd 5380 root mem REG 253,0 1718232 1867780 /lib64/libc-2.5.so rsyslogd 5380 root mem REG 253,0 23360 1867787 /lib64/libdl-2.5.so rsyslogd 5380 root mem REG 253,0 145872 1867797 /lib64/libpthread-2.5.so rsyslogd 5380 root mem REG 253,0 85544 1867815 /lib64/libz.so.1.2.3 rsyslogd 5380 root mem REG 253,0 53448 1867801 /lib64/librt-2.5.so rsyslogd 5380 root mem REG 253,0 92816 1868016 /lib64/libresolv-2.5.so rsyslogd 5380 root mem REG 253,0 20384 1867990 /lib64/rsyslog/lmnsd_ptcp.so rsyslogd 5380 root mem REG 253,0 53880 1867802 /lib64/libnss_files-2.5.so rsyslogd 5380 root mem REG 253,0 23736 1867800 /lib64/libnss_dns-2.5.so rsyslogd 5380 root mem REG 253,0 20768 1867988 /lib64/rsyslog/lmnet.so rsyslogd 5380 root mem REG 253,0 11488 1867982 /lib64/rsyslog/imfile.so rsyslogd 5380 root mem REG 253,0 24040 1867983 /lib64/rsyslog/imklog.so rsyslogd 5380 root mem REG 253,0 11536 1867987 /lib64/rsyslog/imuxsock.so rsyslogd 5380 root mem REG 253,0 13152 1867989 /lib64/rsyslog/lmnetstrms.so rsyslogd 5380 root mem REG 253,0 8400 1867992 /lib64/rsyslog/lmtcpclt.so rsyslogd 5380 root 0r REG 0,3 0 4026531848 /proc/kmsg rsyslogd 5380 root 1u IPv4 1200589517 0t0 TCP 10.10.10.90 t:40629->10.10.10.90:5514 (ESTABLISHED) rsyslogd 5380 root 2u IPv4 1200589527 0t0 UDP *:45801 rsyslogd 5380 root 3w REG 253,3 17999744 2621483 /var/log/messages rsyslogd 5380 root 4w REG 253,3 13383 2621484 /var/log/secure rsyslogd 5380 root 5w REG 253,3 7180 2621493 /var/log/maillog rsyslogd 5380 root 6w REG 253,3 43321 2621529 /var/log/cron rsyslogd 5380 root 7w REG 253,3 0 2621494 /var/log/spooler rsyslogd 5380 root 8w REG 253,3 0 2621495 /var/log/boot.log rsyslogd 5380 root 9r REG 253,3 1064271998 2621464 /var/log/custom-application.monolog.log rsyslogd 5380 root 10u unix 0xffff81081fad2e40 0t0 1200589511 /dev/log You can see that there are nowhere near 28 logfiles actually being read. I really had to get one file monitored, so I moved it to the top, and it picked it up, but I'd like to be able to monitor all 28+ files, and not have to worry. OS is Centos 5.5 Kernel 2.6.18-308.el5 rsyslogd 3.22.1, compiled with: FEATURE_REGEXP: Yes FEATURE_LARGEFILE: Yes FEATURE_NETZIP (message compression): Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No Atomic operations supported: Yes Runtime Instrumentation (slow code): No Questions: Why is rsyslogd only monitoring a very small subset of the files? How can I fix this so that all the files are monitored?

  Read the article

• Why aren't Heroku syslog drains logging to rsyslogd?

  - by Benjamin Oakes

  I'm having a problem using syslog drains as described in https://devcenter.heroku.com/articles/logging. To summarize, I have an Ubuntu 10.04 instance on EC2 that is running rsyslogd. I've also set up the security groups as they describe, and added a syslog drain using a command like heroku drains:add syslog://host1.example.com:514. I can send messages from the Heroku console to my rsyslogd instance via nc. I see them appear in the log file, so I know there isn't a firewall/security group issue.  However, Heroku does not seem to be forwarding log messages to the server that heroku drains lists. I would expect to see HTTP requests, Rails messages, etc. Is there something else I can try to figure this out? I'm new to rsyslogd, so I could easily be missing something.

  Read the article

• Rsyslog is not working properly, it does not log anything

  - by Victor Henriquez

  I'm running a Debian server and a couple of days ago my rsyslog started to behave very weird, the daemon is running but it doesn't seem to do anything. Many people use the system but I'm the only one with (legal) root access. I'm using the default rsyslogd configuration (if you think is relevant I'll attach it, but it's the one that comes with the package). After I rotated all the log files, they have remained empty: # ls -l /var/log/*.log -rw-r--r-- 1 root root 0 Jun 27 00:25 /var/log/alternatives.log -rw-r----- 1 root adm 0 Jun 26 13:03 /var/log/auth.log -rw-r----- 1 root adm 0 Jun 26 13:03 /var/log/daemon.log -rw-r--r-- 1 root root 0 Jun 27 00:25 /var/log/dpkg.log -rw-r----- 1 root adm 0 Jun 26 13:03 /var/log/kern.log -rw-r----- 1 root adm 0 Jun 26 13:03 /var/log/lpr.log -rw-r----- 1 root adm 0 Jun 26 13:03 /var/log/mail.log -rw-r----- 1 root adm 0 Jun 26 13:03 /var/log/user.log Any try to force a log writing does not have any effect: # logger hey # ls -l /var/log/messages -rw-r----- 1 root adm 0 Jun 26 13:03 /var/log/messages Lsof shows that rsyslogd does not have any log files opened: # lsof -p 1855 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME rsyslogd 1855 root cwd DIR 202,0 4096 2 / rsyslogd 1855 root rtd DIR 202,0 4096 2 / rsyslogd 1855 root txt REG 202,0 342076 21649 /usr/sbin/rsyslogd rsyslogd 1855 root mem REG 202,0 38556 32153 /lib/i386-linux-gnu/i686/cmov/libnss_nis-2.13.so rsyslogd 1855 root mem REG 202,0 79728 32165 /lib/i386-linux-gnu/i686/cmov/libnsl-2.13.so rsyslogd 1855 root mem REG 202,0 26456 32163 /lib/i386-linux-gnu/i686/cmov/libnss_compat-2.13.so rsyslogd 1855 root mem REG 202,0 297500 1061058 /usr/lib/rsyslog/imuxsock.so rsyslogd 1855 root mem REG 202,0 42628 32170 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so rsyslogd 1855 root mem REG 202,0 22784 1061106 /usr/lib/rsyslog/imklog.so rsyslogd 1855 root mem REG 202,0 1401000 32169 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so rsyslogd 1855 root mem REG 202,0 30684 32175 /lib/i386-linux-gnu/i686/cmov/librt-2.13.so rsyslogd 1855 root mem REG 202,0 9844 32157 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so rsyslogd 1855 root mem REG 202,0 117009 32154 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so rsyslogd 1855 root mem REG 202,0 79980 17746 /usr/lib/libz.so.1.2.3.4 rsyslogd 1855 root mem REG 202,0 18836 1061094 /usr/lib/rsyslog/lmnet.so rsyslogd 1855 root mem REG 202,0 117960 31845 /lib/i386-linux-gnu/ld-2.13.so rsyslogd 1855 root 0u unix 0xebe8e800 0t0 640 /dev/log rsyslogd 1855 root 3u FIFO 0,5 0t0 2474 /dev/xconsole rsyslogd 1855 root 4u unix 0xebe8e400 0t0 645 /var/spool/postfix/dev/log rsyslogd 1855 root 5r REG 0,3 0 4026532176 /proc/kmsg I was so frustrated that even reinstall the rsyslog package, but it still refuses to log anything: # apt-get remove --purge rsyslog # apt-get install rsyslog I thought someone had hacked the system, so run rkhunter, chkrootkit, unhide in an attempt to find hide processes / ports and nmap in a remote host to compare with the ports shown by netstat. And I know this doesn't mean anything, but all looks ok. The system also have an iptables firewall that is very restrictive with incoming / outgoing connections. This is driving me crazy, any idea what is going on here? [EDIT - disk space info] # df -h Filesystem Size Used Avail Use% Mounted on rootfs 24G 22G 629M 98% / /dev/root 24G 22G 629M 98% / devtmpfs 10M 112K 9.9M 2% /dev tmpfs 76M 48K 76M 1% /run tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 151M 40K 151M 1% /tmp tmpfs 151M 0 151M 0% /run/shm

  Read the article

• Rsyslogd not listening on port

  - by amorfis

  I installed rsyslogd on ubuntu server, started it and everything looks fine, but the port the server should listen on is not opened. ubuntu@node7:~$ sudo service rsyslog restart rsyslog stop/waiting rsyslog start/running, process 14114 Netstat shows it is not listening: ubuntu@node7:~$ netstat -tlan Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 320 172.22.0.17:22 10.8.8.38:61335 ESTABLISHED tcp6 0 0 :::22 :::* LISTEN tcp6 0 0 :::2776 :::* LISTEN tcp6 0 0 :::2777 :::* LISTEN tcp6 0 0 172.22.0.17:2777 172.22.0.11:56554 ESTABLISHED tcp6 0 0 172.22.0.17:2776 172.22.0.11:39780 ESTABLISHED This is how /etc/rsyslog.conf looks like (most comments omitted): ubuntu@node7:~$ cat /etc/rsyslog.conf ################# #### MODULES #### ################# $ModLoad imuxsock # provides support for local system logging $ModLoad imklog # provides kernel logging support (previously done by rklogd) $ModLoad imtcp $InputTCPServerRun 514 ########################### #### GLOBAL DIRECTIVES #### ########################### $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat $RepeatedMsgReduction on $WorkDirectory /var/spool/rsyslog $FileOwner syslog $FileGroup adm $FileCreateMode 0640 $DirCreateMode 0755 $Umask 0022 $PrivDropToUser syslog $PrivDropToGroup adm $IncludeConfig /etc/rsyslog.d/*.conf In /etc/rsyslog.d/35-server-per-host.conf I have following lines, and I suspect this can be the cause. What does it mean? # Stop processing of all non-local messages. You can process remote messages # on levels less than 35. :fromhost-ip,!isequal,"127.0.0.1" ~ and if it is, how could I change it to have server listening and receiving and logging messages? UPDATE: I commented out suspected line, but still it's not listening on port 514

  Read the article

• Rsyslog mail module not working

  - by Henry-Nicolas Tourneur

  Hi *, I would like to email snort alerts from my Debian Lenny fw. Syslog is sending log messages from the firewalls to a central rsyslog. On my central rsyslog, I got something like : $ModLoad ommail $ActionMailSMTPServer server.company.local $ActionMailFrom [email protected] $ActionMailTo [email protected] $ActionExecOnlyOnceEveryInterval 1 $template mailSubject,"[SNORT] Alert from %hostname%" $template mailBody,"Snort message\r\nmsg='%msg%'" $ActionMailSubject mailSubject if $msg regexp 'snort[[0-9]]: [[0-9]:[0-9]:[0-9]].*' then ommail:;mailBody But I doesn't get any mails, I even can trigger snort with something like ping -s 1400, it logs things like following but still no mail ! 2010-01-08T09:25:58+00:00 Hostname snort[4429]: [1:499:4] ICMP Large ICMP Packet [Classification: Potentially Bad Traffic] [Priority: 2]: {ICMP} ip_dest - ip_src Any idea ?

  Read the article

• syslog message format question

  - by razor

  is there a way with rsyslog or syslog-ng to allow tab delimited messages? I'm currently using rsyslog and the c syslog() function to generate syslog messages from within my app. The problem is that I use \t and they get translated into #011 when I view the logs. Ideally I'd like to have the other control chars translated except tab. is something like this possible?

  Read the article

• rsyslog server on windows

  - by user42327

  Is there a windows utility that functions as a rsyslog server?

  Read the article

• What sysadmin must do to run OS with damaged /lib/libc.so file ? / rsyslogd daemon logrotation / deny checking list of running processes

  - by Virtual_Lotos

  What sysadmin must do to run OS with damaged /lib/libc.so file ? In other words, how command interpreter should be configured to be able to run system with corrupted /lib/libc.so file ? Do I have to move it to /var catalog ? Does the command interpreter must be statically compiled or have setuid attribute or perhaps must be a symbolic link to /bin/sh or must be no larger than 2MB ? How to prevent a user from checking list of processes started by another user ? How do I forbid a user to see which processes are running by another user ? What do I have to keep in mind when I want to make rsyslogd daemon logrotation ?

  Read the article

• Properly Configured Rsyslog on CentOS

  - by Gaia

  I'm trying to configure Rsyslog 5.8.10 on CentOS 6.4 to send Apache's error and access logs to a remote server. It's working, but I have a couple questions. A) I would like to use as few queues (and resources) as possible. I send error logs to server A, send access logs to server A, send both logs in one stream to server B. Should I specify one queue per external service (2 queues) or one queue per stream (3 queues, as I have now)? This is what I have: $ActionResumeInterval 10 $ActionQueueSize 100000 $ActionQueueDiscardMark 97500 $ActionQueueHighWaterMark 80000 $ActionQueueType LinkedList $ActionQueueFileName logglyaccessqueue $ActionQueueCheckpointInterval 100 $ActionQueueMaxDiskSpace 1g $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on $ActionQueueTimeoutEnqueue 10 $ActionQueueDiscardSeverity 0 if $syslogtag startswith 'www-access' then @@logs-01.loggly.com:514;logglyaccess $ActionResumeInterval 10 $ActionQueueSize 100000 $ActionQueueDiscardMark 97500 $ActionQueueHighWaterMark 80000 $ActionQueueType LinkedList $ActionQueueFileName logglyerrorsqueue $ActionQueueCheckpointInterval 100 $ActionQueueMaxDiskSpace 1g $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on $ActionQueueTimeoutEnqueue 10 $ActionQueueDiscardSeverity 0 if $syslogtag startswith 'www-errors' then @@logs-01.loggly.com:514;logglyerrors $DefaultNetstreamDriverCAFile /etc/syslog.papertrail.crt # trust these CAs $ActionSendStreamDriver gtls # use gtls netstream driver $ActionSendStreamDriverMode 1 # require TLS $ActionSendStreamDriverAuthMode x509/name # authenticate by hostname $ActionResumeInterval 10 $ActionQueueSize 100000 $ActionQueueDiscardMark 97500 $ActionQueueHighWaterMark 80000 $ActionQueueType LinkedList $ActionQueueFileName papertrailqueue $ActionQueueCheckpointInterval 100 $ActionQueueMaxDiskSpace 1g $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on $ActionQueueTimeoutEnqueue 10 $ActionQueueDiscardSeverity 0 *.* @@logs.papertrailapp.com:XXXXX;papertrailstandard & ~ B) Does a queue block get used over and over by every send action that follows it or only by the first one or only until it encounters a send followed by a discard action (~)? C) How do I reset a queue block so that an upcoming send action does not use a queue at all? D) Does a TLS block get used over and over by every send action that follows it or only by the first one or only until it encounters a send followed by a discard action (~)? E) How do I reset a TLS block so that an upcoming send action does not use TLS at all? F) If I run rsyslog -N1 I get: rsyslogd -N1 rsyslogd: version 5.8.10, config validation run (level 1), master config /etc/rsyslog.conf rsyslogd: WARNING: rsyslogd is running in compatibility mode. Automatically generated config directives may interfer with your rsyslog.conf settings. We suggest upgrading your config and adding -c5 as the first rsyslogd option. rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad immark rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: MarkMessagePeriod 1200 rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad imuxsock rsyslogd: End of config validation run. Bye. Where do I place the -c5 so that it doesnt run in compatibility mode anymore?

  Read the article

• Linux clock loses 10 minutes every week

  - by PaKempf

  One of my linux server's clock loses 10 minutes every now and then, nearly every week. I update the time so it stays correct, and although it doesn't really bother me, i'd like to fix it. I've been searching around a bit. Nothing can be responsible in the crontab, and i can't find any related message in the logs. Some people seem to use ntp to fix that kind of issue, but i'd prefer not to use an unecessary component on it. Uname result : Linux unis-monitor 2.6.32-5-686 #1 SMP Mon Feb 25 01:04:36 UTC 2013 i686 GNU/Linux Cat message : cat messages Jul 14 06:25:06 unis-monitor rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="882" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'. Jul 15 06:25:05 unis-monitor rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="882" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'. Cat syslog cat syslog Jul 15 06:25:05 unis-monitor rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="882" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'. Jul 15 06:39:01 unis-monitor /USR/SBIN/CRON[15272]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete) Jul 15 07:09:01 unis-monitor /USR/SBIN/CRON[15465]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete) Jul 15 07:17:01 unis-monitor /USR/SBIN/CRON[15521]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Jul 15 07:39:01 unis-monitor /USR/SBIN/CRON[15662]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete) Jul 15 08:09:01 unis-monitor /USR/SBIN/CRON[15855]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete) Jul 15 08:17:01 unis-monitor /USR/SBIN/CRON[15911]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Jul 15 08:39:01 unis-monitor /USR/SBIN/CRON[16052]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete) Jul 15 09:09:01 unis-monitor /USR/SBIN/CRON[16273]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete) So if you have any clue of where to look or what i could use to monitor those date change ? Here is some more infos : the server is a virtual server hosted on HyperV on a win 2012 server. Don't know if it changes anything, seen the other servers hosted don't have this issue...

  Read the article

• Ubuntu server spontaneous reboot

  - by user1941407

  I have got two ubuntu 12.04 servers(xeon e3). Sometimes(several days) each server spontaneously reboots. HDDs and other hardware are ok. Which logfile can help find a reason of the problem? UPDATED. hardware: xeon e3 processor, intel server motherboard, 32gb ddr3 ecc, mdadm mirror hdd raid for system, mdadm ssd raid for database(postgres). Both servers have similar (not identical) components. Smart is OK. It seems that the problem is in the software. Python process and database are running on this servers. Syslog (time of reboot): Aug 23 13:42:23 xeon hddtemp[1411]: /dev/sdc: WDC WD15NPVT-00Z2TT0: 34 C Aug 23 13:42:23 xeon hddtemp[1411]: /dev/sdd: WDC WD15NPVT-00Z2TT0: 34 C Aug 23 13:43:24 xeon hddtemp[1411]: /dev/sdc: WDC WD15NPVT-00Z2TT0: 34 C Aug 23 13:43:24 xeon hddtemp[1411]: /dev/sdd: WDC WD15NPVT-00Z2TT0: 34 C Aug 23 13:44:14 xeon sensord: Chip: acpitz-virtual-0 Aug 23 13:44:14 xeon sensord: Adapter: Virtual device Aug 23 13:44:14 xeon sensord: temp1: 27.8 C Aug 23 13:44:14 xeon sensord: temp2: 29.8 C Aug 23 13:44:14 xeon sensord: Chip: coretemp-isa-0000 Aug 23 13:44:14 xeon sensord: Adapter: ISA adapter Aug 23 13:44:14 xeon sensord: Physical id 0: 37.0 C Aug 23 13:44:14 xeon sensord: Core 0: 37.0 C Aug 23 13:44:14 xeon sensord: Core 1: 37.0 C Aug 23 13:44:14 xeon sensord: Core 2: 37.0 C Aug 23 13:44:14 xeon sensord: Core 3: 37.0 C Aug 23 13:44:24 xeon hddtemp[1411]: /dev/sdc: WDC WD15NPVT-00Z2TT0: 34 C Aug 23 13:44:24 xeon hddtemp[1411]: /dev/sdd: WDC WD15NPVT-00Z2TT0: 34 C Aug 23 13:47:01 xeon kernel: imklog 5.8.6, log source = /proc/kmsg started. Aug 23 13:47:01 xeon rsyslogd: [origin software="rsyslogd" swVersion="5.8.6" x-pid="582" x-info="http://www.rsyslog.com"] start Aug 23 13:47:01 xeon rsyslogd: rsyslogd's groupid changed to 103 Aug 23 13:47:01 xeon rsyslogd: rsyslogd's userid changed to 101 Aug 23 13:47:00 xeon rsyslogd-2039: Could not open output pipe '/dev/xconsole' [try http://www.rsyslog.com/e/2039 ] Aug 23 13:47:01 xeon kernel: [ 0.000000] Initializing cgroup subsys cpuset Aug 23 13:47:01 xeon kernel: [ 0.000000] Initializing cgroup subsys cpu Aug 23 13:47:01 xeon kernel: [ 0.000000] Initializing cgroup subsys cpuacct Aug 23 13:47:01 xeon kernel: [ 0.000000] Linux version 3.11.0-26-generic (buildd@komainu) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #45~precise1-Ubuntu SMP Tue Jul 15 04:02:35 UTC 2014 (Ubuntu 3.11.0-26.45~precise1-generic 3.11.10.12) Aug 23 13:47:01 xeon kernel: [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.11.0-26-generic root=UUID=0daa7f53-6c74-47d2-873e-ebd339cd39b0 ro splash quiet vt.handoff=7 Aug 23 13:47:01 xeon kernel: [ 0.000000] KERNEL supported cpus: Aug 23 13:47:01 xeon kernel: [ 0.000000] Intel GenuineIntel Aug 23 13:47:01 xeon kernel: [ 0.000000] AMD AuthenticAMD Aug 23 13:47:01 xeon kernel: [ 0.000000] Centaur CentaurHauls Aug 23 13:47:01 xeon kernel: [ 0.000000] e820: BIOS-provided physical RAM map: Aug 23 13:47:01 xeon kernel: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009bbff] usable Aug 23 13:47:01 xeon kernel: [ 0.000000] BIOS-e820: [mem 0x000000000009bc00-0x000000000009ffff] reserved Dmseg - nothing strange.

  Read the article

• cups log kills ubuntu 12.04 and sudoer permissions changed

  - by peterretief

  I am using Ubuntu 12.04 as a desktop and recently had a weird crash with the log file for cups filling up the entire drive and not letting me back in, also what changed was /var/lib/sudo had changed from root to peter (me) I didn't make this change - I checked the history! I set the sudoers back to root and capped the max size for cups log Anyone had a similar experience? It feels like someone is messing around with my settings Is there any way to trace how the error occurred? Logs auth.log Jan 1 02:04:13 peter-desktop lightdm: pam_unix(lightdm:session): session opened for user lightdm by (uid=0) Jan 1 02:04:13 peter-desktop lightdm: pam_ck_connector(lightdm:session): nox11 mode, ignoring PAM_TTY :0 Jan 1 02:06:53 peter-desktop lightdm: pam_unix(lightdm:session): session opened for user lightdm by (uid=0) Jan 1 02:06:53 peter-desktop lightdm: pam_ck_connector(lightdm:session): nox11 mode, ignoring PAM_TTY :0 syslog Jan 1 02:04:13 peter-desktop rsyslogd: [origin software="rsyslogd" swVersion="5.8.6" x-pid="903" x-info="http://www.rsyslog.com"] start Jan 1 02:04:13 peter-desktop rsyslogd: rsyslogd's groupid changed to 103 Jan 1 02:04:13 peter-desktop rsyslogd: rsyslogd's userid changed to 101 Jan 1 02:04:13 peter-desktop rsyslogd-2039: Could not open output pipe '/dev/xconsole' [try http://www.rsyslog.com/e/2039 ] Jan 1 02:04:13 peter-desktop bluetoothd[898]: Failed to init gatt_example plugin Jan 1 02:04:13 peter-desktop kernel: [ 0.000000] Initializing cgroup subsys cpuset Jan 1 02:04:13 peter-desktop kernel: [ 0.000000] Initializing cgroup subsys cpu Jan 1 02:04:13 peter-desktop kernel: [ 0.000000] Linux version 3.2.0-25-generic-pae (buildd@palmer) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #40-Ubuntu SMP Wed May 23 22:11:24 UTC 2012 (Ubuntu 3.2.0-25.40-generic-pae 3.2.18) Jan 1 02:04:13 peter-desktop kernel: [ 0.000000] KERNEL supported cpus: Jan 1 02:04:13 peter-desktop kernel: [ 0.000000] Intel GenuineIntel Jan 1 02:04:13 peter-desktop kernel: [ 0.000000] AMD AuthenticAMD Jan 1 02:04:13 peter-desktop kernel: [ 0.000000] NSC Geode by NSC

  Read the article

• Hang while starting several daemons [solved]

  - by Adrian Lang

  I’m running a Debian Squeeze AMD64 server. Target runlevel after boot is runlevel 2, which includes rsyslogd, cron, sshd and some other stuff, but not dovecot, postfix, apache2, etc. The system fails to reach runlevel 2 with several symptoms: The system hangs at trying to start rsyslogd Booting into runlevel 1 works, then login from the console works Starting rsyslogd from runlevel 1 via /etc/init.d/rsyslog hangs Starting runlevel 2 with rsyslogd disabled works But then, logging in via console fails: I get the motd, and then nothing Starting sshd from runlevel 1 succeeds But then, I cannot login via ssh. Sometimes password ssh login gives me the motd and then nothing, sometimes not even this. Trying to offer a public key seems to annoy the sshd enough to not talk to me any further. When rebooting from runlevel 1, the server hangs at trying to stop apache2 (which is not running, so this really should be trivial). Trying to stop apache2 when logged in in runleve 1 does hang as well. And that’s just the stuff which fails all the time. RAM has been tested, dmesg shows no problems. I have no clue. Update: (shortened) output from rsyslogd -c4 -d called in runlevel 1 rsyslogd 4.6.4 startup, compatibility mode 4, module path '' caller requested object 'net', not found (iRet -3003) Requested to load module 'lmnet' loading module '/user/lib/rsyslog/lmnet.so' module of type 2 being loaded conf.c requested ref for 'lmnet', refcount 1 rsylog runtime initialized, version 4.6.4, current users 1 syslogd.c requested ref for 'lmnet', refcount now 2 I can kill rsyslogd with Strg+C, then. /var/log shows none of the configured log files, though. Update2: Thanks to @DerfK I still have no clue, but at least I narrowed down the problem. I’m now testing with /etc/init.d/apache2 stop (without an apache2 running, of course) which hangs as well and looks like an even more obvious failure. After some testing I found out that a file with one single line: /usr/sbin/apache2ctl configtest /dev/null 2&1 hangs, while the same line executed in an interactive shell works. I was not able to further reduce this line while, i. e. every single part, the stream redirections and the commando itself is necessary to reproduce the hang. @DerfK also pointed me to strace which gave a shallow hint about what kind of hang we have here: wait4(-1for the init scripts futex(0xsomepointer, FUTEX_WAIT_PRIVATE, 2, NULL for rsyslogd / apache2 binaries called by the init scripts The system was installed as a Debian Lenny by my hoster in autumn 2011, I upgraded it to Squeeze immediately and kept it up to date with Squeeze, which then used to be testing. There were no big changes, though. I guess I never tried to reboot the system before. Update3: I found the problem. My /etc/nsswitch.conf specified ldap as hosts lookup backup, which is not available at that time of the boot. Relying on dns solely fixes my boot problems.

  Read the article

• Hang while starting several daemons

  - by Adrian Lang

  I’m running a Debian Squeeze AMD64 server. Target runlevel after boot is runlevel 2, which includes rsyslogd, cron, sshd and some other stuff, but not dovecot, postfix, apache2, etc. The system fails to reach runlevel 2 with several symptoms: The system hangs at trying to start rsyslogd Booting into runlevel 1 works, then login from the console works Starting rsyslogd from runlevel 1 via /etc/init.d/rsyslog hangs Starting runlevel 2 with rsyslogd disabled works But then, logging in via console fails: I get the motd, and then nothing Starting sshd from runlevel 1 succeeds But then, I cannot login via ssh. Sometimes password ssh login gives me the motd and then nothing, sometimes not even this. Trying to offer a public key seems to annoy the sshd enough to not talk to me any further. When rebooting from runlevel 1, the server hangs at trying to stop apache2 (which is not running, so this really should be trivial). Trying to stop apache2 when logged in in runleve 1 does hang as well. And that’s just the stuff which fails all the time. RAM has been tested, dmesg shows no problems. I have no clue. Update: (shortened) output from rsyslogd -c4 -d called in runlevel 1 rsyslogd 4.6.4 startup, compatibility mode 4, module path '' caller requested object 'net', not found (iRet -3003) Requested to load module 'lmnet' loading module '/user/lib/rsyslog/lmnet.so' module of type 2 being loaded conf.c requested ref for 'lmnet', refcount 1 rsylog runtime initialized, version 4.6.4, current users 1 syslogd.c requested ref for 'lmnet', refcount now 2 I can kill rsyslogd with Strg+C, then. /var/log shows none of the configured log files, though. Update2: Thanks to @DerfK I still have no clue, but at least I narrowed down the problem. I’m now testing with /etc/init.d/apache2 stop (without an apache2 running, of course) which hangs as well and looks like an even more obvious failure. After some testing I found out that a file with one single line: /usr/sbin/apache2ctl configtest /dev/null 2&1 hangs, while the same line executed in an interactive shell works. I was not able to further reduce this line while, i. e. every single part, the stream redirections and the commando itself is necessary to reproduce the hang. @DerfK also pointed me to strace which gave a shallow hint about what kind of hang we have here: wait4(-1for the init scripts futex(0xsomepointer, FUTEX_WAIT_PRIVATE, 2, NULL for rsyslogd / apache2 binaries called by the init scripts The system was installed as a Debian Lenny by my hoster in autumn 2011, I upgraded it to Squeeze immediately and kept it up to date with Squeeze, which then used to be testing. There were no big changes, though. I guess I never tried to reboot the system before.

  Read the article

• Customize rsyslogd messages to show the sender of the message; not the receiver

  - by Nimmy Lebby

  I'm forwarding the WiFi router's log messages to our sysadmin box (sb3). This is the stanza in /etc/rsyslog.conf: # WiFi router log :fromhost-ip, isequal,'10.3.291.2' /var/log/wifi-router.log & ~ However, the log looks like this: Dec 23 10:41:58 sb3 dnsmasq-dhcp[253]: DHCPACK(br0) 10.3.292.133 xx:xx:xx:xx:xx:xx dg-ipad I want to customize so that anything logged to wifi-router.log does not mention sb3 but indicates the sender of the log message. How would I do this?

  Read the article

• 10.10 freezing when sftp downloading

  - by aGr

  My ubuntu 10.10 is freezing during downloading from my other computer on sftp. I thought it might be some nautilus issues so I tried it via command line and I got the same thing - after few minutes the whole computer freezes. Mostly the numlock led is blinking (I've heard somewhere that this means a kernel panic), but not in 100% cases. I dunno if that helps but here is a log from /var/log/message in the time that this happened. At least I hope so - it wasn't that big, when it happened before. But this looks quite "errorish", right? (isn't complete - see bottom) Jan 5 17:57:49 tomas-ntb kernel: imklog 4.2.0, log source = /proc/kmsg started. Jan 5 17:57:49 tomas-ntb rsyslogd: [origin software="rsyslogd" swVersion="4.2.0" x-pid="953" x-info="http://www.rsyslog.com"] (re)start Jan 5 17:57:49 tomas-ntb rsyslogd: rsyslogd's groupid changed to 103 Jan 5 17:57:49 tomas-ntb rsyslogd: rsyslogd's userid changed to 101 Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] Initializing cgroup subsys cpuset Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] Initializing cgroup subsys cpu Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] Linux version 2.6.35-24-generic (buildd@vernadsky) (gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5) ) #42-Ubuntu SMP Thu Dec 2 01:41:57 UTC 2010 (Ubuntu 2.6.35-24.42-generic 2.6.35.8) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-provided physical RAM map: Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 0000000000000000 - 000000000009e800 (usable) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 000000000009e800 - 00000000000a0000 (reserved) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 0000000000100000 - 00000000dffa8000 (usable) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 00000000dffa8000 - 00000000dffb0000 (ACPI NVS) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 00000000dffb0000 - 00000000dffbff00 (ACPI data) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 00000000dffbff00 - 00000000dfff0000 (ACPI NVS) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 00000000dfff0000 - 00000000e0000000 (reserved) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 0000000100000000 - 0000000120000000 (usable) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] BIOS-e820: 0000000120000000 - 0000000140000000 (reserved) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] Notice: NX (Execute Disable) protection cannot be enabled: non-PAE kernel! Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] DMI 2.5 present. Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] AMI BIOS detected: BIOS may corrupt low RAM, working around it. Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] last_pfn = 0xdffa8 max_arch_pfn = 0x100000 Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] Scanning 0 areas for low memory corruption Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] modified physical RAM map: Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] modified: 0000000000000000 - 0000000000010000 (reserved) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] modified: 0000000000010000 - 000000000009e800 (usable) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] modified: 000000000009e800 - 00000000000a0000 (reserved) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] modified: 00000000000e0000 - 0000000000100000 (reserved) Jan 5 17:57:49 tomas-ntb kernel: [ 0.000000] modified: 0000000000100000 - 00000000dffa8000 (usable) ... to long - for whole and better view click here

  Read the article

• SELinux - Allow multiple services access to same /home/dir

  - by Mike Purcell

  I currently have SELinux enabled and have been able to configure apache to allow access to /home/src/web with a chcon command granting the 'httpd_sys_content_t' type. But now I am trying to serve the rsyslogd.conf file from the same directory, but every time I start rsyslogd I see an entry in my audit log saying that rsyslogd was denied access. My question is, is it possible to grant two applications the ability to access the same directory, while still keeping SELinux enabled? Current perms on /home/src: drwxr-xr-x. src src unconfined_u:object_r:httpd_sys_content_t:s0 src Audit log message: type=AVC msg=audit(1349113476.272:1154): avc: denied { search } for pid=9975 comm="rsyslogd" name="/" dev=dm-2 ino=2 scontext=unconfined_u:system_r:syslogd_t:s0 tcontext=system_u:object_r:home_root_t:s0 tclass=dir type=SYSCALL msg=audit(1349113476.272:1154): arch=c000003e syscall=2 success=no exit=-13 a0=7f9ef0c027f5 a1=0 a2=1b6 a3=0 items=0 ppid=9974 pid=9975 auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=30 comm="rsyslogd" exe="/sbin/rsyslogd" subj=unconfined_u:system_r:syslogd_t:s0 key=(null) -- Edit -- Came across this post, which is sort of what I am trying to accomplish. However when I viewed the list of allowed sebool params, the only relating to syslog was: syslogd_disable_trans (SELinux Service Protection), seems like I can maintain the current SELinux 'type' on the /home/src/ dir, but set the bool on syslogd_disable_trans to false. I wonder if there is a better approach?

  Read the article

• No sound lenovo t60 alsa ad1981 iec958

  - by Nate

  Any help on getting the sound to come through my lenovo t60 build in speakers, headphones, or mic would greatly be appreciated. The three buttons to increase, decrease sound seem to work. Bios has sound card enabled and the buttons beep when pressed. When going to Utube or playing music, no sound is heard. Thanks Nate Feb 23 - Didn't see anything specific in the sys logs with Rhythmbox when connecting my ipod. Rhythmbox is playing, but still no sound. Here is the syslog details for today. Output is set to analog output. Feb 23 17:42:32 itgis01398 rsyslogd: [origin software="rsyslogd" swVersion="4.2.0" x-pid="824" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'. Feb 23 17:42:33 itgis01398 rsyslogd: [origin software="rsyslogd" swVersion="4.2.0" x-pid="824" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'. Feb 23 17:42:49 itgis01398 anacron[968]: Job `cron.daily' terminated Feb 23 17:42:49 itgis01398 anacron[968]: Job `cron.weekly' started Feb 23 17:42:49 itgis01398 anacron[12067]: Updated timestamp for job `cron.weekly' to 2011-02-23 Feb 23 17:42:53 itgis01398 anacron[968]: Job `cron.weekly' terminated Feb 23 17:42:53 itgis01398 anacron[968]: Normal exit (2 jobs run) Feb 23 18:01:19 itgis01398 kernel: [ 2731.324067] usb 1-5: new high speed USB device using ehci_hcd and address 3 Feb 23 18:01:19 itgis01398 kernel: [ 2731.482879] Initializing USB Mass Storage driver... Feb 23 18:01:19 itgis01398 kernel: [ 2731.483061] usb-storage 1-5:1.0: Quirks match for vid 05ac pid 1205: 10 Feb 23 18:01:19 itgis01398 kernel: [ 2731.483116] scsi6 : usb-storage 1-5:1.0 Feb 23 18:01:19 itgis01398 kernel: [ 2731.483306] usbcore: registered new interface driver usb-storage Feb 23 18:01:19 itgis01398 kernel: [ 2731.483310] USB Mass Storage support registered. Feb 23 18:01:20 itgis01398 kernel: [ 2732.481116] scsi 6:0:0:0: Direct-Access Apple iPod 1.62 PQ: 0 ANSI: 0 Feb 23 18:01:20 itgis01398 kernel: [ 2732.482466] sd 6:0:0:0: Attached scsi generic sg2 type 0 Feb 23 18:01:20 itgis01398 kernel: [ 2732.485095] sd 6:0:0:0: [sdb] Adjusting the sector count from its reported value: 7999488 Feb 23 18:01:20 itgis01398 kernel: [ 2732.485110] sd 6:0:0:0: [sdb] 7999487 512-byte logical blocks: (4.09 GB/3.81 GiB) Feb 23 18:01:20 itgis01398 kernel: [ 2732.487933] sd 6:0:0:0: [sdb] Write Protect is off Feb 23 18:01:20 itgis01398 kernel: [ 2732.487941] sd 6:0:0:0: [sdb] Mode Sense: 64 00 00 08 Feb 23 18:01:20 itgis01398 kernel: [ 2732.487947] sd 6:0:0:0: [sdb] Assuming drive cache: write through Feb 23 18:01:20 itgis01398 kernel: [ 2732.489927] sd 6:0:0:0: [sdb] Adjusting the sector count from its reported value: 7999488 Feb 23 18:01:20 itgis01398 kernel: [ 2732.491150] sd 6:0:0:0: [sdb] Assuming drive cache: write through Feb 23 18:01:20 itgis01398 kernel: [ 2732.491163] sdb: sdb1 sdb2 Feb 23 18:01:20 itgis01398 kernel: [ 2732.510428] sd 6:0:0:0: [sdb] Adjusting the sector count from its reported value: 7999488 Feb 23 18:01:20 itgis01398 kernel: [ 2732.511288] sd 6:0:0:0: [sdb] Assuming drive cache: write through Feb 23 18:01:20 itgis01398 kernel: [ 2732.511297] sd 6:0:0:0: [sdb] Attached SCSI removable disk Feb 23 18:01:21 itgis01398 kernel: [ 2733.746675] FAT: invalid media value (0x2f) Feb 23 18:01:21 itgis01398 kernel: [ 2733.746682] VFS: Can't find a valid FAT filesystem on dev sdb1. Feb 23 18:01:22 itgis01398 upstart-udev-bridge[330]: Env must be KEY=VALUE pairs Feb 23 18:02:07 itgis01398 kernel: [ 2780.115826] sd 6:0:0:0: [sdb] Unhandled sense code Feb 23 18:02:07 itgis01398 kernel: [ 2780.115835] sd 6:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Feb 23 18:02:07 itgis01398 kernel: [ 2780.115844] sd 6:0:0:0: [sdb] Sense Key : Medium Error [current] Feb 23 18:02:07 itgis01398 kernel: [ 2780.115855] Info fld=0x0 Feb 23 18:02:07 itgis01398 kernel: [ 2780.115859] sd 6:0:0:0: [sdb] Add. Sense: Unrecovered read error Feb 23 18:02:07 itgis01398 kernel: [ 2780.115870] sd 6:0:0:0: [sdb] CDB: Read(10): 28 00 00 08 fd e9 00 00 f0 00 Feb 23 18:02:07 itgis01398 kernel: [ 2780.115892] end_request: I/O error, dev sdb, sector 589289 Feb 23 18:02:49 itgis01398 kernel: [ 2821.351464] sd 6:0:0:0: [sdb] Unhandled sense code Feb 23 18:02:49 itgis01398 kernel: [ 2821.351473] sd 6:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Feb 23 18:02:49 itgis01398 kernel: [ 2821.351482] sd 6:0:0:0: [sdb] Sense Key : Medium Error [current] Feb 23 18:02:49 itgis01398 kernel: [ 2821.351493] Info fld=0x0 Feb 23 18:02:49 itgis01398 kernel: [ 2821.351497] sd 6:0:0:0: [sdb] Add. Sense: No additional sense information Feb 23 18:02:49 itgis01398 kernel: [ 2821.351507] sd 6:0:0:0: [sdb] CDB: Read(10): 28 00 00 08 fe d9 00 00 10 00 Feb 23 18:02:49 itgis01398 kernel: [ 2821.351530] end_request: I/O error, dev sdb, sector 589529 Feb 23 18:17:01 itgis01398 CRON[12709]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) volume is all of the way up.

  Read the article

• SmartOS reboots spontaneously

  - by Alex

  I run a SmartOS system on a Hetzner EX4S (Intel Core i7-2600, 32G RAM, 2x3Tb SATA HDD). There are six virtual machines on the host: [root@10-bf-48-7f-e7-03 ~]# vmadm list UUID TYPE RAM STATE ALIAS d2223467-bbe5-4b81-a9d1-439e9a66d43f KVM 512 running xxxx1 5f36358f-68fa-4351-b66f-830484b9a6ee KVM 1024 running xxxx2 d570e9ac-9eac-4e4f-8fda-2b1d721c8358 OS 1024 running xxxx3 ef88979e-fb7f-460c-bf56-905755e0a399 KVM 1024 running xxxx4 d8e06def-c9c9-4d17-b975-47dd4836f962 KVM 4096 running xxxx5 4b06fe88-db6e-4cf3-aadd-e1006ada7188 KVM 9216 running xxxx5 [root@10-bf-48-7f-e7-03 ~]# The host reboots several times a week with no crash dump in /var/crash and no messages in the /var/adm/messages log. Basically /var/adm/messages looks like there was a hard reset: 2012-11-23T08:54:43.210625+00:00 10-bf-48-7f-e7-03 rsyslogd: -- MARK -- 2012-11-23T09:14:43.187589+00:00 10-bf-48-7f-e7-03 rsyslogd: -- MARK -- 2012-11-23T09:34:43.165100+00:00 10-bf-48-7f-e7-03 rsyslogd: -- MARK -- 2012-11-23T09:54:43.142065+00:00 10-bf-48-7f-e7-03 rsyslogd: -- MARK -- 2012-11-23T10:14:43.119365+00:00 10-bf-48-7f-e7-03 rsyslogd: -- MARK -- 2012-11-23T10:34:43.096351+00:00 10-bf-48-7f-e7-03 rsyslogd: -- MARK -- 2012-11-23T10:54:43.073821+00:00 10-bf-48-7f-e7-03 rsyslogd: -- MARK -- 2012-11-23T10:57:55.610954+00:00 10-bf-48-7f-e7-03 genunix: [ID 540533 kern.notice] #015SunOS Release 5.11 Version joyent_20121018T224723Z 64-bit 2012-11-23T10:57:55.610962+00:00 10-bf-48-7f-e7-03 genunix: [ID 299592 kern.notice] Copyright (c) 2010-2012, Joyent Inc. All rights reserved. 2012-11-23T10:57:55.610967+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: lgpg 2012-11-23T10:57:55.610971+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: tsc 2012-11-23T10:57:55.610974+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: msr 2012-11-23T10:57:55.610978+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: mtrr 2012-11-23T10:57:55.610981+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: pge 2012-11-23T10:57:55.610984+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: de 2012-11-23T10:57:55.610987+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: cmov 2012-11-23T10:57:55.610995+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: mmx 2012-11-23T10:57:55.611000+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: mca 2012-11-23T10:57:55.611004+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: pae 2012-11-23T10:57:55.611008+00:00 10-bf-48-7f-e7-03 unix: [ID 223955 kern.info] x86_feature: cv8 The problem is that sometimes the host loses the network interface on reboot so we need to perform a manual hardware reset to bring it back. We do not have physical or virtual access to the server console - no KVM, no iLO or anything like this. So, the only way to debug is to analyze crash dumps/log files. I am not a SmartOS/Solaris expert so I am not sure how to proceed. Is there any equivalent of Linux netconsole for SmartOS? Can I just redirect the console output to the network port somehow? Maybe I am missing something obvious and crash information is located somewhere else.

  Read the article

• Remote reboot over ssh does not restart

  - by Finn Årup Nielsen

  I would like to remotely reboot my Ubuntu 12.04 LTS server via ssh. I do sudo reboot and I loose connection and the server connection does not reappear. It does not ping. When I go the the physical computer with a screen attached I see a black screen and hear that the server is still on. I do a hard power off (press power on button for a few seconds) and the server halts. After I press power on the server boots with no problem. As far as I remember the remote reboot has previously worked on that server. I wonder if sudo reboot & will help? I suppose I could also try sudo shutdown -r and see if that does any difference. I have listed an excerpt of /etc/log/syslog below. The last thing it records is the stopping of the logging. Oct 24 10:14:49 servername kernel: [1354427.594709] init: cron main process (1060) killed by TERM signal Oct 24 10:14:49 servername kernel: [1354427.594908] init: irqbalance main process (1080) killed by TERM signal Oct 24 10:14:49 servername kernel: [1354427.595299] init: tty1 main process (1424) killed by TERM signal Oct 24 10:14:49 servername kernel: [1354427.637747] init: plymouth-upstart-bridge main process (20873) terminated with status 1 Oct 24 10:14:49 servername kernel: Kernel logging (proc) stopped. Oct 24 10:14:49 servername rsyslogd: [origin software="rsyslogd" swVersion="5.8.6" x-pid="876" x-info="http://www.rsyslog.com"] exiting on signal 15. Oct 24 10:25:34 servername kernel: imklog 5.8.6, log source = /proc/kmsg started. Oct 24 10:25:34 servername rsyslogd: [origin software="rsyslogd" swVersion="5.8.6" x-pid="862" x-info="http://www.rsyslog.com"] start

  Read the article

• Randomly displayed flashing lines, no response to all shortcuts, just power off. [syslog included]

  - by B. Roland

  Hello! I have an old machine, and I want to use for that to learn employees how to use Ubuntu, and to be easyer to switch from Windows. I've been installed 10.04, and updated, but this strange stuff is happend. Graphical installion failed, same strange thing. With alternate workd. Sometimes, when I boot up, a boot message displayed: Keyboard failure..., often diplayed after reboot, and after shutdown, when I haven't plugged off from AC. I replaced the keyboard yet, same failure... If I powered off, and plugged off from AC, no keyboard problems displayed in boot time. Details Configuration: Dell OptiPlex GX60 - in original cover, no changes. 256 MB DDR 166 MHz Intel® Celeron® Processor 2.40 GHz Dell 0C3207 Base Board I know, that is not enough, but I have three other Nec compuers, with nearly similar config, and they works well with 9.10, 10.04, 10.10. Live CDs I've been tried with 10.04 and 10.10, but the problem is displayed too. With 9.10 no strange things displayed, but it froze, during a simple apt-get install. Syslog An error loop is logged here, but I paste the whole startup and error lines. The flashing lines are displayed sometimes immediately after login, but sometimes after 10 minutes, but once occured, that nothing happend. Strange thing is displayed immediately after login: here. An other boot, after some minutes, strange lines, and loop in log appeard: here. The loop should be that: Jan 23 00:20:08 machine_name kernel: [ 46.782212] [drm:i915_gem_entervt_ioctl] *ERROR* Reenabling wedged hardware, good luck Jan 23 00:20:08 machine_name kernel: [ 47.100033] [drm:i915_hangcheck_elapsed] *ERROR* Hangcheck timer elapsed... GPU hung Jan 23 00:20:08 machine_name kernel: [ 47.100045] render error detected, EIR: 0x00000000 Jan 23 00:20:08 machine_name kernel: [ 47.101487] [drm:i915_do_wait_request] *ERROR* i915_do_wait_request returns -5 (awaiting 16 at 9) Jan 23 00:20:11 machine_name kernel: [ 49.152020] [drm:i915_gem_idle] *ERROR* hardware wedged Jan 23 00:20:11 machine_name gdm-simple-slave[1245]: WARNING: Unable to load file '/etc/gdm/custom.conf': No such file or directory Jan 23 00:20:11 machine_name acpid: client 1239[0:0] has disconnected Jan 23 00:20:11 machine_name acpid: client connected from 1247[0:0] Jan 23 00:20:11 machine_name acpid: 1 client rule loaded UPDATE Added syslog things: before errors, error loop, the complete shutdown(after the big updates): Jan 28 20:40:30 machine_name rtkit-daemon[1339]: Sucessfully called chroot. Jan 28 20:40:30 machine_name rtkit-daemon[1339]: Sucessfully dropped privileges. Jan 28 20:40:30 machine_name rtkit-daemon[1339]: Sucessfully limited resources. Jan 28 20:40:30 machine_name rtkit-daemon[1339]: Running. Jan 28 20:40:30 machine_name rtkit-daemon[1339]: Watchdog thread running. Jan 28 20:40:30 machine_name rtkit-daemon[1339]: Canary thread running. Jan 28 20:40:30 machine_name rtkit-daemon[1339]: Sucessfully made thread 1337 of process 1337 (n/a) owned by '1001' high priority at nice level -11. Jan 28 20:40:30 machine_name rtkit-daemon[1339]: Supervising 1 threads of 1 processes of 1 users. Jan 28 20:40:32 machine_name rtkit-daemon[1339]: Sucessfully made thread 1345 of process 1337 (n/a) owned by '1001' RT at priority 5. Jan 28 20:40:32 machine_name rtkit-daemon[1339]: Supervising 2 threads of 1 processes of 1 users. Jan 28 20:40:32 machine_name rtkit-daemon[1339]: Sucessfully made thread 1349 of process 1337 (n/a) owned by '1001' RT at priority 5. Jan 28 20:40:32 machine_name rtkit-daemon[1339]: Supervising 3 threads of 1 processes of 1 users. Jan 28 20:40:37 machine_name pulseaudio[1337]: ratelimit.c: 2 events suppressed Jan 28 20:41:33 machine_name AptDaemon: INFO: Initializing daemon Jan 28 20:41:44 machine_name kernel: [ 167.691563] lo: Disabled Privacy Extensions Jan 28 20:47:33 machine_name AptDaemon: INFO: Quiting due to inactivity Jan 28 20:47:33 machine_name AptDaemon: INFO: Shutdown was requested Jan 28 20:59:50 machine_name kernel: [ 1253.840513] lo: Disabled Privacy Extensions Jan 28 21:17:02 machine_name CRON[1874]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Jan 28 21:17:38 machine_name kernel: [ 2321.553239] lo: Disabled Privacy Extensions Jan 28 22:07:44 machine_name kernel: [ 5327.840254] lo: Disabled Privacy Extensions Jan 28 22:17:02 machine_name CRON[2665]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Jan 28 22:32:38 machine_name sudo: pam_sm_authenticate: Called Jan 28 22:32:38 machine_name sudo: pam_sm_authenticate: username = [some_user] Jan 28 22:32:38 machine_name sudo: pam_sm_authenticate: /home/some_user is already mounted Jan 28 22:57:03 machine_name kernel: [ 8286.641472] lo: Disabled Privacy Extensions Jan 28 22:57:24 machine_name sudo: pam_sm_authenticate: Called Jan 28 22:57:24 machine_name sudo: pam_sm_authenticate: username = [some_user] Jan 28 22:57:24 machine_name sudo: pam_sm_authenticate: /home/some_user is already mounted Jan 28 23:07:42 machine_name kernel: [ 8925.272030] [drm:i915_hangcheck_elapsed] *ERROR* Hangcheck timer elapsed... GPU hung Jan 28 23:07:42 machine_name kernel: [ 8925.272048] render error detected, EIR: 0x00000000 Jan 28 23:07:42 machine_name kernel: [ 8925.272093] [drm:i915_do_wait_request] *ERROR* i915_do_wait_request returns -5 (awaiting 171453 at 171452) Jan 28 23:07:45 machine_name kernel: [ 8928.868041] [drm:i915_gem_idle] *ERROR* hardware wedged Jan 28 23:08:10 machine_name acpid: client 925[0:0] has disconnected Jan 28 23:08:10 machine_name acpid: client connected from 8127[0:0] Jan 28 23:08:10 machine_name acpid: 1 client rule loaded Jan 28 23:08:11 machine_name kernel: [ 8955.046248] [drm:i915_gem_entervt_ioctl] *ERROR* Reenabling wedged hardware, good luck Jan 28 23:08:12 machine_name kernel: [ 8955.364016] [drm:i915_hangcheck_elapsed] *ERROR* Hangcheck timer elapsed... GPU hung Jan 28 23:08:12 machine_name kernel: [ 8955.364027] render error detected, EIR: 0x00000000 Jan 28 23:08:12 machine_name kernel: [ 8955.364407] [drm:i915_do_wait_request] *ERROR* i915_do_wait_request returns -5 (awaiting 171457 at 171452) Jan 28 23:08:14 machine_name kernel: [ 8957.472025] [drm:i915_gem_idle] *ERROR* hardware wedged Jan 28 23:08:14 machine_name acpid: client 8127[0:0] has disconnected Jan 28 23:08:14 machine_name acpid: client connected from 8141[0:0] Jan 28 23:08:14 machine_name acpid: 1 client rule loaded Jan 28 23:08:15 machine_name kernel: [ 8958.671722] [drm:i915_gem_entervt_ioctl] *ERROR* Reenabling wedged hardware, good luck Jan 28 23:08:15 machine_name kernel: [ 8958.988015] [drm:i915_hangcheck_elapsed] *ERROR* Hangcheck timer elapsed... GPU hung Jan 28 23:08:15 machine_name kernel: [ 8958.988026] render error detected, EIR: 0x00000000 Jan 28 23:08:15 machine_name kernel: [ 8958.989400] [drm:i915_do_wait_request] *ERROR* i915_do_wait_request returns -5 (awaiting 171459 at 171452) Jan 28 23:08:16 machine_name init: tty4 main process (848) killed by TERM signal Jan 28 23:08:16 machine_name init: tty5 main process (856) killed by TERM signal Jan 28 23:08:16 machine_name NetworkManager: nm_signal_handler(): Caught signal 15, shutting down normally. Jan 28 23:08:16 machine_name init: tty2 main process (874) killed by TERM signal Jan 28 23:08:16 machine_name init: tty3 main process (875) killed by TERM signal Jan 28 23:08:16 machine_name init: tty6 main process (877) killed by TERM signal Jan 28 23:08:16 machine_name init: cron main process (890) killed by TERM signal Jan 28 23:08:16 machine_name init: tty1 main process (1146) killed by TERM signal Jan 28 23:08:16 machine_name avahi-daemon[644]: Got SIGTERM, quitting. Jan 28 23:08:16 machine_name avahi-daemon[644]: Leaving mDNS multicast group on interface eth0.IPv4 with address 10.238.11.134. Jan 28 23:08:16 machine_name acpid: exiting Jan 28 23:08:16 machine_name init: avahi-daemon main process (644) terminated with status 255 Jan 28 23:08:17 machine_name kernel: Kernel logging (proc) stopped. Jan 28 23:09:00 machine_name kernel: imklog 4.2.0, log source = /proc/kmsg started. Jan 28 23:09:00 machine_name rsyslogd: [origin software="rsyslogd" swVersion="4.2.0" x-pid="516" x-info="http://www.rsyslog.com"] (re)start Jan 28 23:09:00 machine_name rsyslogd: rsyslogd's groupid changed to 103 Jan 28 23:09:00 machine_name rsyslogd: rsyslogd's userid changed to 101 Jan 28 23:09:00 machine_name rsyslogd-2039: Could no open output file '/dev/xconsole' [try http://www.rsyslog.com/e/2039 ] When I hit the On/Off button, the system shuts down normally. May be it a hardware problem, but I don't know... Can you say something useful to solve my problem?

  Read the article

• Computer crashes on resume from standby almost every time

  - by Los Frijoles

  I am running Ubuntu 12.04 on a Core i5 2500K and ASRock Z68 Pro3-M motherboard (no graphics card, hd is a WD Green 1TB, and cd drive is some cheap lite-on drive). Since installing 12.04, my computer has been freezing after resume, but not every time. When I start to resume, it starts going normally with a blinking cursor on the screen and then sometimes it will continue on to the gnome 3 unlock screen. Most of the time, however, it will blink for a little bit and then the monitor will flip modes and shut off due to no signal. Pressing keys on the keyboard gets no response (num lock light doesn't respond, Ctrl-Alt-F1 fails to drop it into a terminal, Ctrl-Alt-Backspace doesn't work) and so I assume the computer is crashed. The worst part is, the logs look entirely normal. Here is my system log during one of these crashes and my subsequent hard poweroff and restart: Jun 6 21:54:43 kcuzner-desktop udevd[10448]: inotify_add_watch(6, /dev/dm-2, 10) failed: No such file or directory Jun 6 21:54:43 kcuzner-desktop udevd[10448]: inotify_add_watch(6, /dev/dm-2, 10) failed: No such file or directory Jun 6 21:54:43 kcuzner-desktop udevd[10448]: inotify_add_watch(6, /dev/dm-1, 10) failed: No such file or directory Jun 6 21:54:43 kcuzner-desktop udevd[12419]: inotify_add_watch(6, /dev/dm-0, 10) failed: No such file or directory Jun 6 21:54:43 kcuzner-desktop udevd[10448]: inotify_add_watch(6, /dev/dm-0, 10) failed: No such file or directory Jun 6 22:09:01 kcuzner-desktop CRON[9061]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth 1 -maxdepth 1 -type f -cmin +$(/usr/lib/php5/maxlifetime) ! -execdir fuser -s {} 2>/dev/null \; -delete) Jun 6 22:17:01 kcuzner-desktop CRON[22142]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Jun 6 22:39:01 kcuzner-desktop CRON[26909]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth 1 -maxdepth 1 -type f -cmin +$(/usr/lib/php5/maxlifetime) ! -execdir fuser -s {} 2>/dev/null \; -delete) Jun 6 22:54:21 kcuzner-desktop kernel: [57905.560822] show_signal_msg: 36 callbacks suppressed Jun 6 22:54:21 kcuzner-desktop kernel: [57905.560828] chromium-browse[9139]: segfault at 0 ip 00007f3a78efade0 sp 00007fff7e2d2c18 error 4 in chromium-browser[7f3a76604000+412b000] Jun 6 23:05:43 kcuzner-desktop kernel: [58586.415158] chromium-browse[21025]: segfault at 0 ip 00007f3a78efade0 sp 00007fff7e2d2c18 error 4 in chromium-browser[7f3a76604000+412b000] Jun 6 23:09:01 kcuzner-desktop CRON[13542]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth 1 -maxdepth 1 -type f -cmin +$(/usr/lib/php5/maxlifetime) ! -execdir fuser -s {} 2>/dev/null \; -delete) Jun 6 23:12:43 kcuzner-desktop kernel: [59006.317590] usb 2-1.7: USB disconnect, device number 8 Jun 6 23:12:43 kcuzner-desktop kernel: [59006.319672] sd 7:0:0:0: [sdg] Synchronizing SCSI cache Jun 6 23:12:43 kcuzner-desktop kernel: [59006.319737] sd 7:0:0:0: [sdg] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK Jun 6 23:17:01 kcuzner-desktop CRON[26580]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Jun 6 23:19:04 kcuzner-desktop acpid: client connected from 29925[0:0] Jun 6 23:19:04 kcuzner-desktop acpid: 1 client rule loaded Jun 6 23:19:07 kcuzner-desktop rtkit-daemon[1835]: Successfully made thread 30131 of process 30131 (n/a) owned by '104' high priority at nice level -11. Jun 6 23:19:07 kcuzner-desktop rtkit-daemon[1835]: Supervising 1 threads of 1 processes of 1 users. Jun 6 23:19:07 kcuzner-desktop rtkit-daemon[1835]: Successfully made thread 30162 of process 30131 (n/a) owned by '104' RT at priority 5. Jun 6 23:19:07 kcuzner-desktop rtkit-daemon[1835]: Supervising 2 threads of 1 processes of 1 users. Jun 6 23:19:07 kcuzner-desktop rtkit-daemon[1835]: Successfully made thread 30163 of process 30131 (n/a) owned by '104' RT at priority 5. Jun 6 23:19:07 kcuzner-desktop rtkit-daemon[1835]: Supervising 3 threads of 1 processes of 1 users. Jun 6 23:19:07 kcuzner-desktop bluetoothd[1140]: Endpoint registered: sender=:1.239 path=/MediaEndpoint/HFPAG Jun 6 23:19:07 kcuzner-desktop bluetoothd[1140]: Endpoint registered: sender=:1.239 path=/MediaEndpoint/A2DPSource Jun 6 23:19:07 kcuzner-desktop bluetoothd[1140]: Endpoint registered: sender=:1.239 path=/MediaEndpoint/A2DPSink Jun 6 23:19:07 kcuzner-desktop rtkit-daemon[1835]: Successfully made thread 30166 of process 30166 (n/a) owned by '104' high priority at nice level -11. Jun 6 23:19:07 kcuzner-desktop rtkit-daemon[1835]: Supervising 4 threads of 2 processes of 1 users. Jun 6 23:19:07 kcuzner-desktop pulseaudio[30166]: [pulseaudio] pid.c: Daemon already running. Jun 6 23:19:10 kcuzner-desktop acpid: client 2942[0:0] has disconnected Jun 6 23:19:10 kcuzner-desktop acpid: client 29925[0:0] has disconnected Jun 6 23:19:10 kcuzner-desktop acpid: client connected from 1286[0:0] Jun 6 23:19:10 kcuzner-desktop acpid: 1 client rule loaded Jun 6 23:19:31 kcuzner-desktop bluetoothd[1140]: Endpoint unregistered: sender=:1.239 path=/MediaEndpoint/HFPAG Jun 6 23:19:31 kcuzner-desktop bluetoothd[1140]: Endpoint unregistered: sender=:1.239 path=/MediaEndpoint/A2DPSource Jun 6 23:19:31 kcuzner-desktop bluetoothd[1140]: Endpoint unregistered: sender=:1.239 path=/MediaEndpoint/A2DPSink Jun 6 23:28:12 kcuzner-desktop kernel: imklog 5.8.6, log source = /proc/kmsg started. Jun 6 23:28:12 kcuzner-desktop rsyslogd: [origin software="rsyslogd" swVersion="5.8.6" x-pid="1053" x-info="http://www.rsyslog.com"] start Jun 6 23:28:12 kcuzner-desktop rsyslogd: rsyslogd's groupid changed to 103 Jun 6 23:28:12 kcuzner-desktop rsyslogd: rsyslogd's userid changed to 101 Jun 6 23:28:12 kcuzner-desktop rsyslogd-2039: Could not open output pipe '/dev/xconsole' [try http://www.rsyslog.com/e/2039 ] Jun 6 23:28:12 kcuzner-desktop modem-manager[1070]: <info> Loaded plugin Ericsson MBM Jun 6 23:28:12 kcuzner-desktop modem-manager[1070]: <info> Loaded plugin Sierra Jun 6 23:28:12 kcuzner-desktop modem-manager[1070]: <info> Loaded plugin Generic Jun 6 23:28:12 kcuzner-desktop modem-manager[1070]: <info> Loaded plugin Huawei Jun 6 23:28:12 kcuzner-desktop modem-manager[1070]: <info> Loaded plugin Linktop Jun 6 23:28:12 kcuzner-desktop bluetoothd[1072]: Failed to init gatt_example plugin Jun 6 23:28:12 kcuzner-desktop bluetoothd[1072]: Listening for HCI events on hci0 Jun 6 23:28:12 kcuzner-desktop NetworkManager[1080]: <info> NetworkManager (version 0.9.4.0) is starting... Jun 6 23:28:12 kcuzner-desktop NetworkManager[1080]: <info> Read config file /etc/NetworkManager/NetworkManager.conf Jun 6 23:28:12 kcuzner-desktop NetworkManager[1080]: <info> VPN: loaded org.freedesktop.NetworkManager.pptp Jun 6 23:28:12 kcuzner-desktop NetworkManager[1080]: <info> DNS: loaded plugin dnsmasq Jun 6 23:28:12 kcuzner-desktop kernel: [ 0.000000] Initializing cgroup subsys cpuset Jun 6 23:28:12 kcuzner-desktop kernel: [ 0.000000] Initializing cgroup subsys cpu Sorry it's so huge; the restart happens at 23:28:12 I believe and all I see is that chromium segfaulted a few times. I wouldn't think a segfault from an individual program on the computer would crash it, but could that be the issue?

  Read the article

• How to read oom-killer syslog messages?

  - by Grant

  I have a Ubuntu 12.04 server which sometimes dies completely - no SSH, no ping, nothing until it is physically rebooted. After the reboot, I see in syslog that the oom-killer killed, well, pretty much everything. There's a lot of detailed memory usage information in them. How do I read these logs to see what caused the OOM issue? The server has far more memory than it needs, so it shouldn't be running out of memory. Oct 25 07:28:04 nldedip4k031 kernel: [87946.529511] oom_kill_process: 9 callbacks suppressed Oct 25 07:28:04 nldedip4k031 kernel: [87946.529514] irqbalance invoked oom-killer: gfp_mask=0x80d0, order=0, oom_adj=0, oom_score_adj=0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529516] irqbalance cpuset=/ mems_allowed=0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529518] Pid: 948, comm: irqbalance Not tainted 3.2.0-55-generic-pae #85-Ubuntu Oct 25 07:28:04 nldedip4k031 kernel: [87946.529519] Call Trace: Oct 25 07:28:04 nldedip4k031 kernel: [87946.529525] [] dump_header.isra.6+0x85/0xc0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529528] [] oom_kill_process+0x5c/0x80 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529530] [] out_of_memory+0xc5/0x1c0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529532] [] __alloc_pages_nodemask+0x72c/0x740 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529535] [] __get_free_pages+0x1c/0x30 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529537] [] get_zeroed_page+0x12/0x20 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529541] [] fill_read_buffer.isra.8+0xaa/0xd0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529543] [] sysfs_read_file+0x7d/0x90 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529546] [] vfs_read+0x8c/0x160 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529548] [] ? fill_read_buffer.isra.8+0xd0/0xd0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529550] [] sys_read+0x3d/0x70 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529554] [] sysenter_do_call+0x12/0x28 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529555] Mem-Info: Oct 25 07:28:04 nldedip4k031 kernel: [87946.529556] DMA per-cpu: Oct 25 07:28:04 nldedip4k031 kernel: [87946.529557] CPU 0: hi: 0, btch: 1 usd: 0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529558] CPU 1: hi: 0, btch: 1 usd: 0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529560] CPU 2: hi: 0, btch: 1 usd: 0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529561] CPU 3: hi: 0, btch: 1 usd: 0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529562] CPU 4: hi: 0, btch: 1 usd: 0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529563] CPU 5: hi: 0, btch: 1 usd: 0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529564] CPU 6: hi: 0, btch: 1 usd: 0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529565] CPU 7: hi: 0, btch: 1 usd: 0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529566] Normal per-cpu: Oct 25 07:28:04 nldedip4k031 kernel: [87946.529567] CPU 0: hi: 186, btch: 31 usd: 179 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529568] CPU 1: hi: 186, btch: 31 usd: 182 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529569] CPU 2: hi: 186, btch: 31 usd: 132 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529570] CPU 3: hi: 186, btch: 31 usd: 175 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529571] CPU 4: hi: 186, btch: 31 usd: 91 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529572] CPU 5: hi: 186, btch: 31 usd: 173 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529573] CPU 6: hi: 186, btch: 31 usd: 159 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529574] CPU 7: hi: 186, btch: 31 usd: 164 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529575] HighMem per-cpu: Oct 25 07:28:04 nldedip4k031 kernel: [87946.529576] CPU 0: hi: 186, btch: 31 usd: 165 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529577] CPU 1: hi: 186, btch: 31 usd: 183 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529578] CPU 2: hi: 186, btch: 31 usd: 185 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529579] CPU 3: hi: 186, btch: 31 usd: 138 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529580] CPU 4: hi: 186, btch: 31 usd: 155 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529581] CPU 5: hi: 186, btch: 31 usd: 104 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529582] CPU 6: hi: 186, btch: 31 usd: 133 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529583] CPU 7: hi: 186, btch: 31 usd: 170 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529586] active_anon:5523 inactive_anon:354 isolated_anon:0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529586] active_file:2815 inactive_file:6849119 isolated_file:0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529587] unevictable:0 dirty:449 writeback:10 unstable:0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529587] free:1304125 slab_reclaimable:104672 slab_unreclaimable:3419 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529588] mapped:2661 shmem:138 pagetables:313 bounce:0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529591] DMA free:4252kB min:780kB low:972kB high:1168kB active_anon:0kB inactive_anon:0kB active_file:4kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15756kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:11564kB slab_unreclaimable:4kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1 all_unreclaimable? yes Oct 25 07:28:04 nldedip4k031 kernel: [87946.529594] lowmem_reserve[]: 0 869 32460 32460 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529599] Normal free:44052kB min:44216kB low:55268kB high:66324kB active_anon:0kB inactive_anon:0kB active_file:616kB inactive_file:568kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:890008kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:407124kB slab_unreclaimable:13672kB kernel_stack:992kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2083 all_unreclaimable? yes Oct 25 07:28:04 nldedip4k031 kernel: [87946.529602] lowmem_reserve[]: 0 0 252733 252733 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529606] HighMem free:5168196kB min:512kB low:402312kB high:804112kB active_anon:22092kB inactive_anon:1416kB active_file:10640kB inactive_file:27395920kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:32349872kB mlocked:0kB dirty:1796kB writeback:40kB mapped:10640kB shmem:552kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:1252kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Oct 25 07:28:04 nldedip4k031 kernel: [87946.529609] lowmem_reserve[]: 0 0 0 0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529611] DMA: 6*4kB 6*8kB 6*16kB 5*32kB 5*64kB 4*128kB 2*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 4232kB Oct 25 07:28:04 nldedip4k031 kernel: [87946.529616] Normal: 297*4kB 180*8kB 119*16kB 73*32kB 67*64kB 47*128kB 35*256kB 13*512kB 5*1024kB 1*2048kB 1*4096kB = 44052kB Oct 25 07:28:04 nldedip4k031 kernel: [87946.529622] HighMem: 1*4kB 6*8kB 27*16kB 11*32kB 2*64kB 1*128kB 0*256kB 0*512kB 4*1024kB 1*2048kB 1260*4096kB = 5168196kB Oct 25 07:28:04 nldedip4k031 kernel: [87946.529627] 6852076 total pagecache pages Oct 25 07:28:04 nldedip4k031 kernel: [87946.529628] 0 pages in swap cache Oct 25 07:28:04 nldedip4k031 kernel: [87946.529629] Swap cache stats: add 0, delete 0, find 0/0 Oct 25 07:28:04 nldedip4k031 kernel: [87946.529630] Free swap = 3998716kB Oct 25 07:28:04 nldedip4k031 kernel: [87946.529631] Total swap = 3998716kB Oct 25 07:28:04 nldedip4k031 kernel: [87946.571914] 8437743 pages RAM Oct 25 07:28:04 nldedip4k031 kernel: [87946.571916] 8209409 pages HighMem Oct 25 07:28:04 nldedip4k031 kernel: [87946.571917] 159556 pages reserved Oct 25 07:28:04 nldedip4k031 kernel: [87946.571917] 6862034 pages shared Oct 25 07:28:04 nldedip4k031 kernel: [87946.571918] 123540 pages non-shared Oct 25 07:28:04 nldedip4k031 kernel: [87946.571919] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Oct 25 07:28:04 nldedip4k031 kernel: [87946.571927] [ 421] 0 421 709 152 3 0 0 upstart-udev-br Oct 25 07:28:04 nldedip4k031 kernel: [87946.571929] [ 429] 0 429 773 326 5 -17 -1000 udevd Oct 25 07:28:04 nldedip4k031 kernel: [87946.571931] [ 567] 0 567 772 224 4 -17 -1000 udevd Oct 25 07:28:04 nldedip4k031 kernel: [87946.571932] [ 568] 0 568 772 231 7 -17 -1000 udevd Oct 25 07:28:04 nldedip4k031 kernel: [87946.571934] [ 764] 0 764 712 103 1 0 0 upstart-socket- Oct 25 07:28:04 nldedip4k031 kernel: [87946.571936] [ 772] 103 772 815 164 5 0 0 dbus-daemon Oct 25 07:28:04 nldedip4k031 kernel: [87946.571938] [ 785] 0 785 1671 600 1 -17 -1000 sshd Oct 25 07:28:04 nldedip4k031 kernel: [87946.571940] [ 809] 101 809 7766 380 1 0 0 rsyslogd Oct 25 07:28:04 nldedip4k031 kernel: [87946.571942] [ 869] 0 869 1158 213 3 0 0 getty Oct 25 07:28:04 nldedip4k031 kernel: [87946.571943] [ 873] 0 873 1158 214 6 0 0 getty Oct 25 07:28:04 nldedip4k031 kernel: [87946.571945] [ 911] 0 911 1158 215 3 0 0 getty Oct 25 07:28:04 nldedip4k031 kernel: [87946.571947] [ 912] 0 912 1158 214 2 0 0 getty Oct 25 07:28:04 nldedip4k031 kernel: [87946.571949] [ 914] 0 914 1158 213 1 0 0 getty Oct 25 07:28:04 nldedip4k031 kernel: [87946.571950] [ 916] 0 916 618 86 1 0 0 atd Oct 25 07:28:04 nldedip4k031 kernel: [87946.571952] [ 917] 0 917 655 226 3 0 0 cron Oct 25 07:28:04 nldedip4k031 kernel: [87946.571954] [ 948] 0 948 902 159 3 0 0 irqbalance Oct 25 07:28:04 nldedip4k031 kernel: [87946.571956] [ 993] 0 993 1145 363 3 0 0 master Oct 25 07:28:04 nldedip4k031 kernel: [87946.571957] [ 1002] 104 1002 1162 333 1 0 0 qmgr Oct 25 07:28:04 nldedip4k031 kernel: [87946.571959] [ 1016] 0 1016 730 149 2 0 0 mdadm Oct 25 07:28:04 nldedip4k031 kernel: [87946.571961] [ 1057] 0 1057 6066 2160 3 0 0 /usr/sbin/apach Oct 25 07:28:04 nldedip4k031 kernel: [87946.571963] [ 1086] 0 1086 1158 213 3 0 0 getty Oct 25 07:28:04 nldedip4k031 kernel: [87946.571965] [ 1088] 33 1088 6191 1517 0 0 0 /usr/sbin/apach Oct 25 07:28:04 nldedip4k031 kernel: [87946.571967] [ 1089] 33 1089 6191 1451 1 0 0 /usr/sbin/apach Oct 25 07:28:04 nldedip4k031 kernel: [87946.571969] [ 1090] 33 1090 6175 1451 3 0 0 /usr/sbin/apach Oct 25 07:28:04 nldedip4k031 kernel: [87946.571971] [ 1091] 33 1091 6191 1451 1 0 0 /usr/sbin/apach Oct 25 07:28:04 nldedip4k031 kernel: [87946.571972] [ 1092] 33 1092 6191 1451 0 0 0 /usr/sbin/apach Oct 25 07:28:04 nldedip4k031 kernel: [87946.571974] [ 1109] 33 1109 6191 1517 0 0 0 /usr/sbin/apach Oct 25 07:28:04 nldedip4k031 kernel: [87946.571976] [ 1151] 33 1151 6191 1451 1 0 0 /usr/sbin/apach Oct 25 07:28:04 nldedip4k031 kernel: [87946.571978] [ 1201] 104 1201 1803 652 1 0 0 tlsmgr Oct 25 07:28:04 nldedip4k031 kernel: [87946.571980] [ 2475] 0 2475 2435 812 0 0 0 sshd Oct 25 07:28:04 nldedip4k031 kernel: [87946.571982] [ 2494] 0 2494 1745 839 1 0 0 bash Oct 25 07:28:04 nldedip4k031 kernel: [87946.571984] [ 2573] 0 2573 3394 1689 0 0 0 sshd Oct 25 07:28:04 nldedip4k031 kernel: [87946.571986] [ 2589] 0 2589 5014 457 3 0 0 rsync Oct 25 07:28:04 nldedip4k031 kernel: [87946.571988] [ 2590] 0 2590 7970 522 1 0 0 rsync Oct 25 07:28:04 nldedip4k031 kernel: [87946.571990] [ 2652] 104 2652 1150 326 5 0 0 pickup Oct 25 07:28:04 nldedip4k031 kernel: [87946.571992] Out of memory: Kill process 421 (upstart-udev-br) score 1 or sacrifice child Oct 25 07:28:04 nldedip4k031 kernel: [87946.572407] Killed process 421 (upstart-udev-br) total-vm:2836kB, anon-rss:156kB, file-rss:452kB Oct 25 07:28:04 nldedip4k031 kernel: [87946.573107] init: upstart-udev-bridge main process (421) killed by KILL signal Oct 25 07:28:04 nldedip4k031 kernel: [87946.573126] init: upstart-udev-bridge main process ended, respawning Oct 25 07:28:34 nldedip4k031 kernel: [87976.461570] irqbalance invoked oom-killer: gfp_mask=0x80d0, order=0, oom_adj=0, oom_score_adj=0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461573] irqbalance cpuset=/ mems_allowed=0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461576] Pid: 948, comm: irqbalance Not tainted 3.2.0-55-generic-pae #85-Ubuntu Oct 25 07:28:34 nldedip4k031 kernel: [87976.461578] Call Trace: Oct 25 07:28:34 nldedip4k031 kernel: [87976.461585] [] dump_header.isra.6+0x85/0xc0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461588] [] oom_kill_process+0x5c/0x80 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461591] [] out_of_memory+0xc5/0x1c0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461595] [] __alloc_pages_nodemask+0x72c/0x740 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461599] [] __get_free_pages+0x1c/0x30 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461602] [] get_zeroed_page+0x12/0x20 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461606] [] fill_read_buffer.isra.8+0xaa/0xd0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461609] [] sysfs_read_file+0x7d/0x90 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461613] [] vfs_read+0x8c/0x160 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461616] [] ? fill_read_buffer.isra.8+0xd0/0xd0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461619] [] sys_read+0x3d/0x70 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461624] [] sysenter_do_call+0x12/0x28 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461626] Mem-Info: Oct 25 07:28:34 nldedip4k031 kernel: [87976.461628] DMA per-cpu: Oct 25 07:28:34 nldedip4k031 kernel: [87976.461629] CPU 0: hi: 0, btch: 1 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461631] CPU 1: hi: 0, btch: 1 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461633] CPU 2: hi: 0, btch: 1 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461634] CPU 3: hi: 0, btch: 1 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461636] CPU 4: hi: 0, btch: 1 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461638] CPU 5: hi: 0, btch: 1 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461639] CPU 6: hi: 0, btch: 1 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461641] CPU 7: hi: 0, btch: 1 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461642] Normal per-cpu: Oct 25 07:28:34 nldedip4k031 kernel: [87976.461644] CPU 0: hi: 186, btch: 31 usd: 61 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461646] CPU 1: hi: 186, btch: 31 usd: 49 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461647] CPU 2: hi: 186, btch: 31 usd: 8 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461649] CPU 3: hi: 186, btch: 31 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461651] CPU 4: hi: 186, btch: 31 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461652] CPU 5: hi: 186, btch: 31 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461654] CPU 6: hi: 186, btch: 31 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461656] CPU 7: hi: 186, btch: 31 usd: 30 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461657] HighMem per-cpu: Oct 25 07:28:34 nldedip4k031 kernel: [87976.461658] CPU 0: hi: 186, btch: 31 usd: 4 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461660] CPU 1: hi: 186, btch: 31 usd: 204 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461662] CPU 2: hi: 186, btch: 31 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461663] CPU 3: hi: 186, btch: 31 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461665] CPU 4: hi: 186, btch: 31 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461667] CPU 5: hi: 186, btch: 31 usd: 31 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461668] CPU 6: hi: 186, btch: 31 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461670] CPU 7: hi: 186, btch: 31 usd: 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461674] active_anon:5441 inactive_anon:412 isolated_anon:0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461674] active_file:2668 inactive_file:6922842 isolated_file:0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461675] unevictable:0 dirty:836 writeback:0 unstable:0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461676] free:1231664 slab_reclaimable:105781 slab_unreclaimable:3399 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461677] mapped:2649 shmem:138 pagetables:313 bounce:0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461682] DMA free:4248kB min:780kB low:972kB high:1168kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15756kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:11560kB slab_unreclaimable:4kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:5687 all_unreclaimable? yes Oct 25 07:28:34 nldedip4k031 kernel: [87976.461686] lowmem_reserve[]: 0 869 32460 32460 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461693] Normal free:44184kB min:44216kB low:55268kB high:66324kB active_anon:0kB inactive_anon:0kB active_file:20kB inactive_file:1096kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:890008kB mlocked:0kB dirty:4kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:411564kB slab_unreclaimable:13592kB kernel_stack:992kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1816 all_unreclaimable? yes Oct 25 07:28:34 nldedip4k031 kernel: [87976.461697] lowmem_reserve[]: 0 0 252733 252733 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461703] HighMem free:4878224kB min:512kB low:402312kB high:804112kB active_anon:21764kB inactive_anon:1648kB active_file:10652kB inactive_file:27690268kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:32349872kB mlocked:0kB dirty:3340kB writeback:0kB mapped:10592kB shmem:552kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:1252kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Oct 25 07:28:34 nldedip4k031 kernel: [87976.461708] lowmem_reserve[]: 0 0 0 0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461711] DMA: 8*4kB 7*8kB 6*16kB 5*32kB 5*64kB 4*128kB 2*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 4248kB Oct 25 07:28:34 nldedip4k031 kernel: [87976.461719] Normal: 272*4kB 178*8kB 76*16kB 52*32kB 42*64kB 36*128kB 23*256kB 20*512kB 7*1024kB 2*2048kB 1*4096kB = 44176kB Oct 25 07:28:34 nldedip4k031 kernel: [87976.461727] HighMem: 1*4kB 45*8kB 31*16kB 24*32kB 5*64kB 3*128kB 1*256kB 2*512kB 4*1024kB 2*2048kB 1188*4096kB = 4877852kB Oct 25 07:28:34 nldedip4k031 kernel: [87976.461736] 6925679 total pagecache pages Oct 25 07:28:34 nldedip4k031 kernel: [87976.461737] 0 pages in swap cache Oct 25 07:28:34 nldedip4k031 kernel: [87976.461739] Swap cache stats: add 0, delete 0, find 0/0 Oct 25 07:28:34 nldedip4k031 kernel: [87976.461740] Free swap = 3998716kB Oct 25 07:28:34 nldedip4k031 kernel: [87976.461741] Total swap = 3998716kB Oct 25 07:28:34 nldedip4k031 kernel: [87976.524951] 8437743 pages RAM Oct 25 07:28:34 nldedip4k031 kernel: [87976.524953] 8209409 pages HighMem Oct 25 07:28:34 nldedip4k031 kernel: [87976.524954] 159556 pages reserved Oct 25 07:28:34 nldedip4k031 kernel: [87976.524955] 6936141 pages shared Oct 25 07:28:34 nldedip4k031 kernel: [87976.524956] 124602 pages non-shared Oct 25 07:28:34 nldedip4k031 kernel: [87976.524957] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Oct 25 07:28:34 nldedip4k031 kernel: [87976.524966] [ 429] 0 429 773 326 5 -17 -1000 udevd Oct 25 07:28:34 nldedip4k031 kernel: [87976.524968] [ 567] 0 567 772 224 4 -17 -1000 udevd Oct 25 07:28:34 nldedip4k031 kernel: [87976.524971] [ 568] 0 568 772 231 7 -17 -1000 udevd Oct 25 07:28:34 nldedip4k031 kernel: [87976.524973] [ 764] 0 764 712 103 3 0 0 upstart-socket- Oct 25 07:28:34 nldedip4k031 kernel: [87976.524976] [ 772] 103 772 815 164 2 0 0 dbus-daemon Oct 25 07:28:34 nldedip4k031 kernel: [87976.524979] [ 785] 0 785 1671 600 1 -17 -1000 sshd Oct 25 07:28:34 nldedip4k031 kernel: [87976.524981] [ 809] 101 809 7766 380 1 0 0 rsyslogd Oct 25 07:28:34 nldedip4k031 kernel: [87976.524983] [ 869] 0 869 1158 213 3 0 0 getty Oct 25 07:28:34 nldedip4k031 kernel: [87976.524986] [ 873] 0 873 1158 214 6 0 0 getty Oct 25 07:28:34 nldedip4k031 kernel: [87976.524988] [ 911] 0 911 1158 215 3 0 0 getty Oct 25 07:28:34 nldedip4k031 kernel: [87976.524990] [ 912] 0 912 1158 214 2 0 0 getty Oct 25 07:28:34 nldedip4k031 kernel: [87976.524992] [ 914] 0 914 1158 213 1 0 0 getty Oct 25 07:28:34 nldedip4k031 kernel: [87976.524995] [ 916] 0 916 618 86 1 0 0 atd Oct 25 07:28:34 nldedip4k031 kernel: [87976.524997] [ 917] 0 917 655 226 3 0 0 cron Oct 25 07:28:34 nldedip4k031 kernel: [87976.524999] [ 948] 0 948 902 159 5 0 0 irqbalance Oct 25 07:28:34 nldedip4k031 kernel: [87976.525002] [ 993] 0 993 1145 363 3 0 0 master Oct 25 07:28:34 nldedip4k031 kernel: [87976.525004] [ 1002] 104 1002 1162 333 1 0 0 qmgr Oct 25 07:28:34 nldedip4k031 kernel: [87976.525007] [ 1016] 0 1016 730 149 2 0 0 mdadm Oct 25 07:28:34 nldedip4k031 kernel: [87976.525009] [ 1057] 0 1057 6066 2160 3 0 0 /usr/sbin/apach Oct 25 07:28:34 nldedip4k031 kernel: [87976.525012] [ 1086] 0 1086 1158 213 3 0 0 getty Oct 25 07:28:34 nldedip4k031 kernel: [87976.525014] [ 1088] 33 1088 6191 1517 0 0 0 /usr/sbin/apach Oct 25 07:28:34 nldedip4k031 kernel: [87976.525017] [ 1089] 33 1089 6191 1451 1 0 0 /usr/sbin/apach Oct 25 07:28:34 nldedip4k031 kernel: [87976.525019] [ 1090] 33 1090 6175 1451 1 0 0 /usr/sbin/apach Oct 25 07:28:34 nldedip4k031 kernel: [87976.525021] [ 1091] 33 1091 6191 1451 1 0 0 /usr/sbin/apach Oct 25 07:28:34 nldedip4k031 kernel: [87976.525024] [ 1092] 33 1092 6191 1451 0 0 0 /usr/sbin/apach Oct 25 07:28:34 nldedip4k031 kernel: [87976.525026] [ 1109] 33 1109 6191 1517 0 0 0 /usr/sbin/apach Oct 25 07:28:34 nldedip4k031 kernel: [87976.525029] [ 1151] 33 1151 6191 1451 1 0 0 /usr/sbin/apach Oct 25 07:28:34 nldedip4k031 kernel: [87976.525031] [ 1201] 104 1201 1803 652 1 0 0 tlsmgr Oct 25 07:28:34 nldedip4k031 kernel: [87976.525033] [ 2475] 0 2475 2435 812 0 0 0 sshd Oct 25 07:28:34 nldedip4k031 kernel: [87976.525036] [ 2494] 0 2494 1745 839 1 0 0 bash Oct 25 07:28:34 nldedip4k031 kernel: [87976.525038] [ 2573] 0 2573 3394 1689 3 0 0 sshd Oct 25 07:28:34 nldedip4k031 kernel: [87976.525040] [ 2589] 0 2589 5014 457 3 0 0 rsync Oct 25 07:28:34 nldedip4k031 kernel: [87976.525043] [ 2590] 0 2590 7970 522 1 0 0 rsync Oct 25 07:28:34 nldedip4k031 kernel: [87976.525045] [ 2652] 104 2652 1150 326 5 0 0 pickup Oct 25 07:28:34 nldedip4k031 kernel: [87976.525048] [ 2847] 0 2847 709 89 0 0 0 upstart-udev-br Oct 25 07:28:34 nldedip4k031 kernel: [87976.525050] Out of memory: Kill process 764 (upstart-socket-) score 1 or sacrifice child Oct 25 07:28:34 nldedip4k031 kernel: [87976.525484] Killed process 764 (upstart-socket-) total-vm:2848kB, anon-rss:204kB, file-rss:208kB Oct 25 07:28:34 nldedip4k031 kernel: [87976.526161] init: upstart-socket-bridge main process (764) killed by KILL signal Oct 25 07:28:34 nldedip4k031 kernel: [87976.526180] init: upstart-socket-bridge main process ended, respawning Oct 25 07:28:44 nldedip4k031 kernel: [87986.439671] irqbalance invoked oom-killer: gfp_mask=0x80d0, order=0, oom_adj=0, oom_score_adj=0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439674] irqbalance cpuset=/ mems_allowed=0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439676] Pid: 948, comm: irqbalance Not tainted 3.2.0-55-generic-pae #85-Ubuntu Oct 25 07:28:44 nldedip4k031 kernel: [87986.439678] Call Trace: Oct 25 07:28:44 nldedip4k031 kernel: [87986.439684] [] dump_header.isra.6+0x85/0xc0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439686] [] oom_kill_process+0x5c/0x80 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439688] [] out_of_memory+0xc5/0x1c0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439691] [] __alloc_pages_nodemask+0x72c/0x740 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439694] [] __get_free_pages+0x1c/0x30 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439696] [] get_zeroed_page+0x12/0x20 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439699] [] fill_read_buffer.isra.8+0xaa/0xd0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439702] [] sysfs_read_file+0x7d/0x90 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439704] [] vfs_read+0x8c/0x160 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439707] [] ? fill_read_buffer.isra.8+0xd0/0xd0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439709] [] sys_read+0x3d/0x70 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439712] [] sysenter_do_call+0x12/0x28 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439714] Mem-Info: Oct 25 07:28:44 nldedip4k031 kernel: [87986.439714] DMA per-cpu: Oct 25 07:28:44 nldedip4k031 kernel: [87986.439716] CPU 0: hi: 0, btch: 1 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439717] CPU 1: hi: 0, btch: 1 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439718] CPU 2: hi: 0, btch: 1 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439719] CPU 3: hi: 0, btch: 1 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439720] CPU 4: hi: 0, btch: 1 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439721] CPU 5: hi: 0, btch: 1 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439722] CPU 6: hi: 0, btch: 1 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439723] CPU 7: hi: 0, btch: 1 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439724] Normal per-cpu: Oct 25 07:28:44 nldedip4k031 kernel: [87986.439725] CPU 0: hi: 186, btch: 31 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439726] CPU 1: hi: 186, btch: 31 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439727] CPU 2: hi: 186, btch: 31 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439728] CPU 3: hi: 186, btch: 31 usd: 0 Oct 25 07:28:44 nldedip4k031 kernel: [87986.439729] CPU 4: hi: 186, btch: 31 usd: 0 Oct 25 07:33:48 nldedip4k031 kernel: imklog 5.8.6, log source = /proc/kmsg started. Oct 25 07:33:48 nldedip4k031 rsyslogd: [origin software="rsyslogd" swVersion="5.8.6" x-pid="2880" x-info="http://www.rsyslog.com"] start Oct 25 07:33:48 nldedip4k031 rsyslogd: rsyslogd's groupid changed to 103 Oct 25 07:33:48 nldedip4k031 rsyslogd: rsyslogd's userid changed to 101 Oct 25 07:33:48 nldedip4k031 rsyslogd-2039: Could not open output pipe '/dev/xconsole' [try http://www.rsyslog.com/e/2039 ]

  Read the article

• How to keep haproxy log messages out of /var/log/syslog

  - by itsadok

  I set up haproxy logging via rsyslogd using the tips from this article, and everything seems to be working fine. The log files get the log messages. However, every log message from haproxy also shows up at /var/log/syslog. This means that once the server goes live, the syslog will be quite useless, as it will be run over with haproxy log messages. I would like to filter out those messages from /var/log/syslog. After going over the rsyslogd documentation, I tried to change the file /etc/rsyslog.d/50-default.conf thus: *.*;auth,authpriv.none;haproxy.none -/var/log/syslog I simply added the ;haproxy.nonepart. After restarting rsyslogd it stopped working completely until I reverted my changes. What am I doing wrong?

  Read the article

• how to troubleshoot using rsyslog to output to a mysql database

  - by ChrisNZ

  Using FreeBSD 8.0 32 bit. I have installed rsyslogd 5.5.5 with ommysql. (installed ports /usr/ports/sysutils/rsyslog55 and /usr/ports/sysutils/rsyslog55-mysql) My rsyslog.conf file looks like: $ModLoad imudp $ModLoad imtcp $ModLoad ommysql $ModLoad immark.so $ModLoad imuxsock.so $ModLoad imklog.so $OptimizeForUniprocessor on $AllowedSender UDP, 10.0.0.0/8 $UDPServerAddress 0.0.0.0 $UDPServerRun 514 $UDPServerTimeRequery 2 # +SG560 *.* :ommysql:127.0.0.1,Syslog,sysloguser,mypassword My command line flags for rsyslogd are: -c5 -4 Checking the code with -c5 -N1 returns no errors. I have confirmed that rsyslogd is working by changing the last line to say: *.* /var/log/snapgear.log which results in messages appearing in the snapgear.log file. So it is probably something to do with my MySQL setup If I do: mysql -u sysloguser -p Syslog Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 56 Server version: 5.0.86 FreeBSD port: mysql-server-5.0.86 mysql> select * from SystemEvents; Empty set (0.00 sec) mysql> :-( I have confirmed that sysloguser has full privileges for the Syslog database. If I run rsyslogd on the console in debug mode: /usr/local/sbin/rsyslogd -f /usr/local/etc/rsyslog.conf -c5 -n -d I can see this sequence of events each time a message is received: 9244.376687256:28359280: main Q: entry added, size now log 1, phys 1 entries 9244.376705694:28359280: main Q: EnqueueMsg advised worker start 9244.376726647:28359280: Listening on UDP syslogd socket 4 (IPv4/port 514). 9244.376728602:28359280: --------imUDP calling select, active file descriptors (max 4): 4 9244.376890075:283593c0: wti 0x28306e80: worker awoke from idle processing 9244.376892031:283593c0: we deleted 0 objects and enqueued 0 objects 9244.376893986:283593c0: delete batch from store, new sizes: log 1, phys 1 9244.376895942:283593c0: msgConsumer processes msg 0/1 9244.376897898:283593c0: msg parser: flags 70, from '~NOTRESOLVED~', msg 'Jun 29 17:32:24 SG560 kernel: (20000629T1732244' 9244.376900132:283593c0: parse using parser list 0x283080e8 (the default list). 9244.376902088:283593c0: dropped LF at very end of message (DropTrailingLF is set) 9244.376904044:283593c0: Parser 'rsyslog.rfc5424' returned -2160 9244.376905999:283593c0: Message will now be parsed by the legacy syslog parser (one size fits all... ;)). 9244.376907955:283593c0: Parser 'rsyslog.rfc3164' returned 0 9244.376909910:283593c0: testing filter, f_pmask 255 9244.376911866:283593c0: Called action, logging to ommysql 9244.376918012:283593c0: actionTryResume: action state: susp, next retry (if applicable): 1277869250 [now 1277869244] 9244.376919967:283593c0: action call returned -2123 9244.376921923:283593c0: tryDoAction: unexpected error code -2123, finalizing 9244.376926113:283593c0: actionTryResume: action state: susp, next retry (if applicable): 1277869250 [now 1277869244] 9244.376928069:283593c0: ruleset: get iRet 0 from rule.ProcessMsg() 9244.376930024:283593c0: ruleset.ProcessMsg() returns 0 9244.376931980:283593c0: regular consumer finished, iret=0, szlog 0 sz phys 1 9244.376933936:283593c0: XXX: enqueueing data element 0 of 1 9244.376935891:283593c0: we deleted 1 objects and enqueued 0 objects 9244.376938126:283593c0: delete batch from store, new sizes: log 0, phys 0 9244.376940082:283593c0: regular consumer finished, iret=4, szlog 0 sz phys 0 9244.376942037:283593c0: main Q:Reg/w0: worker IDLE, waiting for work. .... I can see the Action Call to ommysql returns unexpected error code -2123 Now I am stuck! Any ideas on what to look for next? Perhaps I there are extra ports I need to install? I will be very grateful for any assistance here!

  Read the article

1 2 3  | Next Page >