AIX Grid Control 10.2.0.5 Communication and Monitoring Issue since 31-DEC-2010
Posted
by jayatheertha.rao(at)oracle.com
on Oracle Blogs
See other posts from Oracle Blogs
or by jayatheertha.rao(at)oracle.com
Published on Thu, 30 Dec 2010 02:13:46 -0700
Indexed on
2010/12/30
11:57 UTC
Read the original article
Hit count: 1192
Master Notes
|10.2.0.5_aix_agent
|10.2.0.5_aix_grid_control
|10.2.0.5_grid_control_aix
|aix
|Javax.net.ssl.SSLPeerUnverifiedException
|Javax.net.ssl.SSLPeerUnverifiedException_gri_control_aix
|Javax.net.ssl.SSLPeerUnverifiedException_grid_control
|aix_10.2.0.5_grid_Control
|aix_grid_control
|grid_control
|peer_not_authenticated
Detailed symptoms for Oracle Management Server (OMS) 10.2.0.5 on AIX
- Oracle Management Service 10.2.0.5 instances on AIX 5L remain active and functional, but the OMS instances fail to communicate with the Grid Control Management Agents.
An SSLPeerUnverified exception will be reported in the file $ORACLE_HOME/sysman/log/emoms.trc when OMS attempts to connect with an Agent:
Javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificateChain(DashoA12275)
at oracle.sysman.emSDK.emd.comm.EMDClient.authenticateHTTPConnection(EMDClient.java:2002)
at oracle.sysman.emSDK.emd.comm.EMDClient.getConnection(EMDClient.java:1877)
at oracle.sysman.emSDK.emd.comm.EMDClient.getConnection(EMDClient.java:1810)
at oracle.sysman.emSDK.emd.comm.EMDClient.verifyHttpConnection(EMDClient.java:2540)
at oracle.sysman.emSDK.emd.comm.EMDClient.getResponseForRequest(EMDClient.java:2323)
at oracle.sysman.emSDK.emd.comm.EMDClient.getUploadManagerStatus(EMDClient.java:4853)
at oracle.sysman.eml.admin.rep.emdConfig.EmdConfigTargetsData.getEmdUploadData
(EmdConfigTargetsData.java:1640)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
This error may be reported when:
- Accessing the Agent home page in Grid Control
- Setting preferred credentials for a target monitored by the Agent
- Managing metrics for a target monitored by the Agent - The jobs scheduled to be run by Agents can become non-responsive
The OMS log file $ORACLE_HOME/sysman/log/emoms.trc can show:
2010-12-31 00:06:58,204 [JobWorker 430:Thread-34] DEBUG emSDK.comm getStreamResponse.4015 - oracle.sysman.em
SDK.emd.comm.CommException: java.io.IOException: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
oracle.sysman.emSDK.emd.comm.CommException: java.io.IOException: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at oracle.sysman.emSDK.emd.comm.EMDClient.getStreamResponse_(EMDClient.java:4088)
at oracle.sysman.emSDK.emd.comm.EMDClient.getStreamResponse(EMDClient.java:4009)
at oracle.sysman.emSDK.emd.comm.EMDClient.remoteOperation(EMDClient.java:3404)
at oracle.sysman.emdrep.jobs.CommandManager.requestRemoteCommand(CommandManager.java:765)
at oracle.sysman.emdrep.jobs.commands.RemoteOp.executeCommand(RemoteOp.java:434)
at oracle.sysman.emdrep.jobs.commands.RemoteOp.executeCommand(RemoteOp.java:491)
at oracle.sysman.emdrep.jobs.BaseJobWorker.runStep(BaseJobWorker.java:614)
at oracle.sysman.emdrep.jobs.BaseJobWorker.doOneOperation(BaseJobWorker.java:738)
at oracle.sysman.emdrep.jobs.JobWorker.doOneOperation(JobWorker.java:306)
at oracle.sysman.emdrep.jobs.JobWorker.run(JobWorker.java:288)
at oracle.sysman.util.threadPoolManager.WorkerThread.run(Worker.java:261)
Detailed symptoms for Grid Control Management Agent 10.2.0.5 on AIX
- Beginning 31-DEC-2010 00:00:00, 10.2.0.5 Management Agents running on the AIX 5L operating system will fail to monitor Oracle Application Server targets. As a result, the Availability Status for the Oracle Application Server targets will be in the "Metric Error" state.
NOTE: The 10.2.0.5.0 Agents would experience these errors regardless of the version/platform of the OMS.
The following metric error is seen in the console for the Oracle Application Server targets monitored by a Grid Control Management Agent 10.2.0.5 installed on AIX and experiencing a Root Certificate Authority issue:
Message oracle.sysman.emSDK.emd.fetchlet.FetchletException: oracle.sysman.emSDK.emd.fetchlet.FetchletException: oracle.sysman.emSDK.emd.fetchlet.FetchletException: oracle.sysman.emSDK.emd.comm.CommException: java.io.IOException: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
The Grid Control Management Agent log file $ORACLE_HOME/sysman/log/emagentfetchlet.log (or $ORACLE_HOME/hostname/sysman/log/emagentfetchlet.log for a clustered Agent) includes the following errors:
2010-12-31 00:01:03,626 [nmefmgr_getJNIFetchlet] ERROR ias.ResponseMetric getResponseMetric.154 - Unable to
compute application server status
oracle.sysman.emSDK.emd.fetchlet.FetchletException: oracle.sysman.emSDK.emd.comm.CommException: java.io.IOEx
ception: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at oracle.sysman.ias.ias.ResponseMetric.getResponseMetric(ResponseMetric.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at oracle.sysman.emd.fetchlets.JavaWrapperFetchlet.getMetric(JavaWrapperFetchlet.java:217)
at oracle.sysman.emd.fetchlets.FetchletWrapper.getMetric(FetchletWrapper.java:382) - Beginning 31-DEC-2010, 10.2.0.5 Management Agents on the AIX 5L platform will fail to secure or re-secure with Oracle Management Service (OMS). This failure will cause installation of 10.2.0.5 Agents on the AIX 5L platform to fail.
NOTE: The 10.2.0.5.0 Agents would experience these errors regardless of the version/platform of the OMS.
The "emctl secure agent" command will fail with the following error, which will be written to the $ORACLE_HOME/sysman/log/secure.log file (or $ORACLE_HOME/hostname/sysman/log/secure.log for a clustered Agent) :
2011-01-03 21:06:11,941 [main] ERROR agent.SecureAgentCmd main.207 - Failed
to secure the Agent:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at
com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificateChain(DashoA6275)
at
oracle.sysman.emctl.secure.agent.SecureAgentCmd.checkUpload(SecureAgentCmd.java:478)
at
oracle.sysman.emctl.secure.agent.SecureAgentCmd.secureAgent(SecureAgentCmd.java:249)
at
oracle.sysman.emctl.secure.agent.SecureAgentCmd.main(SecureAgentCmd.java:200)
© Oracle Blogs or respective owner