This is part 3 of a three-part blog series that summarizes the most commonly implemented configuration changes to improve performance and operation of a large Enterprise Manager 12c environment. A “large” environment is categorized by the number of agents, targets and users. See the Oracle Enterprise Manager Cloud Control Advanced Installation and Configuration Guide chapter on Sizing for more details on sizing your environment properly.

• Part 1 of this series covered recommended configuration changes for the OMS and Repository
• Part 2 covered recommended changes for the Weblogic server
• Part 3 covers general configuration recommendations and a few known issues

The entire series can be found in the My Oracle Support note titled Oracle Enterprise Manager 12c Configuration Best Practices [1553342.1].

Configuration Recommendations

Configure E-Mail Notifications for EM related Alerts

In some environments, the notifications for events for different target types may be sent to different support teams (i.e. notifications on host targets may be sent to a platform support team). However, the EM application administrators should be well informed of any alerts or problems seen on the EM infrastructure components.

Recommendation: Create a new Incident rule for monitoring all EM components and setup the notifications to be sent to the EM administrator(s). The notification methods available can create or update an incident, send an email or forward to an event connector. To setup the incident rule set follow the steps below. Note that each individual rule in the rule set can have different actions configured.

1.  To create an incident rule for monitoring the EM components, click on Setup / Incidents / Incident Rules. On the All Enterprise Rules page, click on the out-of-box rule called “Incident management Ruleset for all targets” and then click on the Actions drop down list and select “Create Like Rule Set…”

2. For the rule set name, enter a name such as MTM Ruleset. Under the Targets tab, select “All targets of types” and select “OMS and Repository” from the drop down list. This target type contains all of the key EM components (OMS servers, repository, domains, etc.)

3. Click on the Rules tab. To edit a rule, click on the rule name and click on Edit as seen below

4. Modify the following rules:

a. Incident creation Rule for metric alerts

i. Leave the Type set as is but change the Severity to add Warning by clicking on the drop down list and selecting “Warning”. Click Next.

ii.  Add or modify the actions as required (i.e. add email notifications). Click Continue and then click Next.

iii. Leave the Name and description the same and click Next.

iv. Click Continue on the Review page.

b. Incident creation Rule for target unreachable.

i.   Leave the Type set as is but change the Target type to add OMS and Repository by clicking on the drop down list selecting “OMS and Repository”. Click Next.

ii.  Add or modify the actions as required (i.e. add email notifications) Click Continue and then click Next.

iii. Leave the Name and description the same and click Next.

iv. Click Continue on the Review page.

5.  Modify the actions for any other rule as required and be sure to click the “Save” push button to save the rule set or all changes will be lost.

Configure Out-of-Band Notifications for EM Agent

Out-of-Band notifications act as a backup when there’s a complete EM outage or a repository database issue. This is configured on the agent of the OMS server and can be used to send emails or execute another script that would create a trouble ticket. It will send notifications about the following issues:

• Repository Database down

• All OMS are down

• Repository side collection job that is broken or has an invalid schedule

• Notification job that is broken or has an invalid schedule

Recommendation: To setup Out-of-Band Notifications, refer to the MOS note “How To Setup Out Of Bound Email Notification In 12c” (Doc ID 1472854.1)

Modify the Performance Test for the EM Console Service

The EM Console Service has an out-of-box defined performance test that will be run to determine the status of this service. The test issues a request via an HTTP method to a specific URL. By default, the HTTP method used for this test is a GET but for performance reasons, should be changed to HEAD. The URL used for this request is set to point to a specific OMS server by default. If a multi-OMS system has been implemented and the OMS servers are behind a load balancer, then the URL in this section must be modified to point to the load balancer name instead of a specific server name. If this is not done and a portion of the infrastructure is down then the EM Console Service will show down as this test will fail.

Recommendation: Modify the HTTP Method for the EM Console Service test and the URL if required following the detailed steps below.

1.  To create an incident rule for monitoring the EM components, click on Targets / Services. From the list of services, click on the EM Console Service.

2. On the EM Console Service page, click on the Test Performance tab.

3.  At the bottom of the page, click on the Web Transaction test called EM Console Service Test

4.  Click on the Service Tests and Beacons breadcrumb near the top of the page.

5.  Under the Service Tests section, make sure the EM Console Service Test is selected and click on the Edit push button.

6.  Under the Transaction section, make sure the Access Logout page transaction is selected and click on the Edit push button

7) Under the Request section, change the HTTP Method from the default of GET to the recommended value of HEAD. The URL in this section must be modified to point to the load balancer name instead of a specific server name if multi-OMSes have been implemented.

Check for Known Issues

Job Purge Repository Job is Shown as Down

This issue is caused after upgrading EM from 12c to 12cR2. On the Repository page under Setup ? Manage Cloud Control ? Repository, the job called “Job Purge” is shown as down and the Next Scheduled Run is blank. Also, repvfy reports that this is a missing DBMS_SCHEDULER job.

Recommendation: In EM 12cR2, the apply_purge_policies have been moved from the MGMT_JOB_ENGINE package to the EM_JOB_PURGE package. To remove this error, execute the commands below:

$ repvfy verify core -test 2 -fix

To confirm that the issue resolved, execute

$ repvfy verify core -test 2

It can also be verified by refreshing the Job Service page in EM and check the status of the job, it should now be Up.

Configure the Listener Targets in EM with the Listener Password (where required)

EM will report this error every time it is encountered in the listener log file. In a RAC environment, typically the grid home and rdbms homes are owned by different OS users. The listener always runs from the grid home. Only the listener process owner can query or change the listener properties. The listener uses a password to allow other OS users (ex. the agent user) to query the listener process for parameters. EM has a default listener target metric that will query these properties. If the agent is not permitted to do this, the TNS incident (TNS-1190) will be logged in the listener’s log file. This means that the listener targets in EM also need to have this password set. Not doing so will cause many TNS incidents (TNS-1190). Below is a sample of this error from the listener log file:

Recommendation: Set a listener password and include it in the configuration of the listener targets in EM

For steps on setting the listener passwords, see MOS notes: 260986.1 , 427422.1