Wednesday, February 2, 2011

Create userdefined reports in Grid Control

You can define your own reports in Grid Control.

Login as sys on the Grid Repository and execute the following:

SQL> insert into sysman.mgmt_ip_report_elem_def values('target_user_table_from_sql','database_target_type','target_user_table_from_sql',
'oracle.sysman.db.util.reports.TableTargetRenderController',0,NULL,'EM',NULL,
'10.2.0.1.0');


SQL> insert into SYSMAN.MGMT_IP_ELEM_PARAM_CLASSES
(ELEMENT_NAME_NLSID, ELEMENT_TYPE_NLSID, ELEMENT_PARAM_CLASS, DISPLAY_ORDER)
values ('target_user_table_from_sql','database_target_type'

,'oracle.sysman.eml.ip.render.elem.HeaderParamController',1);

SQL> insert into SYSMAN.MGMT_IP_ELEM_PARAM_CLASSES
(ELEMENT_NAME_NLSID, ELEMENT_TYPE_NLSID, ELEMENT_PARAM_CLASS, DISPLAY_ORDER)
values ('target_user_table_from_sql', 'database_target_type', 'oracle.sysman.eml.ip.render.elem.SQLStatementParamController', 2);

sql> Commit;

Once completed, you will get a new report element called "target_user_table_from _sql" on the report page and you will be able to create reports that query target databases.

Reset the Grid Control agent

If you would like to reset the agent and clear all the files do the following :

$AGENT_HOME/bin/emctl stop agent
$rm $AGENT_HOME/sysman/emd/agntstmp.txt
$rm $AGENT_HOME/sysman/emd/lastupld.xml
$rm $AGENT_HOME/sysman/emd/state/*.*
$rm $AGENT_HOME/sysman/emd/upload/*.*
$AGENT_HOME/bin/emctl start agent

Policy evaluation cycle

The following table describes the default evaluation cycles for Grid Control 10g, which will also determine when a violation is cleared :
  1.     Configuration Policies:  once every 12 hrs
  2.     Storage Policies:  once every 3 days
  3.     Object Policies:  once every 12 hrs
  4.     Security Policies:  once every day
These intervals are fixed for EM version 10.1.0.
Note that they are tied to the polling frequency of the associated metric.

If you would like to reevaluate the policies right now than perform the following steps :

$cd $AGENT_HOME/bin
$./emctl stop agent
$./emctl start agent
$./emctl clearstate agent
$./emctl upload agent
$./emctl clearstate agent

ypmatch: the domainname hasn't been set on this machine

Metric error received in Grid Control 10.1.0.4.0 view for AIX 5.2 target host: "ypmatch: the domainname hasn't been set on this machine."

The cause of this problem is that "bos.net.nis.client" fileset is installed, but NIS is not configured for use. The existence of this fileset causes the ypmatch command to be used as part of host metric collection, but this command will fail if NIS is not configured for use on the system.

To disable the file give the command: chmod -x /usr/bin/ypmatch, so the command cannot be sent and the error is not generated.

Error rendering element. Exception: ORA-01017: invalid username/password; logon denied

When starting a report from OEM the above error occurs. Probably the MGMT_VIEW password has been changed. See metalink : How To Change The MGMT_VIEW Password Used By Grid Control Repository [ID 374382.1]

To change the MGMT_VIEW user password:
For 10.2.0.1.0 to 10.2.0.4.0
1. Stop the OMS
cd OMS $OH/bin
$ ./emctl stop oms
2. Confirm that there is a recent backup of the repository
3. Change the MGMT_VIEW user password:
There is an API call to reset the password for MGMT_VIEW:
SQL> connect sysman/<pwd> SQL> exec mgmt_view_priv.change_view_user_password('<Insert PWD here>');
4. Start the OMS
cd OMS $OH/bin
$ ./emctl start oms

For 10.2.0.5.0 and 11.1
Starting from Grid Control 10.2.0.5.0 there is a new "emctl" command to change the MGMT_VIEW user password:
cd <OMS_HOME>/bin
> $ emctl config oms -change_view_user_pwd [-sysman_pwd <sysman_pwd>] [-user_pwd <user_pwd>] [-auto_generate]
The above does not need the emkey to be copied into the repository.
-sysman_pwd
 The password for the SYSMAN user.

-user_pwd
 The new password for theMGMT_VIEW user. This is an optional parameter and if it is not specified, the password is auto generated.

-auto_generate
 If this option is specified, the password is auto generated.

Failed to initialize the target manager

When starting the agent the following error occured :

$ emctl start agent
Oracle Enterprise Manager 10g Release 4 Grid Control 10.2.0.4.0.
Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved.
Starting agent ...... failed.
Failed to initialize the Target Manager.


The problem was that the Grid Control agent $AGENT_HOME/sysman/emd/targets.xml file was empty. I recreated the file using :

$agentca -d -f

Either rediscover(-d) or reconfigure(-f) option is mandatory
From metalink :
How To Recreate a Targets.xml File in Grid Control 10.2.x Using 'agentca' Commands [ID 365252.1]   

$ cd $ORACLE_HOME/sysman/emd   (where $ORACLE_HOME is the Agent's home)
$ touch targets.xml


Once you have created the targets.xml file, view the Agent's $ORACLE_HOME/sysman/config/emd.properties file. 
Copy the values for:
...
agentSeed=12345678
...
EMD_URL=http://hostname.oracle.com:3872/emd/main
...

For example:
<Targets AGENT_SEED="12345678">
            <Target TYPE="oracle_emd" NAME="hostname:3872"/>
            <Target TYPE="host" NAME="hostname"/>
</Targets>


Run agentca -d to force a new discovery of the targets on the host
$agentca -d

password for the agent (DBSNMP} needs to be (re)entered.

Collection status : Disabled by Upload Manager

When retrieving the status of the Grid Control agent I noticed that the collection status has a value of "Disabled by Upload Manager". This error is caused by a file that can't be uploaded.

$ ./emctl status agent
Oracle Enterprise Manager 10g Release 3 Grid Control 10.2.0.3.0.
Copyright (c) 1996, 2007 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version     : 10.2.0.3.0
OMS Version       : 10.2.0.3.0
Protocol Version  : 10.2.0.2.0
Agent Home        : /app/oracle/product/agent10g
Agent binaries    : /app/oracle/product/agent10g
Agent Process ID  : 1634352
Parent Process ID : 807000
Agent URL         :
https://hostname:3872/emd/main/
Repository URL    : https://hostname:1159/em/upload
Started at        : 2009-04-20 14:42:37
Started by user   : oracle
Last Reload       : 2009-04-20 14:42:37
Last successful upload                       : (none)
Last attempted upload                        : (none)
Total Megabytes of XML files uploaded so far :     0.00
Number of XML files pending upload           :      882
Size of XML files pending upload(MB)         :    50.40
Available disk space on upload filesystem    :    14.71%
Collection Status                            : Disabled by Upload Manager
Last successful heartbeat to OMS             : 2009-04-20 14:42:56
---------------------------------------------------------------
Agent is Running and Ready


[AGENT /app/oracle/product/agent10g/bin ]
$ ./emctl upload
Oracle Enterprise Manager 10g Release 3 Grid Control 10.2.0.3.0.
Copyright (c) 1996, 2007 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
EMD upload error: Failed to upload file A0000017.xml, ret = -2


If you remove the file from $AGENT_HOME/sysman/emd/upload the upload will succeed and the status will be normal again.

Failed to start HTTP listener

When starting the Grid Control agent, the following error occured :
$ ./emctl start agent
Oracle Enterprise Manager 10g Release 3 Grid Control 10.2.0.3.0. 
Copyright (c) 1996, 2007 Oracle Corporation.  All rights reserved.
Starting agent ....... failed.
Failed to start HTTP listener.
Consult the log files in: /app/oracle/product/agent10g/sysman/log

If you have a look at the processes, you´ll see that the agent is still running.

[AGENT /app/oracle/product/agent10g/bin ]
$ ps -ef|grep -i em
  oracle  131100       1   0   Dec 16      - 101:31 /app/oracle/product/agent10g/bin/emagent
    root  110732       1   0   Aug 06      -  0:00 /usr/lib/errdemon
  oracle  508066  782468   0 08:57:42  pts/0  0:00 grep -i em

After killing the running agent everything works fine :
[AGENT /app/oracle/product/agent10g/bin ]
$ kill -9 131100

[AGENT /app/oracle/product/oma10.2.0/agent10g/bin ]
$ ps -ef|grep -i em
    root  110732       1   0   Aug 06      -  0:00 /usr/lib/errdemon
  oracle  508066  782468   0 08:57:42  pts/0  0:00 grep -i em

[AGENT /app/oracle/product/agent10g/bin ]
$ ./emctl start agent
Oracle Enterprise Manager 10g Release 3 Grid Control 10.2.0.3.0. 
Copyright (c) 1996, 2007 Oracle Corporation.  All rights reserved.
Starting agent ............ started.

Remove targets or agents from repository

There are some issues while removing targets or agents from Grid Control. After removing the target, the removed agent is still seeing by the Grid Control. There is still a reference in the repository to the removed targets or agents.

You can use the following instructions as sysman to get rid of the removed agent or target :

To clean it manually, connect to the OEM Grid Repository and perform the following:
SQL> select target_name from mgmt_targets where target_type=’oracle_emd’;

This command shows all the registered targets. Then to manually remove a host from the list execute:
SQL> exec mgmt_admin.cleanup_agent('host:port');

or use the alternative
SQL> exec mgmt_admin.delete_target_internal('host:port', 'oracle_emd');

Collection errors agent

When the agent can't collect metrics due to collection errors please have a look at the collection message. I ran into the following error :

The number of Volume groups to be monitored is larger than 10. Please add the Volume groups to be monitored in the config file emagent_storage.config

The solution is simple. Add the volumes groups on the target host to the emagent_storage.config located in
$ORACLE_HOME/sysman/emd/emagent_storage.config. The neccessary volume groups can be found with then lsvg command. Put the volume groups between the volumes:START tag en volumes:END tag.

$cat emagent_storage.config
Volumes:START
rootvg
app_lpn2vg
NLSHDvg
EastEuSHDvg
NLINTvg
PLSHDvg
ROINTvg
V9TSTvg
BMvg
Oradumpvg
HUSHDvg
arc_lpn2vg
PLFHDvg
HUINTvg
pag_lpn2vg
PLINTvg
CZCOMvg
COMvg
NLCOMvg
HUCOMvg
PLCOMvg
ROCOMvg
Volumes:END