SCOM Installation Headache !!!

I am now slowly picking up my SCOM skills and what i can say is that, it never fails to surprise me each time , where chances are, you might get stuck in the simplest thing you can imagine.

In another posting which i was supposed to implement SCOM 2012 infrastructure, installation of the SCOM servers should be straight forward, but in end, it took me almost 3 days to find out what’s wrong. I got hit by the error below.

With the error message like this, you might think that it is probably due to incorrect passwords or the length of the password might be a bit too long. Had made sure that the passwords are correct and even reduce its length for testing. In the end it still won’t work.
Sensing something is not very right, i opened up the SCOM installation logs located in  “C:\users\<SCOM installation account>\AppData\Local\SCOM\LOGS” , which shows you the below

Open up the “OpsMgrSetupWizard”, try to see if the following entries are logged,

If it is, chances are you might want to check with your client to see if there was any “Authoritative Restore” done to their AD recently.
You may want to refer to this kb974886 article. Since i was not able to re-produce the problem in the lab environment, i am not able to comment if this hotfix does resolve the issue. If anyone of you out there who had the similar problem as mine and tested this hotfix, appreciate your valuable comments.
The reason why it took me a while before realizing this was because there isn’t much documentation on this, and I've decide to make a post for this so that this could be for reference in the future.

Upgrading System Center Operations Manager 2007 R2 to 2012

With the launching of System Center 2012, you might be tasked to upgrade existing SCOM 2007 R2 infrastructure to 2012. Information relates to the upgrade can be found here http://technet.microsoft.com/en-us/library/hh205974.aspx . To cut the story short, this is what you need to make sure that the upgrade is successful

In your existing SCOM 2007 R2, (well, at least in my testing environment) it should come with CU 5 installed. Next, before the upgrade, make sure that the server is running 64 bit OS. (can refer to list of support configuration for 2012 here). Next, you will have to install all the pre-requisites needed for SCOM 2012 such as the .Net Framework 4, ReportViewer and Silverlight. (for this, the SCOM 2012 setup wizard will run a pre-requisite checks for you, before you proceed.)

It is told that the agents will need to be upgraded, before monitoring can be resumed. But how are you going to upgrade the agents, if your environment monitors a large number of servers, for e.g. more than 100 servers ? If you are going to manually upgrade it, then it might take time.

Another way you may try is to upgrade the agents, after the RMS and MS had been successfully upgraded to 2012. When you launch the SCOM 2012 console, under the “Pending Management” section, SCOM 2012 will actually show you the list of Agents that require update. What you need to do is to manually approve the upgrade for the agents and it will be automatically handled by SCOM 2012 after this.

Performance of SCOM Agent

We know that in order to have a machine / server to be monitored by SCOM, you will have to install an agent on the target machine / server, only then, monitoring will start to kick in, provided that the Agent Service is started.

Next, how do you know if the performance of the Agent is affected over time ? Recently involved in another project which gave me an opportunity to implement SCOM monitoring in a larger environment, I came across the following alert.

Thanks to my helpful senior who is always ready to guide, I do not have the spend to much time searching over the net for solution. Apparently the alert above might be due to the cache of the Agent has exceeded the threshold. you may try to restart the Agent service again, but if you are still encountering the similar alert, probably you can “rebuild” the cache of the SCOM Agent again.

Here’s what you can do.
1 – login to the server that raised this alert

2 – locate the “Health Service Store” folder in the <SCOM Agent Installation folder>\Health Service State folder.

3 – Stop the Agent Service.

4 – Once Stopped, rename the current “Health Service Store” folder to another name.

5 – Start the Agent Service. Once started, you should see a new “Health Service Store” folder created by itself, and looking at the even viewer, the SCOM Agent will now start to download the MPs from the management servers again.

6 – Refresh the Health Explorer or Reset that specific monitor and let it settle for a while and you should a Healthy state for this monitor.

SCOM 2007 Installation checklist

I finally got involved in my first SCOM project and realized that client’s IT environment might not be as ideally as how we want it to be, in terms of their IT security policy / GPO. However, below is a some checklist which i think it is handy before we proceed with SCOM installation without much hiccups.

• SCOM action account is created and granted with relevant permissions. Make sure that the SCOM action account is granted “log on as a service” and “log on as a batch job”. “Log on as a batch job” is required, especially if you are using SCOM action account to run the Service Level Dashboard. If not, the application pool for SLD in the IIS will be immediately stopped, once the SLD website is loaded.
• Check if “System cryptography: Use FIPS compliant algorithms for encryption, hashing and signing” is enabled and can be disabled if it is. If not, then you might need to apply kb911722 for the SCOM web console, if it is displaying "the implementation was not part of the windows platform FIPS validated cryptographic algorithms" at your browser.
• The rest will the same as my previous SCOM 101 post.

SQL 2008 R2 Reporting Services

Installing the SQL 2008 R2 reporting services is definitely a simple and straight forward step where you can just check to enable the option during the SQL 2008 R2 server installation. Often, it is the configuration part that gives you the headache.

When you try to access the SQL reporting or reportserver sites, you might encounter the following error message in the web browser, or the event logs below (screen captures). You might want to check the credentials used to run the SQL reporting service.

“The report server cannot decrypt the symmetric key that is used to access sensitive or encrypted data in a report server database. You must either restore a backup key or delete all encrypted content. (rsReportServerDisabled) Get Online Help

Bad Data. (Exception from HRESULT: 0x80090005)”

 

Tried changing the account used to run the reporting service to use the SCOM admin account and it works !!!

Installing Service Level Dashboard

Had tried to install Service Level Dashboard and even though it was successful, the report does not seems to be appearing. Checked the logs, it says, “Message :System.Data.SqlClient.SqlException: Could not find stored procedure 'GetSLADetails'.” and tried re-installing for a few times but not successful.
The message above was all about “GetSLADetails” stored procedure was not found in the database. This stored procedure should be located in SCOM’s Data Warehouse database, which it did not happen to the be there . Checking the net, seems that it might be due to the stored proc. was not successfully created during the installation of the SCOM Service Level Dashboard MP, manually re-run the SQL script to create this stored proc. should do the trick (at least it worked for me !!)
Please refer to the page here for the stored procedure.

SCOM 101

Did my first lab on my own after a few days of introductory sessions to System Center Operations Manager 2007 R2, which I will refer it as SCOM from now on.

Lesson 1 : Installation
Stuffs that you need

• server (yeah, of course you need at least one)
• SCOM installation DVD and some pre-requisite software / server feature/roles you need to install
• SQL server installation DVD
• A full list of requirements to run SCOM , can be located here

You can install all the stuffs in a single server. To make things a little bit more interesting, i had setup a 3 VMs (1 DC, 1 SQL server and 1 SCOM server, all running on Windows Server 2008 R2 sp1 Ent version.

Steps

1. install the .Net Framework, IIS service (SQL, SCOM Servers)
   install AJAX extension 1.0 (in the SCOM server, if you are separating the roles)
2. install the SQL server (I installed SQL server 2008 sp1) for this test
3. if the SQL server is installed in a separate server, then you will have to run the "DBCreateWizard.exe" file that comes with the SCOM installation DVD / CD. If you create an empty Database for SCOM manually, it might not work as you might encounter an error saying something like “invalid Microsoft Operations Manager database version” –> see below
   
   
4. if you are installing a separate SQL server, then there is a few extra steps you need to do. First, you should uncheck the database option, while install SCOM
   
   
5. Select the target SQL server where the “DBCreateWizard.exe” has setup the database for you before this. The SCOM database name will be using the default database name
   
   
6. One thing you need to make sure is that the firewall settings in your SQL server should allow the SQL traffic. This is because Windows server 2008 will be default, disallow all network traffic, except those required by the OS itself.
7. Once this is done, installation is pretty much straight forward where SCOM will prompt you for the type of features you want to install and the type of account to run those features.

Managed to install SCOM after a few trials and looking up the net for resources. Will be starting to explore more its functionality after this. Stay tuned.