Category Archives: Microsoft

OCS 2007 R2 Schema Prep Failure “failure occurred attempting to check the schema state. please ensure active directory is reachable"

A customer called in today with an issue preparing their OCS 2007 R2 environment. The customer had previously started installation on a 2008 R2 server, and started over with a 2008 R1 server. They had only completed the Active Directory Preparations prior to starting over. The issue was when they started on the server they were unable to see the schema prep, they were receiving this error in the install GUI:

ocsschemaprepfailure

A few interesting things here, the machine is joined to the domain, I could contact all domain controllers, I could modify the schema using the schema MMC snapin. However, the OCS install run via command line or GUI would not contact active directory.

Through some quick googling I found that the installer queries the SRV records for contacting the PDC in active directory. This SRV record is:

_ldap._tcp.pdc._msdcs. DnsDomainName

After pointing to a DNS issue, the customer realized their server was pointing to a public DNS server, not an active directory integrated server, which did not have the SRV records needed to perform these tasks. Once the DNS server was changed, the installer read the Active Directory Preparation as Completed and you could do a proper nslookup on those SRV records

srvldapquery

Advertisements

Collecting OCS Performance Data For PSS

It has been a while since I have had a post over here, guess you can blame the holiday season as well as the busy beginning of the year at Winxnet. Anyways, I have been working with PSS on an issue with external live meeting through the edge for quite some time now, and with that has been lots of performance monitor collection, after clicking through all of the different collectors multiple times, I decided to create some templates to have for future use and wanted to share them. Its nothing special, no awesome script or anything complicated, but a very basic tool that may be useful to anyone going forward.

http://cid-389bd51b03b1f8f9.office.live.com/embedicon.aspx/Public/Performance%20Monitor%20Templates
Above is a link for access to these files on my sky drive. If you have any issues accessing these please leave a comment or contact me via email and I will get them to you.

They are very easy to use, once downloaded, open Reliability and Performance Monitor from either your Front End or Edge Server…

image

Expand “Data Collector Sets” and right click on the User Defined folder. Choose New->Data Collector Set

image

Name the collector set whatever you would like and make sure to choose Create from a template.

Click next to access the next page in the configuration wizard. Choose Browse and locate the XML file you downloaded containing the template information. Once you select that file the page should look like this:

image

The next two screens will ask you where to save this file, I would suggest a drive with plenty of space as these can get very large depending on the amount of traffic on your server and how long they are running.

 

When you are ready to collect data simply right click on the set you created and choose start.

image

Once you are ready to analyze data, or send to Microsoft PSS for data analysis you simply choose stop, and you will have a file in the location you specified. Microsoft PSS uses a tool called PAL(Performance Analysis of Logs) which is an open source application written by a Microsoft employee. This tool can be found here:  http://www.codeplex.com/PAL If you are feeling up to performing some of your own analysis this is a great tool to use. I may try to post some more detailed information on using this tool soon.

 

The templates included in my link include the following Counters:

 

Logical Disk

Memory

Network Interface

Objects

Paging File

Physical Disk

Process

Processor

TCPv4

Thread

All <LC: > Counters

 

Hopefully soon I will have a new post describing the fix for this strange live meeting issue, until then, Enjoy!

Having trouble creating OCS Shares on non windows file server?

On two recent deployments our team ran into some issue when trying to create pools with file shares created on an EMC NAS device. The install wizard was not seeing those folders as setup properly so it could not complete the install. All the permissions were checked best we could see, but it still did not like the share configuration.

I decided to try creating the shares on one of the front end servers and moving on with installation, I then would xcopy the data over to the EMC NAS, and then modify the file locations for the pool.

There is a great article here that goes over the bulk of this migration process, the one difference for my situation is the xcopy from windows share over to the EMC device.

In my example my file shares were in D:\OCS Shares on local folder and I had mapped the folder on the EMC containing all of the OCS Shares to Z:

xcopy “d:\ocs shares” z: /e /k /o /x /y

 

This basically will copy everything in the D:\ocs shares folder, including sub directories and files to the Z: drive.

All attributes, including all ACL information will be kept, and it will not prompt to overwrite any existing files.

Once we completed this xcopy, we followed the instructions in the above link to modify all WMI settings and all IIS settings, from there we were able to restart services and verify all functionality.

Obviously there is a way to get the EMC NAS file shares setup correctly, however I did not have this knowledge, and as a time saver, this was the best fix for me.

tweetmeme_source = ‘winxnetuc’;
tweetmeme_service = ‘bit.ly’;

How to Disable “Schedule A Conference Call” button in Live Meeting Outlook Add-In

Some organizations will deploy LiveMeeting either without the existence of OCS, or without OCS Audio/Video Conferencing being enabled. By default, the conferencing addin for outlook has a Schedule A Conference Call button regardless of the environment you are connecting it to.

There is a registry entry that can be used to disable this button:

1. Locate and then click to select the following registry subkey:

HKEY_LOCAL_MACHINE\Software\Microsoft\Live Meeting\Addins

Note Use this subkey for x86-based systems. If you are running a x64-based system, locate and click the following subkey:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Live Meeting\Addins

2. After you select the subkey that is specified in step 3, point to New on the Edit menu, and then click DWORD Value.

3. Type RemoveConferenceCall, and then press ENTER.

4. Right-click RemoveConferenceCall, and then click Modify.

5. In the Value data box, type 1, and then click OK.

6. On the File menu, click Exit to close Registry Editor.

If you wish to remove this setting, simply change the Value to 0 and the button will be available again.

A full list of livemeeting registry keys can be found here: http://technet.microsoft.com/en-us/library/dd637135(office.13).aspx

tweetmeme_source = ‘winxnetuc’;
tweetmeme_service = ‘bit.ly’;

CWA Error through F5 Load Balancer: Your Connection was ended. Please Sign in Again. (Error Code: 0-1-482)

 

On a recent deployment we deployed CWA internally and externally using ISA Server 2006. The customer decided they wanted to provide high availability to the CWA service, so we introduced a hardware load balancer to provide that functionality. After we set the two servers with identical site settings behind the load balancer we started having users receive this error when connecting to the CWA site:

cwaerror1

At first glance deploying CWA through a load balancer would seem pretty basic, they are websites you access over https, however there is some key information in the R2 Documentation for deploying CWA behind a load balancer. http://technet.microsoft.com/en-us/library/dd441196(office.13).aspx

Communicator Web Access supports most hardware load balancers, provided that the load balancer:

  • Allows you to set the TCP idle timeout to 1,800 seconds (30 minutes). The TCP idle timeout represents the amount of time the server will wait for information during a session. If you are using a reverse proxy server (such as Microsoft Internet Security and Acceleration Server) then the TCP idle timeout on that computer should also be set to 1,800 seconds.
  • Allows you to use a source network address translation (SNAT) pool if you need to handle more than 65,000 simultaneous connections. SNAT is designed to "hide" multiple servers behind a single IP address (that is, a number of servers can be accessed using just one IP address). With a SNAT pool, servers can be hidden behind multiple IP addresses.
  • Allows you to use cookie persistence when configuring session affinity. With cookie persistence, information about the actual Communicator Web Access server being used for a session is stored in an Internet cookie on the client computer. When configuring the load balancer’s session persistence profile it is recommended that you use "HTTP Cookie Insert." With this configuration method, information about the server to which the client is connected is inserted in the header of the HTTP response from that server as a cookie.

Our issue was related to the persistence profile. When a user connects to CWA they must maintain a connection to the same server as the initial connection or it will not work. The persistence profile, using a HTTP Cookie Insert method will enable this persistence.

We were using an F5 BIG IP LTM Load balancer for this deployment, we actually chose “Source Address Affinity”. Below you can seen a screenshot of the persistence profile used in this configuration.

f5cwaconfig

tweetmeme_source = ‘winxnetuc’;
tweetmeme_service = ‘bit.ly’;

A/V Conferencing From External Issue- SIP 403 Forbidden Error

On a recent deployment I ran into an issue where everything was working correctly except an external user trying to join or create an Audio Video Conference. The customer had an enterprise edition consolidated configuration behind an F5 Load Balancer. Doing our initial sip traces we were able to see a 500 error when the external user would try to join or create a conference.

Start-Line: SIP/2.0 500 The server encountered an unexpected internal error

ms-diagnostics: 3080;reason="Internal Error: AddUser failed";source="front end server fqdn"

I removed most of the trace except the important parts. What you will see in the above trace is the SIP 500 error, and then at the bottom the AddUser is failing on the front end server. This exact symptom with an enterprise pool behind load balancers points to this KB article: http://support.microsoft.com/kb/946091. This fix explains an issue with the load balancer being in DNAT mode instead of SNAT mode. However our F5 was using SNAT for all of the OCS traffic, and the pool setting was correctly set to not be in DNAT mode.

Running more traces another error popped up which was a SIP 403 Forbidden:

SIP/2.0 403 Forbidden

SERVER: RTCC/3.5.0.0 MRAS/2.0
ms-edge-proxy-message-trust: ms-source-type=EdgeProxyGenerated;ms-ep-fqdn=Edge Internal interfacefqdn;ms-source-verified-user=verified
Ms-diagnostics: 9006;source="Edge Internal interfacefqdn";reason="Forbidden";component="Media Relay Authentication Service"

This basically means that the front end server is not able to get media relay authentication from the edge server A/V internal interface.

If this is happening you will also see an error in the event logs:

Log Name:      Office Communications Server
Source:        OCS Audio-Video Conferencing Server
Date:          9/25/2009 4:12:14 PM
Event ID:      32018
Task Category: (1017)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:     FRONT END SERVER FQDN

Description:
The Audio-Video Conferencing Server encountered an error when requesting credentials from the A/V Edge Authentication Service.

A/V Authentication Service Service URI sip:EdgeInternalFQDN@swk.pri;gruu;opaque=srvr:MRAS:HqCEupOMck6C3onsDHul1wAA, Reason: The operation has failed. See the exception’s properties as well as the logs for additional information.
Cause: The Audio-Video Conferencing Server cannot communicate with A/V Authentication Service.
Resolution:
Check the A/V Authentication Service is alive and that network connectivity exists.

Connectivity was available through the internal edge VIP as well as each individual edge server’s internal interface. Also, if you ran an A/V Conferencing Validation on each of the front end servers it would succeed on all tests.

I ran through this with PSS and there were two things we discovered. The first potential issue was on the Internal tab setting of the edge server. Per the Microsoft documentation when doing an enterprise deployment the name that should be listed on the “Internal Servers Authorized to Connect to this edge server” setting is the pool FQDN, not each individual front end server. There has been some debate about whether you should add the FQDN of each front end server to this list as well, because we were seeing the front end servers get denied access to the A/V Authentication service we decided to try it anyways.

edgeinternalsetting(Pictures Modified to protect customer info)

The other change that was made was in the forest global settings section. On the general tab you specify your internal SIP domains and you check one for the default routing domain. In this case the customer AD domain was different from the SIP domain, both were listed, however the AD Domain was checked as the domain to be used for the default routing. Once we changed that setting to have the SIP Domain as the default routing domain and restarted the services on the front end servers, A/V conferencing started functioning properly.

globaldomainsettings 
(Pictures Modified to protect customer info)

I am hoping I can remove each setting and try to narrow it down to one ,but either way the internal interface setting has proved to fix some funky issues in deployments, so both of these may want to be set regardless.

tweetmeme_source = ‘winxnetuc’;
tweetmeme_service = ‘bit.ly’;

Migrating the OCS Enterprise Edition Back-End Databases

There is a blog post here: http://blogs.technet.com/dodeitte/archive/2009/06/05/how-to-move-ocs-enterprise-sql-database-to-another-server.aspx

This post outlines the basic process of taking the databases offline, migrating them, mounting them in the new instance and running LCSCMD to update the pool backened.

I recently did this in a production deployment of R2 and actually found a missing step, there was also a post on the Technet forums with a user having the same issue so I figured I would post the updated process here. This may not always be the case, but a key thing to check, and what ended up being the fix in my situation was the actual pool setting in active directory.

I believe the attribute that the below command updates is msRTCSIP-BackEndServer

LCSCmd.exe /Forest /Action:UpdatePoolBackend /PoolName:<pool name> /poolbe:<SQL instance name (machine\instance name)>

When I ran through this process I found that when I opened ADSI Edit and browsed to this attribute it actually had not changed.

adsieditbackenddatabase

The DN For the pool object will be located at : CN=Poolname,CN=Pools,CN=RTC Service,CN=Microsoft,CN=System/Config Container,DC=Domain,DC=com

 

Again this may not always be the case but in my experience this was thef ix for the issue.

tweetmeme_source = ‘winxnetuc’;
tweetmeme_service = ‘bit.ly’;

R2 July Updates Break Dial In Conferencing Pin Services (If you miss an update)

 

One of the more confusing things about the QFE sets Microsoft releases is that the KB articles specifically say to only apply patches if you are experiencing the issues, however in an environment like OCS if you patch some components, others will not work.

After applying the July round of updates to our internal environment users were no longer able to authenticate to PSTN dial in conferences using their extensions and pin numbers. If the user signed into CWA to check this information they would see this message:

cwaerror

I poked around a bit and noticed one thing that was changed from before I applied the updates, this was not the root cause of the issue but may be something to look out for, my conferencing region was assigned to the wrong location profile, prior to these updates it was assigned to the correct location profile:

confattendant1

As you can guess that is a testing location profile with the number 2 on the end.

 

The root cause of the issue was actually related to the below error message which I was receiving on the EE Front End Server:

Error    8/22/2009 7:40:41 AM    OCS UserPin Service    47019    (1044)

ocsuserpinserviceerror 

This pointed to a database issue, I reviewed the July updates and noticed I had not applied the backend database hotfix, I promptly installed the SQL 2005 Client tools on my front end server and ran this Command to install the July backend database update.

dbupdatecli

Once I installed this update I was able to connect to my CWA server and view my correct dial in information, as well as authenticate with conferencing bridges via PSTN.

I guess it is just important to note that you should install all updates associated with a QFE release or it is possible to run into issues like this.

The secret to VGA and HD Video in OCS 2007 R2

One of the great features in OCS 2007 R2 is the support for HD video for peer to peer calls. This is still not supported in conferences, however you can utilize VGA and HD Cameras in video calls between users. There are a couple of important things to know about this to get it working.

 

You will notice in the Front End settings for the pool you can specify a maximum video quality:

image

This setting will specify the maximum video quality for the pool, however you will need to provide client side policies to enable higher quality video.

If you wish to utilize VGA or HD720p video between two endpoints you will need to ensure the below registry entry is present on both of the clients.

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Microsoft\RTC\Quality]
"MaxAllowedSendVideoSize"=dword:00000002
"MaxAllowedReceiveVideoSize"=dword:00000002

This will allow the two users to send and receive HD video, by default this is not enabled.

You will want to make sure you take bandwidth and PC capabilities into consideration when enabling this feature because video can be intense on the client desktop as well as a bandwidth killer as you can see in the first screenshot with the estimated amount of bandwidth required to send that video stream.

You can find more information about the client registry setting here: http://office.microsoft.com/en-us/help/HA102362191033.aspx

You can find more information about the OCS R2 HD Video enhancements here: http://technet.microsoft.com/en-us/magazine/2009.03.ocsr2.aspx?pr=blog

At this time there are no HD Video endpoints certified for OCS but by the end of this year Tandberg and Microsoft should have desktop USB devices capable of HD720p video with OCS.

Configuring OCS DVT Agent/Organizer Scenario

There is no documentation on configuring the OCS Deployment Validation Tool in the Agent/Organizer Scenario. Setting up the Auto-Answer Agent is pretty straight forward and is documented in detail at Byron Spurlock’s blog here: http://blogs.msdn.com/byrons/archive/2009/06/23/test-posting.aspx

The Agent/Organizer scenario however is a bit more interesting.

To start, you can find the Deployment Validation Tool installation files in the OCS 2007 R2 Resource Kit files which can be downloaded here: http://www.microsoft.com/downloads/details.aspx?familyid=9E79A236-C0DF-4A72-ABA6-9A9602A93ED0&displaylang=en

In the installation directory there is a sub folder for Deployment Validation Tool which has DVTAgent.MSI and DVTOrganizer.MSI.

First off both installs require you to install the UCMA Redistributable package which you can find in the OCS 2007 R2 setup files under Setup\AMD64\Setup\ucmaredist.msi

Once that package is installed you will also need to install SQL Express 2005 SP2 on the server you have picked as the organizer. This should not go on any heavy utilized OCS roles like the front end servers, in my environment I chose to put this on the Monitoring Server. Install SQL Express with all of the default options and continue with the Organizer install.

Before you run the organizer install be sure to create a SIP enabled account for the organizer. In my examples my account is DVT_Organizer@winxnet.com .

Once SQL Express 2005 SP2 is installed you can run the DVTOrganizer.MSI.

The organizer service installs are pretty straight forward, the defaults should work for the installation. However, there is one piece in the Organizer Configuration that will throw you off. By default using the guides for Auto Answering Agent configuration you are instructed to leave the box checked for Use Default Credentials when configuring the Agents. I was having issues where all of my agents were showing offline and the Organizer server would report this error in the event logs:

 

Warning    8/3/2009 9:15:19 PM    OCS Deployment Validation Tool    51019    (1050)

The service wasn’t able to register with focus. It will attempt to reregister automatically.

Service : Organizer, URI: dvt_organizer@winxnet.com
Cause: This might be due to a configuration error, or due to network or focus problems.
Resolution:
Please check the configuration of the service including the account credentials used to register with focus.

After adjusting the Organizer configuration to specify the actual user credentials I was able to get the organizer to register with the focus and recognize the agents as Online.

 

configorganizer1

Once you have the organizer installed you will want to install a couple of agents, try to put them across different subnets or different physical locations to make sure you get diverse scenarios for your call testing. Also try not to put the agent on any OCS roles if possible, I have mine located on terminal servers and desktops in the environment. You will need to create SIP enabled accounts for each agent in the environment. In my example I have DVT_SERVERAGENT@winxnet.com, DVT_DCAGENT@winxnet.com and DVT_DESKTOPAGENT@winxnet.com these represent three separate subnets in my environment. Install the UCMARedist.MSI package on the agent machine and use the default information for the installation.

agentinstall1

During the installation you may also see this error pop up which you can ignore:

agentinstallerror

At the end of the installation you will see the Agent Configuration tool pop up.

You will want to configure the agent as a Unified Communications agent with the SIP URI you have created for the agent and to be safe, manually configure the server information.

agentconfig1

If you receive an error while configuring this agent and you are using Server 2008 or Vista/Windows 7 you will want to make sure UAC is off, or you will want to re open the agent configuration by choosing Run As Administrator.

Once this configuration is complete you will need to setup the service in windows to use the sip account of the agent.

agentconfigservicelogon

Hit OK and start the service, jump back over to the organizer server and open the Organizer Admin Console to add your agents to the roster.

organizeraddagent1

Its important to note that you will need at least 2 agents and an organizer to perform any tests. The organizer will perform tests between the agents in your roster with peer to peer as well as conference calls.

Once in the Admin Console you can view your test suite status, the easiest thing to do is to hit restore to default and it will configure the default tests between all of the agents in your roster.

testsuite

The screenshot above shows the test suite tab with tests in progress and tests that have completed.

You can also view reports and alerts from all of your tests in their respective Tabs. The reports tab will show all tests that were done, if you choose a test and right click you can choose to see the complete details of the test which will look like this.

reportdetails

The report above shows the agents and details such as MOS Scores and Network connectivity and test length. This can be very useful when troubleshooting call quality issues.

If you have any additional questions about configuration please reach out to me, I am constantly testing new ways to use this tool and I will try to do another write up on the MOM Integration for alerting purposes as well.