VMware Horizon View – Performing Maintenance on or Replacing an ESX Host in a Cluster

I wanted to provide some information to help clarify the process for performing maintenance on an ESXi host in a vCenter cluster that has active virtual machines on it. My experience with this type of maintenance is limited to Horizon View virtual desktops, but it is likely also applicable to clusters hosting virtual servers.

Replacing an ESX host in a cluster that has View Composer linked-clone pool installed (1015292) 

“To replace a ESX host in a cluster with deployed View Desktops:
  1. Prepare the new ESX host outside the cluster, verify all datastores that are available to the old ESX hosts in the cluster are accessible to the new ESX host.
  2. Put the old ESX host in Maintenance Mode from the vCenter Server GUI.
  3. Ensure that you have selected the Move powered off and suspended virtual machines to other hosts in the cluster option.
  4. The running virtual machines migrate to other ESX hosts in the cluster, shutdown and suspended virtual machines and replicas are moved as well. The ESX host enters Maintenance Mode.Note: If the Replica virtual machine did not migrate then you need to unprotect it. For instructions on unprotecting the replica virtual machine, see Cannot remove source and replica virtual machines associated with View Composer desktop pools (1008704).
  5. Click the ESX host
  6. Click the Virtual Machines tab to verify that all virtual machines and replicas have been moved.
  7. In View 4.5 and later, if you have unprotected the replica virtual machine then you need to re-protect it. For more information, see Re-protecting a View replica virtual machine (2015006).
  8. Remove the ESX host in Maintenance Mode from the cluster.”

So, in summary, you should be able to put the host into Maintenance Mode and, if you have the cluster configured properly, the virtual desktops that are on it should be migrated over to other available hosts in the cluster.

If you have trouble putting an ESX/ESXi host into maintenance mode, check out ESXi/ESX host fails to go into maintenance mode (1036167).

One specific instance I’d like to note from personal experience: “If an ESXi/ESX host is a part of VMware High Availability (HA) or DRS cluster, check the Admission Control settings. You may have to disable this option if there are not enough resources to ensure fail over capacity. “

Which refers to this setting on your cluster:

AdmissionControl

For a more detailed explanation of Admission Control, see VMware HA Admission Control. The important parts pertaining to our topic of conversation:

“vCenter Server uses admission control to ensure that sufficient resources are available in a cluster to provide failover protection and to ensure that virtual machine resource reservations are respected.”

“Admission control imposes constraints on resource usage and any action that would violate these constraints is not permitted. Examples of actions that could be disallowed include the following:

  • Powering on a virtual machine.
  • Migrating a virtual machine onto a host or into a cluster or resource pool.
  • Increasing the CPU or memory reservation of a virtual machine.”

The second bullet is in bold because in order for a host to enter Maintenance Mode, all of the VMs assigned to that host must first be migrated to other available hosts in the cluster or powered off. If Admission Control is enabled on your cluster and during the VM migration process you violate the HA failover capacity check, the migrations will stop and you’ll likely get an error that the host could not enter maintenance mode.

At this point, you basically have two options; modify your Admission Control Policy settings to provide more failover capacity, or shutdown non-essential VMs on the host entering maintenance mode and disable Admission Control. Use caution doing either of these to avoid overwhelming your other hosts once the VMs have been migrated. If you disable AC, make sure you remember to enable it again once your maintenance has finished.

For some good resources on vSphere HA, check out Yellow-Bricks HS DeepDive and HA cluster configuration: Requirements and steps

-Rick

Advertisements

VMware Horizon View – Resource Dump

When preparing for my VMware Horizon View deployment, I spent a lot of time (as you should too) searching, reading, and parsing through official documentation and expert guides. Most of what I found is easy enough to find with some simple Google searches but, in the interest of consolidation and to save you the trouble, what follows is a resource dump of what I found the most helpful.

VMware Horizon View Infrastructure Planning, Installation, and Administration

VMware Horizon View 5.3 Official Documentation Page

VMware Horizon View Architecture Planning

VMware Horizon View Installation

VMware Horizon View Administration

VMware Horizon View Security

VMware Horizon View Upgrades

Operating System Optimization for VDI

VMware Horizon View Optimization Guide for Windows 7 and Windows 8

VMware View 5 PCoIP Optimization Guide

VMware Horizon View 5.2 Performance and Best Practices

VMware OS Optimization Tool

My Top 10 VMware View Performance Tips

Turbo-charge View Video Performance

“For desktop VMs using VMXnet3 NICs, you can significantly improve the peak video playback performance of your View desktop by simply setting the following registry setting to the value recommended by Microsoft:”
HKLM\System\CurrentControlSet\Services\Afd\Parameters\FastSendDatagramThreshold to 1500

View Accelerated – 3D Graphics with Horizon View 5.2

“Registry change on the VM – [HKEY_LOCAL_MACHINE\SOFTWARE\VMware, Inc.\VMware SVGA DevTap]
“MaxAppFrameRate”=dword:00000000 – If it does not exist it defaults to 30. Set it to 0 to disable any frame cap.”

How to improve VMware View video performance

VMware KB 2010359

Method 1:
1. Power off the virtual machine using the vSphere Client.
2. Right-click the virtual machine and click Edit Settings.
3. Select the Options tab and under Advanced.
4. Click General.
5. Click Configuration Parameters and click Add Row
6. In the Name field enter mks.poll.headlessRates and in the Value field enter 1000 100 2.
7. Click OK.
8. Power on the virtual machine.

VMware View 5.x – Windows 7 Golden Image

“Video Card: Do not “Auto detect” (see VMware KB 1017380), set to 2 displays and 128 MB video memory”

“Remove he following components (features) from the OS (unless you really need them) and reboot VM:
• Games
• Media Features – Windows DVD Maker
• Media Features – Windows Media Center
• Print and Document Services – Internet Printing Client
• Print and Document Services – Windows Fax and Scan
• Tablet PC components
• Windows Gadget Platform”

Suggested changes to VMware View Optimization Script for Windows 7

-Rick

VMware Horizon View Enable/Disable Teradici APEX PCoIP Offload Card Indicators

Here’s a good tip for testing your Teradici APEX PCoIP Offload cards to make sure they’re working properly in your VDI environment.

On your ESXi hosts, run the following commands to enable and disable an indicator that displays in the upper-left corner of your Virtual Desktops:

Enable indicator:

/opt/teradici/pcoip-ctrl -P “offload_indicator 1”

Disable indicator:

/opt/teradici/pcoip-ctrl -P “offload_indicator 0”

Red = PCoIP display is being offloaded

Blue = PCoIP not being offloaded

I found out about this when I stumbled upon the following VMAdmin UK blog post while researching how to properly configure the APEX cards. The blog has some other good information and tricks that are worth checking out if you’re interested.

http://www.vmadmin.co.uk/vmware/53-view/300-teradiciapexmanage

-Rick

VMware Horizon View Client Login Fails While Using the “Log in as current user” Option

You may run into this issue when attempting to use the “Log in as current user” option in the VMware Horizon View client. This option allows the user to log in to Horizon View using the currently logged on user’s credentials on Windows clients, so it avoids the need to authenticate twice.

To make a long story short, if “Log in as current user” isn’t working then make sure that your Connection Server’s SYSTEM account has “Access this computer from the network” rights on the local system. You can add it via Group Policy if necessary via the Computer Configuration\Policies\Windows Settings\Security Settings\Local Policies\User Rights Assignment\Access this computer from the network policy setting.

For more details, see VMware’s KB on the topic.

-Rick

Create a Windows Security Baseline Group Policy Object with Microsoft Security Compliance Manager (SCM)

Security Compliance Manager (SCM) is a tool that I find extremely useful, especially when preparing for a new Windows OS deployment. And best of all, it’s free!

Included in SCM are Microsoft’s recommended baseline security configurations for just about all of their current Operating Systems, including both desktop and server OSes, as well as some of their flagship applications such as Internet Explorer and Office. You can review and modify these configurations directly in SCM, export the configuration to a GPO Backup folder (as well as to a .cab or .xlsm), and then use that export to create a Group Policy Object to be applied to the appropriate systems in your domain.

I recently used this tool to create a security baseline GPO for Windows Server 2012 R2, so I’ll provide you with the basic steps that I used as a reference.

Please take note that even though the baselines included in SCM are Microsoft’s recommended configurations for security hardening, many of the settings have the potential of having a negative impact on your systems’ performance  and/or your ability to manage them. Because of this I highly recommend taking the time to carefully review and research each setting within a baseline to make sure it will not conflict with any existing policies or procedures in your environment, and making changes as needed.

  1. Download and install Security Compliance Manager 3.0.
  2. In the left pane, expand Microsoft Baselines.
  3. Expand the desired operating system or application version and then select a role. In my case I chose Windows Server 2012 and the basic Member Server Security Compliance role.

SCM1

  1. With the role selected, click on the Duplicate button in the right pane under the Baseline section.

SCM2

  1. Give the duplicate configuration a new name and modify the description if you wish and then click Save.
  2. Your duplicate configuration will show up at the top of the left pane under Custom Baselines, above Microsoft Baselines. Click on it to open the configuration.

SCM3

  1. Take some time to carefully review the configuration settings included in the baseline in the center of the window. You can make changes as needed by clicking on the setting and then modifying the options shown.

SCM4

  1. When you’re finished making any necessary changes, export the configuration by clicking on the GPO Backup (folder) link in the right pane. Be sure to save it somewhere accessible from the system where you manage your domain group policies.

SCM5

  1. Open up the Group Policy Management console and connect to your domain.
  2. Under Forest -> Domains -> MyDomain -> Group Policy Objects, create a new Group Policy Object and name it according to your organization’s GPO naming convention. If you don’t have one, I recommend basing the name off of the baseline configuration you created to distinguish it and make it easier to find, e.g. Windows Server 2012 Security Baseline.
  3. Once created, right-click on the new GPO and click Import Settings…

SCM6

  1. When the Import Settings Wizard appears click Next >.
  2. If you’re attempting to import the configuration settings into an existing GPO rather than with a newly created one, I recommend using the next screen to create a backup of the GPO first. Otherwise, since there are no existing settings to overwrite, click Next > to continue.
  3. Browse to the location of the GPO Backup folder that you exported from SCM earlier and then click Next >.

SCM7

  1. The wizard should detect the baseline in the backup folder and list it in the next window. Click on it and then click Next >.

SCM8

  1. You may get a warning that the backup contains UNC paths. Select Copying them identically from the source and then click Next >.

SCM9

  1. Click Finish to complete the import.

And there you go, you now have a Group Policy Object containing the recommended baseline security settings for your product. From here you can begin linking the GPO to your OUs as needed. I would highly recommend using security filtering and/or a WMI filter to make sure the GPO is only applied to a few select test systems until you’ve gauged how the new settings will impact your environment.

To use my recent experience as an example, I created a security group in Active Directory named Windows Server 2012 GPO Testing, added a single test server to this group, and then added the group to my baseline GPO’s Security Filtering (make sure you also remove Authenticated Users). To be extra careful, I also created a new WMI filter to only return Windows Server 2012 R2 Member Servers and added this to my GPO as well. These help to ensure that my policy will only be applied to servers which are members of my security group, are running Windows Server 2012 R2,  and are not domain controllers, regardless of the OU that I link the policy to in my Active Directory structure.

If you want to create your own custom WMI filter, the process is very simple.

  1. Open the Group Policy Management console and expand Forest -> Domains -> MyDomain -> WMI Filters.
  2. Right-click the WMI Filter container and click New.
  3. Name the new filter appropriately. In my case, I named it Windows Server 2012 Member Server ONLY. Add a description to help others know exactly what your filter does.
  4. Click the Add button to create a new query.
  5. You can leave the namespace as root\CIMv2 and then enter your custom query. To find and return only Windows Server 2012 R2 Member Servers, I used the following query:

select * from Win32_OperatingSystem where Version like “6.3%” and ProductType=”3″

  1. When finished click Save.
  2. You can now use this filter for any GPO that you wish, simply by using the drop-down at the bottom of the Scope tab (same place where you set Security Filtering)

SCM10

For some help creating your own WMI filters, check out the links below.

Create WMI Filters for the GPO

Operating System Version Numbers

-Rick

System Center 2012 R2 Infrastructure Planning (Part 2) (SCCM)

This is part 2 of my System Center 2012 R2 Infrastructure Planning series, which aims to serve as a central resource to aid you in your System Center deployments.

This part focuses on Configuration Manager. Once again, in no particular order:

Configuration Manager

Supported Configurations for Configuration Manager
http://technet.microsoft.com/en-us/library/gg682077.aspx

“Configuration Manager requires several prerequisites to support deploying operating systems. The following prerequisites are required on the site server of each central administration site or primary site before you can install the site or upgrade the site to a new version of Configuration Manager. This requirement applies even when you do not plan to use operating system deployments:

  • For System Center 2012 Configuration Manager with no service pack: Automated Installation Kit (Windows AIK)
  • For System Center 2012 Configuration Manager with service pack 1: Windows Assessment and Deployment Kit 8.0 (Windows ADK)
  • For System Center 2012 R2 Configuration Manager: Windows Assessment and Deployment Kit 8.1″

Planning for Sites and Hierarchies in Configuration Manager
http://technet.microsoft.com/en-us/library/gg712681.aspx

Configuration Manager 2012 Sizing Considerations
http://blogs.msdn.com/b/scstr/archive/2012/05/31/configuration_2d00_manager_2d00_2012_2d00_sizing_2d00_considerations.aspx

Planning for Hardware Configurations for Configuration Manager
http://technet.microsoft.com/en-us/library/hh846235.aspx

“For best performance, use RAID 10 configurations for all data drives and 1Gbps Ethernet network connectivity between site system servers, including the database server.”

“…consider the following general guidelines when you plan for disk space requirements:

  • Each client requires approximately 3 MB of space in the database
  • When planning for the size of the Temp database for a primary site, plan for a size that is 25% to 30% of the site database .mdf file. The actual size can be significantly smaller, or larger, and depends on the performance of the site server and the volume of incoming data over both short and long periods of time.
  • The Temp database size for a central administration site is typically much smaller than that for a primary site.
  • The secondary site database is limited in size to the following:
    • SQL Server 2008 Express: 4 GB
    • SQL Server 2008 R2 Express: 10 GB”

Determine How to Manage Mobile Devices in Configuration Manager
http://technet.microsoft.com/en-us/library/gg682022.aspx

“System Center 2012 Configuration Manager offers limited management for mobile devices when you use the Exchange Server connector for Exchange Active Sync (EAS) capable devices that connect to a server running Exchange Server or Exchange Online.”

System Center 2012 Configuration Manager Best Practices
http://social.technet.microsoft.com/wiki/contents/articles/11215.system-center-2012-configuration-manager-best-practices.aspx

“SQL Collation must be set to “SQL_Latin1_General_CP1_CI_AS”

“Why is it important ? well firstly because it is a setting that most people don’t change (as it’s hidden from view) and secondly it’s set based on your regional settings. When you install SQL Server (which ConfigMgr needs to host it’s database) the SQL Collation is ‘set in stone’ during setup, that’s why knowing what your SQL Collation is and what it should be are important prior to running ConfigMgr setup.”

Determining Whether to Extend the Active Directory Schema for Configuration Manager
http://technet.microsoft.com/en-us/library/gg712272.aspx

Install and Configure Site System Roles for Configuration Manager
http://technet.microsoft.com/en-us/library/hh272770.aspx

About the Asset Intelligence Synchronization Point
http://technet.microsoft.com/en-us/library/cc161864.aspx

Technical Reference for Ports Used in Configuration Manager
http://technet.microsoft.com/en-us/library/hh427328.aspx

SCCM 2012 SP1 SQL Reporting Services on the same server
http://social.technet.microsoft.com/Forums/en-US/5e8d5790-fdce-4c9d-a0c0-f48f9af35b14/sccm-2012-sp1-sql-reporting-services-install-on-same-server?forum=configmanagergeneral

Configure Reporting in Configuration Manager
http://technet.microsoft.com/en-us/library/gg712698.aspx

-Rick

System Center 2012 R2 Infrastructure Planning (Part 1) (SCOM)

So I’ve spent most of this week working on an infrastructure plan and design for System Center 2012 R2 and I wanted to share with you the resources that I found helpful in this endeavor.

The initial plan focuses around Configuration Manager and Operations Manager only, and rather than dump everything into a single post I’ve decided to break them up; so stay tuned for future posts on this topic. Once SCCM and SCOM have been implemented I’ll likely be exploring the other System Center products as well and will continue the series accordingly.

Everyone’s environment is different so I won’t make any specific recommendations, but hopefully this will serve as a useful resource to help you plan your own System Center deployments.

This first post in the System Center 2012 R2 Infrastructure Planning series focuses on Operations Manager and a few SQL Server references that I found handy.

In no particular order….

Operations Manager

Key Concepts
http://technet.microsoft.com/library/hh230741.aspx

System Requirements/Firewall Exceptions
http://technet.microsoft.com/en-us/library/dn249696.aspx

“Operations Manager does not support hosting its databases or SQL Server Reporting Services on a 32-bit edition of SQL Server.”

“SQL Server collation settings for all databases must be one of the following: SQL_Latin1_General_CP1_CI_AS; Latin1_General_100_CI_AS, // EN, IT, DE, PT-BR, NE, PT-PT; French_CI_AS; French_100_CI_AS; Cyrillic_General_CI_AS; Chinese_PRC_CI_AS; Chinese_Simplified_Pinyin_100_CI_AS, // CN simplified; Chinese_Traditional_Stroke_Count_100_CI_AS, // CN traditional, CN-HKJapanese;_CI_AS; Japanese_XJIS_100_CI_AS; Traditional_Spanish_CI_AS; Modern_Spanish_100_CI_AS; or Latin1_General_CI_AS; Cyrillic_General_100_CI_AS, // RU; Korean_100_CI_AS; Czech_100_CI_AS; Hungarian_100_CI_AS; olish_100_CI_AS; and Finnish_Swedish_100_CI_AS. No other collation settings are supported.”

“If you plan to use the Network Monitoring features of System Center 2012 R2 Operations Manager, you should move the tempdb database to a separate disk that has multiple spindles.”

Distributed Deployment of Operations Manager
http://technet.microsoft.com/en-us/library/hh298610.aspx

Single Server Deployment of Operations Manager
http://technet.microsoft.com/en-us/library/hh298612.aspx

Considerations for High Availability and Disaster Recovery
http://technet.microsoft.com/en-us/library/hh920812.aspx

“You should always have two management servers in ANY environment. A second management server allows for failover and easy restore, and a second management server can take on the load if one fails. All management servers are members of the All Management Servers Resource pool, which balances the monitoring load of your management group as new management servers are added, and provides automatic failover for monitoring. The impact of failure of a management server in a distributed environment is minimized, but it increases the workload on additional management servers in the management group until the failed management server is restored.”

Security Considerations
http://technet.microsoft.com/en-us/library/hh487288.aspx

System & Database Sizing Helper Tool
http://blogs.technet.com/b/momteam/archive/2012/04/02/operations-manager-2012-sizing-helper-tool.aspx

Audit Collection (ACS) Database Size Calculator
https://blogs.technet.com/b/momteam/archive/2008/07/02/audit-collection-acs-database-and-disk-sizing-calculator-for-opsmgr-2007.aspx

Operations Manager and VMware…
http://social.technet.microsoft.com/Forums/systemcenter/en-US/51b2d19d-b783-4828-9b7d-bc59f4a44c2b/operations-manager-2012-and-vmware-virtual-hosts-supported-for-installation?forum=operationsmanagerdeployment

“Microsoft supports running all System Center 2012 – Operations Manager server features in any physical or virtual environment that meets the minimum requirements that are stated in this document. However, for performance reasons, we recommend that you store the operational database and data warehouse database on a directly attached physical hard drive, and not on a virtual disk. Specifically, virtual computers that are running any Operations Manager server feature must not use any functionality that does not immediately commit all activity on the virtual computer to the virtual hard drive. This includes making use of point-in-time snapshots and writing changes to a temporary virtual hard drive. This is true for every virtualization technology that is used with Operations Manager.”

Understanding and modifying Data Warehouse Retention and Grooming
http://blogs.technet.com/b/kevinholman/archive/2010/01/05/understanding-and-modifying-data-warehouse-retention-and-grooming.aspx

How to Configure Grooming Settings for the Reporting Data Warehouse Database
http://technet.microsoft.com/en-us/library/hh212806.aspx

Database Size Limits…
http://social.technet.microsoft.com/Forums/systemcenter/en-US/22efc287-a77d-4534-8d68-eb1b32d53b3a/database-size-limits-for-operations-manager-and-operations-manager-data-warehouse

Network Bandwidth Utilization for OpsMan 2007
http://blogs.technet.com/b/momteam/archive/2007/10/22/network-bandwidth-utilization-for-the-various-opsmgr-2007-roles.aspx

Audit Collection Services Capacity Planning
http://technet.microsoft.com/en-us/library/hh212872.aspx

Collecting Security Events Using Audit Collection Services in Operations Manager
http://technet.microsoft.com/en-us/library/hh212908.aspx

How to Install the Operations Manager Reporting Server
http://technet.microsoft.com/en-us/library/hh298611.aspx

Management Packs Installed with Operations Manager
http://technet.microsoft.com/en-us/library/hh212701.aspx

ACS Configuration Help
http://social.technet.microsoft.com/Forums/systemcenter/en-US/e8682287-a4ac-4ebe-942a-b4a71c894a94/scomacs-configuration?forum=operationsmanagergeneral

SQL Server

Hardware and Software Requirements for Installing SQL Server
http://msdn.microsoft.com/en-us/library/ms143506.aspx

Managing SQL Server Workloads with Resource Governor
http://technet.microsoft.com/en-us/library/bb933866(v=sql.105).aspx

Considerations for Installing Reporting Services
http://msdn.microsoft.com/en-us/library/ms143736(v=sql.100).aspx

-Rick

Outlook Prompts for Autodiscover Credentials Mid-Session

This one is for a very specific and probably uncommon scenario, but it drove me (and everyone else) up a wall and took a ton of man, and Microsoft Support, hours to finally resolve, so hopefully this will save some headaches.

Some background; We were in the middle of a migration to a new Exchange Server that sat on a separate domain from our Windows desktop clients with no trust relationship established (long story). We were successfully able to update our users’ Outlook clients to point to the new server address, and when launched Outlook prompted for authentication credentials to connect. This worked well enough, aside from our users being forced to use different credentials to log into their computers and to access their email, and everything functioned pretty much normally once authenticated. To help streamline the process of opening Outlook by avoiding the login prompt on launch, many of our users took to storing their secondary credentials locally using Windows Credentials Manager.

However, we started getting reports from users who used these cached credentials that they were being frequently prompted while Outlook was open, mid-session, to authenticate with an Autodiscover.domain server address. The Autodiscover address was displayed as being on the same domain as the workstation despite no Exchange Server residing there, and the prompt could be cleared by either hitting cancel or using credentials for the new Exchange Server’s domain. Regardless, the prompt would continue to reappear every few hours.

We were banging our heads against the wall for several days, trying everything we could think of and any suggestion we could find on the web, including wild-carding both domain addresses in Credentials Manager (for example *.contoso.com, to borrow from Microsoft), but absolutely nothing worked. Finally we stumbled upon the somewhat counter-intuitive solution with Microsoft Support’s help.

To prevent the Autodiscover prompt from appearing, we effectively had to bypass the use of cached credentials by forcing the prompt for logon credentials on launch via a setting in the user’s Outlook profile. Instructions for doing this are below.
 
In Outlook 2007:
1. Click Tools -> Account Settings
2. On the E-mail tab, highlight the Microsoft Exchange account and click on the Change button
3. Click the More Settings button
4. Click the Security tab
5. Check the box next to Always prompt for logon credentials
6. Click Apply and then OK
7. Click Next and then Finish

In Outlook 2010:
1. Click File -> Info -> Account Settings
2. On the E-mail tab, highlight the Microsoft Exchange account and click on the Change button
3. Click the More Settings button
4. Click the Security tab
5. Check the box next to Always prompt for logon credentials
6. Click Apply and then OK.
7. Click Next and then Finish.

So, again, this had the effect of forcing the prompt for credentials to connect to the Exchange Server when Outlook is first run, even if credentials are cached for that address. Still an inconvenience but, since most of our users would open Outlook and leave it running in the background, many found a single prompt at first was preferable to periodic prompts throughout the day.

-Rick

WSUS Best Practices

Here’s a very good blog post I came across with some WSUS best practices for anyone else looking to implement WSUS for the first time or review your patching strategies.

http://microsoftguru.com.au/2013/07/19/windows-server-patching-best-practices/

Some highlights:

Consultants should take time to test the patches in a non-production environment prior to being deployed to production. This will help to gauge the impact of such changes. Ideally you will have the following patching groups:

1. UAT (UAT1, UAT2, etc)

2. Test Environment (Test1, Test2, etc)

3. Development Environment (Dev1, Dev2 etc)

4. Production (Prod1, Prod2, etc)

If you have clustered environment like SQL, Exchange and SharePoint then create Prod1, prod2 group and place each node on each group. “

System administrators should maintain a log, written or electronic, of all changes to the operating environment, to include hardware, system security software, operating system, and applications. Prior to any changes being implemented on a system, the system administrator should receive approval of stakeholders.”

A scheduled maintenance window must be agreed with business so that application outage and server reboot can maintain a respectable Service Level Agreement (SLA). If you have a large infrastructure with thousands of servers and many regions working round the clock then you must consider application dependencies. A patching schedule can be considered in between every Friday of every month at 6:00 P.M. Friday to 6:00 A.M Monday. Setup maintenance window in system center or deadline for WSUS to make sure patches are applied when you want instead of when patch is available. In this way you will have a complete control over change windows approved by change advisory board (CAB). Do not allow end users to update patches on their client machine according to their wishes and happiness! then user will never install any patch. “

Microsoft strongly recommends that you create the following backups before you install an update rollup, service pack and patch on Exchange and SQL:

  • A full backup of all databases on the server.
  • A full backup of transaction log and log backup
  • A system state backup of the server.
  • A snapshot of virtualized exchange server. Delete snapshot after successful patching and updating. “

Here are some other useful resources for WSUS:

WSUS Role Installation Fails on Windows Server 2012 R2

I was attempting to add the WSUS role on a Windows Server 2012 R2 system earlier this week and I ran into this error during the installation.

“The request to add or remove features on the specified server failed. the operation cannot be completed because the server that you specified requires a restart.”

After several subsequent reboots and attempts to add the role I continued receiving this same error, so I started doing some research. I got several hits and this appears to be a fairly common issue with Server 2012, specifically when choosing to use the Windows Internal Database (WID).

The issue is that WID relies on the NT SERVICE\MSSQL$MICROSOFT##WID account to start the service, and this account must have Log on as a service rights on the system.

Generally I don’t think this would be an issue, and if it is you should be able to simply use the local Group Policy editor (gpedit.msc) or a domain GPO to grant this account Log on as a service rights by using the following policy setting:

Computer Configuration -> Policies -> Windows Settings -> Security Settings -> Local Policies -> User Rights Assignment -> Log on as a service

Open the Log on as a service setting, check the box to Define these policy settings, and then add NT SERVICE\MSSQL$MICROSOFT##WID

However, to maintain security compliance as we move to Server 2012 R2, a domain Group Policy was recently created using Microsoft’s Security Compliance Manager (I may write a separate post on SCM later) and applied. As part of the Windows Server 2012 Security Baseline, Microsoft adds NT SERVICE\ALL SERVICES to the Deny log on as a service security policy. So because ALL SERVICES is being denied, even after adding MSSQL$MICROSOFT##WID to the Allow log on as a service policy locally, the WSUS role still failed to install for me.

To finally get the WSUS role installation to complete, I had to modify this new domain Group Policy and remove the ALL SERVICES account from the Deny log on as a service policy first, then add NT SERVICE\MSSQL$MICROSOFT##WID to the Log on as a service policy setting locally on the server.

For some more details and a few screenshots of the initial troubleshooting and resolution to this issue, check out one of the blog posts that I came across during my research. WSUS Role failed on Windows server 2012 with error…

You may notice that the blog I linked as well as many others I came across recommend just adding NT SERVICE\ALL SERVICES to the Log on as a service setting. Since Microsoft’s baseline security settings for Server 212 specifically DENIES log on as a service access to ALL SERVICES, I opted to only add the account explicitly required by WID to avoid opening up any more security vulnerabilities than I had to.

I’ll need to go back and take another look at our Server 2012 security policy to see if there’s another way around this while still denying log on as a service access to unneeded service accounts. I guess the easiest (and maybe best) thing to do would be to avoid using WID all-together and instead use a SQL Server instance.

That also leads me to another interesting topic that I’ll probably write another post about down the road. I’m installing WSUS with WID for now for the sake of time, but I may migrate it over to a SQL Server once I can. For a preview of that job, check out Migrate Windows Internal Database to SQL Server.

Links in this post:

 

-Rick