Automating Site Updates with Server 2008, NLB.exe, and PowerShell

Apologies First…

Sorry for the hiatus, I recently left my old employer for another opportunity, and have been busy adjusting to the whole environment and culture of my new IT shop. But good things come to those who wait!! I have recently been working on solving a problem that has been plauging our Systems Administration team for a while now…

Until now!!!

NOTE TO 2008 R2 Admins:

The following information is useless to you. I would google information on “2008 R2 Powershell Network Load Balancing cmdlets” or something similar, because there is a MUCH easier way to accomplish what I’m trying to do here. If you still aren’t convinced read the next paragraph and decide whether or not you wish to continue.

The Problem: 2008 is not as cool as R2

Say you have a Windows Server 2008 SP(?) NLB 2-Node Cluster running IIS 7 that serves up a simple web site. Your processes for updating the site are something similar to this (note that logging and alerting via e-mail by your own standards are implied throughout):

  • Step 1: Stop cluster activity on Node 1
  • Step 2: Verify cluster still serves pages successfully
  • Step 3: Perform Scheduled Maintenance job
  • Step 4: Restart cluster activity on Node 1
  • Step 5: Verify once again cluster still serves pages successfully
  • Step 6: Stop cluster activity on Node 2
  • Step 7: Repeat steps 1-5 with Node 2 (in place of Node 1)

This can take a good amount of time doing this manually. Not because it is difficult, but just due to the fact of having to log into multiple servers, use multiple interfaces within those servers, and God forbid we mention needing to log or report on the steps and their levels of success, failure, etc…

So then someone says “hey I bet we can automate this” and you go “Yessir! Shouldn’t be too difficult!!!”, and then you start researching whether or not this is,indeed, possible…

Well if you found yourself doing this like I myself did, then you have probably come to the following 2 conclusions:

1: PowerShell can do anything…but whether it’s easy to accomplish or not is an entirely different matter

2: You hate Server 2008 right now because it doesn’t contain the powershell module for NLB management that 2008 R2 does, which makes this stuff child’s play!

UAC: The bane of all automation’s existence



So…the root of the issue here is that due to the need of elevated priviledges, UAC (User Account Control) will prompt someone who is not THE local administrator on the box (note: THIS MEANS EVEN USERS IN THE LOCAL ADMINISTRATORS GROUP EXCLUDING SPECIFICALLY THE “Administrator” ACCOUNT!) before executing an application or service with as much authority as NLB.exe. NLB.exe is a utility that allows you to work with cluster nodes through a command line. We have to use this because our cluster is of the Windows Server 2008 NON-R2 flavor, therefore is lacking the ability to use the built in cmdlets for NLB Management within PowerShell natively. Things are fine if you are in an administrative command prompt using NLB.exe, but if you want to automate the process then UAC begins to bleed all over your beautiful design….oh what to do!???

Pretty Pictures…

Ok so I know I ramble alot so maybe I can convey the overall scenario with how I picture it in my head. See below:



Well, if you think life is over and your stuck with your 2008 server without an option to upgrade like I was, fear not! I’m writing this post to tell you there is still indeed hope, and I will show you how I’ve managed to accomplish the process outlined above in an “semi-automated” fashion. That’s right…”push a button”, so to speak, and be done!

The Solution(s)

My solution to this problem had no restrictions on “what sat where” as far as scripts, languages, etc, so I devised a solution with the following components:

1: A Powershell script that uses “NLB.exe” to manipulate the cluster and perform the update duties as outlined earlier.

Note: The PoSH script above is set as a scheduled task on BOTH nodes, each running under the “SYSTEM” account (with the option to “run the task whenever we feel like it” is checked).

2: A simple batch file that is setup as a second scheduled task on Node 1.

SPECIAL NOTE:

This will run a simple “schtasks /run /S ‘{ Node2_Name }’ /NT ”” command that will execute the scheduled task on Node 2 once the powershell script finishes. We will use the “runas” account for this job as a domain user that has the appropriate rights on BOTH servers in order for this to work.

The local “SYSTEM” account works great for step 1, and helps us get past the UAC issue when running NLB.exe, but when we want to hand off the job to node 2, you will get “access denied” if the user is not available and authorized to execute remote and local tasks on both systems.

The PowerShell Script for job “NLB_Website_Maintenance” on Node 1:

The script below is from my dev box before I started implementing my special maintenance requirements. Not every scenario may work with my method, but this was for a simple process of creating backup folders, archiving site data, copying the new site, restarting the app pools, etc.

My point here is the code below is a framework of my solution, you should be able to mold the code to fit your needs rather easily (Hopefully! :P).

#########################################################
#	NLB Automation Template Source Code
#	Author: Michael Adams - https://willcode4foodblog.wordpress.com
#	Date:   7/14/2012   
#
#	Support: None
#	Agreement:
#
#    By using this code you agree to assume all responsibilities
#    for the consequences of doing so.  I am not to be held responsible
#    for your systems should issues arise.  Always test automated
#    processes in an isolated test environment prior to implementing
#    into a production-level environment.
#
#########################################################
#
# FUNCTION NAME: "START-NEWPROCESS"
# This is a modified version of the function used
# by Jeff Wouters. His post where I read about it is:
# ------------------------------------------------------
# http://jeffwouters.nl/index.php/2011/11/having-some-fun-with-uac-and-powershell/
# ------------------------------------------------------
function Start-NewProcess(
        [string]$pfile,
        [string]$arguments
    ){
    $p = New-Object System.Diagnostics.Process;
    $p.StartInfo.UseShellExecute = $false;
    $p.StartInfo.Verb = "runas"
    $p.StartInfo.RedirectStandardOutput = $true;
    $p.StartInfo.FileName = $pfile;
    $p.StartInfo.Arguments = $arguments
    [void]$p.Start();
    $p.WaitForExit();
    $p.StandardOutput.ReadToEnd();
}
# FUNCTION NAME: "TEST-SITE"
# This is a modified version of the function used
# by _____________. His post where I read about it is:
# ------------------------------------------------------
# http://
# ------------------------------------------------------
function Test-Site{
param(
[string]$url=$(throw "You must specify a URL to check!")
)
	# Use your favorite URL Query function to return
	# the status of the specified URLs in $arrURLs

	return $status;
}



#############################
# Begin Processing
#############################
# Define URLS we want to test while running maintenance
$arrURLs = @("http://www.contoso.com/default.aspx","http://www.contoso.com/Site1/default.aspx","http://www.contoso.com/Site2/default.aspx")

# Run The DrainStop Command.  I'm outputing to a file just for demo purposes,
# tailor your event handling/logging to your specific requirmenets
Start-NewProcess "nlb.exe" 'drainstop pubwebtest:pubwebtest01' | out-file "C:\Users\user_account\desktop\drainstopout.txt"

# For each URL, make sure it's still up after we take the node offline
$arrURLs | % {

	$currentURL = $_
	$testResult = Test-Site $currentURL
	if($testResult -notmatch "OK")
	{
		# TODO: Add e-mail alert/error handling here
		exit;
	}

}

#############################
# TODO: Add Maintenance Processes - Event Logging - Alert Handling here
#############################

# Once the maintenance is complete, restart the node
Start-NewProcess "nlb.exe" 'start pubwebtest:pubwebtest01' | out-file "C:\Users\user_account\desktop\restartout.txt"

# For each URL, make sure it's still up after we put the node back online
$arrURLs | % {

	$currentURL = $_
	$testResult = Test-Site $currentURL
	if($testResult -notmatch "OK")
	{
		# TODO: Add e-mail alert/error handling here
		exit;
	}

}

# If we haven't exited yet then all must be ok still so let's kick-off the "hand-off" task
# The execution account for "HandOffToNode2" needs to have rights on both boxes, but since
# we're still kicking off a job locally, we can still use the "SYSTEM" account we are currently
# running this script under.
& schtasks /run /S 'Node1' /NT 'HandOffToNode2'


The Batch File For Job “HandOffToNode2” on Node 1:

REM This will execute under a user with appropriate rights on both boxes
schtasks /run /S 'Node2' /NT 'NLB_Website_Maintenance'

The PowerShell Script for job “NLB_Website_Maintenance” on Node 2:

This script is a mirror copy of the script on Node 1 with the following changes:

####################
#  CHANGE 1:
####################
# Change the node name on this line to %ClusterName%:%Node2Name% so for example: "pubwebtest:pubwebtest02" 
Start-NewProcess "nlb.exe" 'drainstop pubwebtest:pubwebtest01' | out-file "C:\Users\user_account\desktop\drainstopout.txt"

####################
#  CHANGE 2:
####################
# Change the node name on this line to %ClusterName%:%Node2Name% so for example: "pubwebtest:pubwebtest02" 
Start-NewProcess "nlb.exe" 'start pubwebtest:pubwebtest01' | out-file "C:\Users\user_account\desktop\restartout.txt"

####################
#  CHANGE 3:
####################
# Either comment this line out at the end or replace it with an e-mail alert to let you 
# know the process has been completed successfully
& schtasks /run /S 'Node1' /NT 'HandOffToNode2'


Conclusions

There’s many ways you can implement this: Using the task scheduler to schedule the jobs so that the dev team can stage update data the day before the morning maintenance window, or using a product like Control-M or System Center Orchestrator to manage the execution of them. While this is by no means the simplest solution to this problem, sometimes you gotta do what you can with what you got…and if you ain’t got Server 2008 R2, handling NLB automation is cumbersome at best. I just hope this info will help people devise even more efficient solutions to this issue!!!

Sorry for not including the complete source…it contained proprietary information so I had to write this post from memory. So if there’s anything I didn’t touch on or that needs clarification please comment and I’ll try to respond in a timely manner!

Till Next Time…

Advertisements

One comment on “Automating Site Updates with Server 2008, NLB.exe, and PowerShell

  1. kiquenet says:

    It’s great mister! thanks.

    More articles about this site updates safely using NLB?

    other reference: (http://www.codeproject.com/Articles/198170/Safely-deploying-changes-to-production-servers)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s