Macaw Solutions Factory

How to setup BizTalk 2009 BAM Portal on a 64 Bit OS in IIS 7

Tue, 17 Nov 2009 10:40 GMT

I just finished setting up BizTalk 2009 (Developer Edition) on a W2008R2 x64 machine with the following components:

Initially, the BizTalk Server Configuration utility failed to install the BAM portal. The error log reported that IIS was in 64 bit mode where 32 bit mode was required.

MSDN guidance on configuring the BAM Portal said that in IIS 6 you should execute this script:

cscript c:\inetpub\adminscripts\adsutil.vbs SET W3SVC/AppPools/Enable32bitAppOnWin64 1

However, it also mentioned this:

However, the MSDN guidance did not specify what you should do if you use IIS 7.

After I found out that in IIS 7 you can specify per application pool whether you want to enable 32 bits processes, I found a clean and easy way to get the BAM Portal to install correctly:

In IIS Manager, select Set Application Pool Defaults:
Set the property “Enable 32-Bit Applications” to True:
Run the BizTalk Server Configuration utility and configure the BAM Portal.
If you want, you can now change the Application Pool Defaults back to what you prefer.

NJoy the green checkmarks.

Detect 32 or 64 bits Windows – regardless of WoW64 – with the PowerShell OSArchitecture function

Mon, 02 Nov 2009 10:09 GMT

In 64-bits versions of Windows Operating Systems, a subsystem called Windows on Windows 64 (WoW64) enables you to run 32 bits applications. Even to date, many Windows applications only exist in a 32 bits version – not the least of which is Microsoft's own Visual Studio(!)

The purpose of WoW64 is to hide many of the structural differences between 32 bits and 64 bits Windows for 32 bits applications. This is a good thing. Most of the time, anyway…

However, when your PowerShell script runs in a 32 bits process on a 64 bits OS, and you want to integrate with 64-bits applications (e.g. you want to integrate SharePoint or SQL Server within Visual Studio), WoW64 can get in the way.

E.g., WoW64 hides the 64 bit registry and changes the environment variables for processor architecture and for the program files folder – in short, WoW64 hides that you are running on a 64 bits OS.

To detect a 64 bits OS from PowerShell on Vista, W2008 and later Windows versions, we could use WMI:

(Get-WMIObject win32_operatingsystem).OSArchitecture

However, this is not supported on W2003.

Fortunately, Microsoft provided an alternative, which is supported on all Windows versions that have Wow64: the IsWow64Process function. Since this is an unmanaged API call to Kernel32.dll, we need to do some extra work to call this in PowerShell.

We can write a simple managed C# wrapper around IsWow64Process:

using System;
using System.Runtime.InteropServices;
 
public class Kernel32
{
    [DllImport("kernel32.dll", SetLastError = true, CallingConvention = CallingConvention.Winapi)]
    [return: MarshalAs(UnmanagedType.Bool)]
    public static extern bool IsWow64Process([In] IntPtr hProcess, [Out] out bool lpSystemInfo);
 
    // This overload returns True if the current process is running on Wow, False otherwise
    public bool CurrentProcessIsWow64()
    {
        bool retVal;
        if (!(IsWow64Process(System.Diagnostics.Process.GetCurrentProcess().Handle, out retVal))) { throw new Exception("IsWow64Process() failed"); }
        return retVal;
    }
}

When we use the PowerShell InvokeCSharp function, we can put this C# inline in our PowerShell script:

Function global:CurrentProcessIsWOW64
{
    # Use some inline C# code to call the unmanaged IsWow64Process() function in Kernel32.dll and return its result:
    $code = @"
        using System;
        using System.Runtime.InteropServices;
 
        public class Kernel32
        {
            [DllImport("kernel32.dll", SetLastError = true, CallingConvention = CallingConvention.Winapi)]
            [return: MarshalAs(UnmanagedType.Bool)]
            public static extern bool IsWow64Process([In] IntPtr hProcess, [Out] out bool lpSystemInfo);
 
            // This overload returns True if the current process is running on Wow, False otherwise
            public bool CurrentProcessIsWow64()
            {
                bool retVal;
                if (!(IsWow64Process(System.Diagnostics.Process.GetCurrentProcess().Handle, out retVal))) { throw new Exception("IsWow64Process() failed"); }
                return retVal;
            }
        }
"@
    InvokeCSharp -code $code -class 'Kernel32' -method 'CurrentProcessIsWow64'
}
 
Function global:ProcessArchitecture
{
    switch ([System.IntPtr]::Size)
    {
        4 { 32 }
        8 { 64 }
        default { throw "Unknown Process Architecture: $([System.IntPtr]::Size * 8) bits" }
    }
}
 
Function global:OSArchitecture
{
    if (((ProcessArchitecture) -eq 32) -and (CurrentProcessIsWOW64)) { 64 } else { ProcessArchitecture }
    # Note that on Vista, W2008 and later Windows versions, we could use (Get-WMIObject win32_operatingsystem).OSArchitecture.
    # We don't use that because we also support W2003
}

So there you have it, some simple PowerShell functions to detect the process architecture and the OS architecture, regardless of whether or not you are running under WoW64.

P.S. I also added these functions to the global functions in the Macaw Solutions Factory, so they can be leveraged by anyone who wants to extend the Factory.

Call inline C# from PowerShell with InvokeCSharp

Tue, 27 Oct 2009 06:58 GMT

Out of the box PowerShell 1.0 makes it very easy to call any managed code in an assembly. However, one of the strong points of a scripted language such as PowerShell is that you can debug and extend it in any environment without the need for development tools or additional source code. This is especially valuable in scripts that integrate heavily with production environments, e.g. with automated deployment scripts such as we use in the Macaw Solutions Factory.

Ideally you would want the ability to call any C# source code from PowerShell, either inline or as a separate source file, effectively adding the maintainability benefits of a scripted language to C#. Based on a library created by Lee Holmes I created an improved PowerShell 1.0 function called InvokeCSharp that does just this.

InvokeCSharp is completely self-contained and can be used in any PowerShell script.

Usage

Here are some simple examples on how to use InvokeCSharp:

Let's say we have a file MyClass.cs:

using System;
 
public class MyClass
{
    public string AString;
 
    public MyClass(string aString)
    {
        AString = aString;
    }
}

Then we can invoke it like this in PowerShell:

1
2
3

# Create a class instance from a .cs file, passing a parameter to the class constructor:
# Note that the class name can be omitted here, since it is implied from the .cs file name
$myClassInstance = InvokeCSharp -file '.\MyClass.cs' -parameters 'A text'

Or we can put some C# code inline in the PowerShell script itself, and invoke it:

# Call some simple inline code. Note that you could also wrap the code in a namespace, 
# add using statements and reference other assemblies.
$code = @"
	public class MyClass2
	{
		public static int MyStaticMethod(int a, int b)
		{
			return a * b;
		}
	}
"@
$result = InvokeCSharp -code $code -class 'MyClass2' -method 'MyStaticMethod' -parameters 3, 7 # Will return 21

Although these are very simple examples, all the rich goodness of PowerShell – .NET integration is available. Passing any type of parameters back and forth is simple and completely transparent. All C# features can be used; e.g. you can use the C# DllImport attribute to call unmanaged API's when COM interop is not available or not preferred.

After initial compilation the performance is equivalent to calling compiled assemblies from PowerShell.

Extending the Macaw Solutions Factory PowerShell scripts with inline C#

I added InvokeCSharp to the global functions in the Macaw Solutions Factory. This makes it possible to simply extend or modify any Factory script – including the scripts that deploy a MAST Product from a package to (production) machines.

For anyone who is more adept with C# than with PowerShell this will make it easier to extend the Factory scripts. This also enables leveraging existing C# libraries from PowerShell while keeping the entire solution maintainable as a script.

Improvements

The improvements I added to InvokeCSharp when compared to the original version are:

The C# code you specify is completely self-contained; no more C# wrapper code where your code is pasted into. This improves readability and maintainability of inline C# code.
Simpler parameter passing; no more wrapper code needed for parameters and return values.
Automatically un-wraps parameters that PowerShell wraps in a PSObject. This eliminates the need to explicitly cast parameters to make them strongly typed on each call to InvokeCSharp, and to always wrap them in @().
Figuring this one out was interesting; it required diving in the innards of the PowerShell Extended Type System (ETS) which gets in the way when you work with .NET reflection classes from PowerShell. PowerShell tries to make the ETS transparent to the PowerShell user, which in general is a good thing but in this case meant it was hard to see what was happening.
Adds support for calling static methods, including overloaded methods, without creating a class instance.
Adds support for creating a class instance and providing constructor parameters.
Caches compiled assemblies instead of class instances. This improves performance because it eliminates duplicate compilation when the same C# code is called more than once. In addition to assembly caching you can of course also cache the object instances using any of the standard PowerShell mechanisms; however this is better handled outside the InvokeCSharp function.
Support for inline C# and separate C# source files; the latter allows you to easily edit the C# in Visual Studio, e.g. to use IntelliSense and compile the source to catch compile time errors.

PowerShell Source

So here are the goods – the complete PowerShell source for InvokeCSharp:

#< #
#.Synopsis
# Create a class instance and/or invoke a method on C# source code.
# For details, see:
# http://vincenth.net/blog/archive/2009/10/27/call-inline-c-from-powershell-with-invokecsharp.aspx
#.Description
# This function compiles the provided C# source code (only if necessary, compiled
# assemblies are cached). The code can be supplied either inline in a string or
# in a source file. Relative source filename paths are interpreted
# relative to the parent folder of the calling script.
# This enables simple organization and reference of related .ps1 and .cs files.
#
# If a method name is specified, that method is invoked with any supplied
# parameters and the function then returns the result of the method.
# Both static and nonstatic methods are supported, including overloaded
# methods.
#
# If no method name is specified, an instance of the specified class is
# created, passing any supplied parameters to the class constructor, and the
# function then returns the class instance. You can then program against the
# class instance in standard PowerShell fashion.
#
# If necessary for compilation, you can specify names of any referenced
# assemblies.
#.Parameter code
# The complete C# source code - effectively this is an inline C# source file.
# Note that it is possible but not necessary to declare classes within a
# namespace.
# Either the code parameter or the file parameter must be specified.
#.Parameter file
# Absolute or relative path to a C# source file. Relative paths (beginning
# with '.\' or '..\') are interpreted relative to the folder that contains
# the calling script (i.e. the script that calls this function).
# Either the code parameter or the file parameter must be specified.
#.Parameter class
# The name of the class to be instantiated / that contains the method to
# be invoked. Include the namespace in which the class is declared, if any.
# If you specify the -file parameter and the class name is identical to the
# file name, you can omit the -class parameter.
#.Parameter method
# Either the name of the method to be invoked, '()' to invoke a
# constructor to create and resturn a class instance, or '' to return
# the cached assembly. The last option allows you to precompile assemblies
# without calling a method or creating a class instance.
#
# If the specified method is not static, an instance of the class will
# be created on the fly.
#.Parameter parameters
# To pass parameters to the constructor or method, specify them as an
# array of objects (e.g. with the comma operator or @() ).
#.Parameter reference
# If necessary, specify an array of assembly filenames to be added as
# assembly references when the source code is compiled.
#.Parameter forceCompile
# Specify this switch to force recompilation. This is useful if you specify
# the file parameter and the file contents has changed (when specifying a
# source file name, the cache key is the file name only - changes to the file
# do not automatically cause cache invalidation).
#.Returns
# Either the cached assembly, a class instance or a method result (depending
# on the value specified for the method parameter).
#.Example
# # Create a class instance from a .cs file, passing a parameter to the class constructor:
# # Note that the class name can be omitted here, since it is implied from the .cs file name
# $myClassInstance = InvokeCSharp -file '.\MyClass.cs' -parameters 'A text'
#
# # Call some simple inline code. Note that you could also wrap the code in a namespace,
# # add using statements and reference other assemblies.
# $code = @"
# public class MyClass2
# {
# public static int MyStaticMethod(int a, int b)
# {
# return a * b;
# }
# }
# "@
# $result = InvokeCSharp -code $code -class 'MyClass2' -method 'MyStaticMethod' -parameters 3, 7 # Will return 21
##>
function global:InvokeCSharp
{
param(
[string] $code = '',
[string] $file = '',
[string] $class = '',
[string] $method = '()',
[Object[]] $parameters = $null,
[string[]] $reference = @(),
[switch] $forceCompile
)

# Stores a cache of generated assemblies. If this library is dot-sourced
# from a script, these objects go away when the script exits.
if(-not (Test-Path Variable:\macaw.solutionsfactory.assemblycache))
{
${GLOBAL:macaw.solutionsfactory.assemblycache} = @{}
}

if (($code -eq '') -and ($file -eq '')) { throw 'Neither code nor file are specified. Specify either one or the other.' }

# If a source file was specified, see if it was already loaded, compiled and cached:
if ($file -ne '')
{
if ($code -ne '') { throw 'Both code and file are specified. Specify either one or the other.' }

# We interpret the current directory as the directory containing the calling script, instead of the currect directory of the current process.
if ($file.StartsWith('.'))
{
$callingScriptFolder = Split-Path -path ((Get-Variable MyInvocation -Scope 1).Value).MyCommand.Path -Parent
$file = Join-Path -Path $callingScriptFolder -ChildPath $file
}

# If no class name is specified, we assume by convention that the file name is equal to the class name.
if ($class -eq '') { $class = [System.IO.Path]::GetFileNameWithoutExtension($file) }

# Use the real full path as the cache key:
$file = [System.IO.Path]::GetFullPath((Convert-Path -path $file))
$cacheKey = $file
}
else
{
# See if the code has already been compiled and cached
$cacheKey = $code
}
if ($class -eq '') { throw 'Required parameter missing: class' }

# See if the code must be (re)compiled:
$cachedAssembly = ${macaw.solutionsfactory.assemblycache}[$cacheKey

Factory Feature – Create Component
Tue, 15 Sep 2009 09:56 GMT
Create Component is a new feature that was added to the Macaw Solutions Factory in version 2.1. \r\n
With Create Component allows you can quickly add instances of the sample template components to your existing solution. You simply select the component type and specify the name of the new component; all other actions are automated. The component is created automatically in the correct project and folder, its namespace is set accordingly, and if the correct project is not present in the solution it is added on the fly with all correct namespaces, project settings etc.\r\n
Note: there are several series of Macaw Solutions Factory posts, on multiple blogs.
You can read all posts on the Macaw Solutions Factory site, or use this feed.\r\n
Basically, the Create Components feature allows you to think and work quickly and the logical level when creating components. You do not need to think about which project, folder, project items or settings are needed to implement the component. Instead, all that stuff is generated for you; you can start directly with implementing the component\'s internals.\r\n
The Create Components feature only works on MSBuild-based projects, so this feature is not available in the Bts2006 and the Sql2005 SS*S solutions.\r\n
This is how Create Component works in detail:\r\n

You start by creating a baseline solution with the MAST Solution Builder, by selecting an action in the Factory Guide under \'Create Functional Area\'.

All you need to do now is type the name of the new Functional Area (VS solution), and select which components types (VS projects) the Functional Area should contain initially.
Note that you only need to select the component types (projects) of which you know for sure that you will need them; you can add component types to an existing solution any time you want.
When you click Create, the new solution is built and all namespaces, project settings etc are set to the correct values.
After you close the solution builder window, you will be asked whether you want to remove the sample components and sample code from the new solution:

And whether you want to open the new solution:

After you make these choices, the Product.config file will be opened in VS instance where you selected the \'Create Functional Area\' action, and the new Functional Area will be added automatically to the first non-Templates machine role:

If you have more than one machine role – other than Templates - in your product, you may want to move the Functional Area to a different machine role now.
\r\n
When the new solution is opened in VS you will see a node \'Create Component\' in the Factory Guide, with a lot of actions in it:

In any non-Templates solution these Create Component actions are generated on the fly, based on the Sample components in the Templates Functional Area for the current Product Dependency.
E.g. in the DotNet3Templates.sln you can easily recognize the sample components that correspond to the actions in above screenshot:

For each project item in a template solution whose name starts with \'Sample\', an action is generated under Create Component. The name of the action specifies both the project name and the component name (without \'Sample\').
If a template project does not contain any sample components, an action is generated to only add the project to your solution (e.g. \'New OperationalManagement\' in the above screenshot).
You can easily modify the template solutions to change, add or remove sample components. You need to select the \”Refresh \'Create Component\' actions\” action to regenerate the actions after you make this type of changes.
\r\n
When you execute a Create Component action, you are prompted for the component name and everything else is done automatically:

When you select OK, the component is added and it is opened in the editor:

If necessary, the proper project for the new component will be created on the fly. You can see whether this will happen beforehand in the name prompt dialog:

When you select OK, the MAST solution builder window will open and the new project will be created:

As you can see in the log window the solution builder performs a lot of actions automatically, e.g. setting namespaces and project properties, and adding the project target to the FxCop project file for this solution.
When you close the MAST solution builder window, the new component will be added and it is opened in the editor:

Macaw Vision on SharePoint Release Management
Tue, 28 Apr 2009 04:57 GMT
The current version of SharePoint (Windows SharePoint Services 3 and Microsoft Office SharePoint Server 2007) delivers a powerful collaboration platform. It also offers excellent customization capabilities through configuration and custom code. This customization and configuration be done at a system level through server based installations, but also by power users through the web based site management and administration UI.

The power, ease and accessibility of customization in SharePoint encourages a quickly growing and diversifying collection of sites, through manual changes applied directly in the production environment. However this poses several big challenges for release management on SharePoint platforms, specifically:

How to deploy and test new functionality through DTAP (Development, Test, Acceptation, Production) environments

How to make new or changed functionality available on existing site instances

How to handle content staging

Based on Macaw's extensive SharePoint project experience, we have developed a strategy on how to address these challenges.

Deploying new functionality through DTAP environments

The key to DTAP-enabling SharePoint is to create automatic setup packages that leverage SharePoint's built-in deployment technologies. Standard SharePoint solution packages (.wsp files) can carry content files, XML definitions and executable files.

In the Macaw Solutions Factory new functionality is implemented using SharePoint Features and (when needed) code. The Factory then automatically generates a WSP file, which in turn is embedded in a ZIP file, together with automatic (PowerShell) installation scripts. This ZIP file is also known as a MAST package (because it is based on the Microsoft .NET Architecture Style which is at the heart of the Factory).

In the development environment these deployment packages are generated; each package contains all information needed to deploy the relevant components of the solution on any machine in any environment. No user input is required.

The same (binary) package then travels through the DTAP environments, being deployed and tested for approval at each stage:

This approach has been used for years in many projects, and has proven to effectively DTAP-enable SharePoint solutions (as well as solutions based on other Microsoft server products).

Note: there are several series of Macaw Solutions Factory posts, on multiple blogs.
You can use this feed to track all Factory posts.

Maintaining existing site instances

The above Factory approach works well for deploying new functionality for new sites. However since at its heart the Factory leverages the standard SharePoint deployment mechanisms, this approach is limited to what type of changes can be deployed through these (XML driven) mechanisms.

E.g., a key XML structure in SharePoint is the Site Definition. Through Site Definitions you can make functionality available for new sites that will be created based on that new site definition. However, once a site has been instantiated, the SharePoint XML driven mechanisms do not allow you to modify the functionality defined by that XML. Changing a Site Definition after it has been instantiated does not modify the existing site instances, and doing this is unsupported by Microsoft – even when you only want to adjust new sites that will be instantiated after the Site Definition modification.

But there is an alternative: code that accesses the SharePoint object model can change any aspect of sites that are already instantiated. The idea is that to change functionality on (selected) existing sites, you do not write new XML features and site definitions for new sites. Instead, you write a piece of code that upgrades a site instance from version N to version N + 1. Then you automatically chain together all relevant updates to make the latest functionality available on any site.

This idea follows the same pattern as the well-known Continuous Database Integration (CDI) strategy (which incidentally is also used in the Factory for SQL Databases): you write small pieces of code that bring the functionality from version N to version N + 1; the tools do the rest.

Based on this idea, Macaw created the Site Management Tool. Like the other tools in the Factory, this approach is a combination of an implementation style (write upgrade code instead of XML definitions) with tooling to manage and automatically deploy the functionality to existing sites. The nice thing is that the same approach also works for new sites, so a specific piece of functionality only needs to be implemented once – either in standard SharePoint XML if no existing sites need to be upgraded, or in upgrade code otherwise. The SMT is itself implemented as a custom site in SharePoint:

The SMT offers considerable flexibility in controlling which updates will be applied to which sites at which moment:

Which updates: the SMT allows you to define Update Profiles, which in essence is a set of changes grouped together. A site can be assigned multiple Update Profiles.

Which sites to update: the SMT allows you to specify a Site Definition, a URL start or a part of the Site Title for each Update Profile, to select which site instances will be updated. In addition you can specify advanced logic in custom code to determine which sites to upgrade. So you can do this based on type of site, the area in your intranet (e.g. for a department), the site topic (a word in the Site Title) or custom logic:

When to update: It is possible to have new sites, that are based on site definitions that you control, receive the relevant updates when they are created (this works with an update feature stapled to the site definition), to have updates distributed through scheduled SharePoint jobs, and on demand from the SMT site.

Schematically the SMT operates like this:

Like with CDI, this coded update approach does not allow automatic rollback or downgrade of functionality to a previous version if the data involved originates from the production environment.

However in our experience, in the rare cases that the rollback opportunities offered by standard SharePoint backups (site backups and database backups) are not sufficient, you can accomplish an downgrade by writing a specific new update to reverse the effects of the previous one.

The Site Management Tool is DTAP enabled by using the Factory to build the upgrade classes. The upgrade classes are then incorporated in the MAST package and are deployed throughout the DTAP environments, allowing the updates themselves to be tested before they are applied in production.

Content Staging

The last challenge is how to handle content staging in combination with a DTAP approach. Although not all SharePoint solutions require content staging, it is often unclear how to integrate content with the other types of changes deployed to a SharePoint solution.

Macaw's approach to realize this is based on two choices:

Have a separate staging environment that is treated as a second Production environment from the DTAP perspective

Perform automatic content deployment from the staging environment to the production environment through the built-in SharePoint Content Deployment technology.

This effectively means that the production environment receives updates from two sources: content packages from the Staging Environment and functionality (MAST) packages via DTAP.

Conclusion

With these three major release management challenges addressed, Macaw feels confident that large SharePoint installations with a rich variety of quickly changing functionality can be effectively maintained.

Vincent Hoogendoorn

Factory Overview, Part 1: A Bird’s Eye View of the Macaw Solutions Factory
Fri, 24 Apr 2009 10:26 GMT
The Macaw Solutions Factory is quite a big thing, even though it fits both small and large solutions. In this post I will take you on a quick tour of the major systems and tools in the Factory. Later posts in the Factory Overview series will zoom in on the details of those systems and tools.

Note: there are several series of Macaw Solutions Factory posts, on multiple blogs.
You can use this feed to track all Factory posts.

In Factory Vision, Part 1: Introducing the Macaw Solutions Factory I described what the Factory does:

“Automate and standardize development and deployment on top of Microsoft Server products, targeting shared infrastructures with .NET, SQL Server, SQL BI, BizTalk and SharePoint”

and how the scope of the Factory is defined:

“All infrastructure, tooling and content used in Macaw's primary process: Realize IT Solutions with Microsoft Technology”.

Given this it's not really surprising that there's too much in the Factory to describe in one post.
So without further ado, let's start the Factory tour. I'll present the tools in the chronological order in which they typically are first used during the application lifecycle.

Microsoft Virtual Machine Manager

One of the first things that you need to do when you start development on a new solution is to provide machines for the development and test environments. Usually you need at least as much machines as there are developers in the team, plus typically build and test servers. Since each machine logically belongs to one solution (not to a person), over time this can result in a lot of machines that must be kept available (for maintenance) but that only need to run occasionally. Virtualization offers great benefits here, allowing flexible creation and storing of machines, as well as additional benefits such as quick rollback to a checkpoint in case a machine installation gets corrupted.

In the Factory we use Microsoft's Virtual Machine Manager, run on a powerful array of virtual machine hosts, to deliver self-servicing creation, access and management of virtual machines. In addition to fully configurable OS installations, we supply preconfigured machine templates for 32 bits and 64 bits versions of Windows Server 2003 and Windows Server 2008. New machines can be created in a matter of minutes.

The Factory Machine Installer

Virtual Machine Manager offers great flexibility through machine templates, but many Microsoft server products do not work well when they are preinstalled on such a template. This is due to the fact that these server products were not designed to handle a machine rename. Renaming machines happens when you instantiate a machine template – you need to select a unique name for the new machine for network access. At the OS level a tool named Sysprep handles this, but many Microsoft server products lack this kind of machine rename support.

In addition to that, a more flexible and less maintenance intensive mechanism is desired to deliver the desired combination of OS, server products and applications that are needed to develop or test a specific solution.

Enter Microsoft's Deployment Toolkit. We use this toolkit to create and maintain the Factory Machine Installer, which allows users to select any desired combination of server products and applications from a single menu. With a single click all selected technologies are installed fully automatic, with all the latest updates. During the remainder of the installation (which can take up to several hours) no user interaction is needed. If a team needs to start (or is extended) tomorrow, you can create all machines simultaneously and have them ready for production work in the morning.

Automatically installing and retaining stable development and test machines saves a lot of time during the application lifecycle. In traditional situations it is not uncommon to spend several days before a new team member has a fully productive development machine for a specific solution.

For more details, see this post on the Factory Machine Installer by Maarten.

Microsoft Visual Studio

The Factory leverages the power of Microsoft Visual Studio, both its built-in functionality and its extensibility. Visual Studio Professional and Team System editions in VS 2005 and VS 2008 versions are fully supported (note that some MS Server products require using VS 2005 because they do not yet support developing in VS 2008). The Factory allows you to mix versions and editions, even within a single Factory product, as needed.

Microsoft Visual Studio Team System, including Team Foundation Server, is the application lifecycle management (ALM) platform of choice for working with the Factory. However we also support alternatives for version control (Subversion) and automated build servers (CruiseControl.NET Build Station).

The freedom of choice that the Factory provides with Visual Studio versions, editions and components, makes the Factory fit both small and large teams, solutions, and development environments.

The MAST standard

At the heart of the Factory design is the Microsoft.NET Application Style (MAST) standard. MAST standardizes both the logical design of an application built with the Factory, and how the logical design is translated to an implementation structure in a working folder. MAST is based on Microsoft Patterns & Practices guidance on application architecture, it has at its core the three-layered services application pattern:

(Note that MAST supports both the above pattern from the 1.0 guidance and the updated pattern from the 2.0 guidance, which adds a services layer and an application façade component type)

The MAST standard provides a standards-based, durable, stable yet flexible implementation structure that is consistent across Microsoft server products and technologies. MAST offers consistency, separation of concerns, alignment with underlying server products, and cost-effective automation through convention over configuration.

For solutions built with the Factory this results in very good maintainability. Experience also proves that the consistent implementation structure across solutions makes it much easier for developers to work on another solution or on multiple solutions.

The Factory IDE

The Factory Integrated Development Environment (IDE) extends Visual Studio with templates, tools and automation to create a highly productive development environment. Virtually all tools and templates leverage detailed knowledge of the standard MAST implementation structure, to achieve maximum automation. MAST significantly reduces the cost of implementing and maintaining the Factory IDE; it also increases the attainable level of automation.

Our choices on how we utilize the available Visual Studio extension mechanisms in the Factory IDE are guided primarily by pragmatism. Rather than see how far we can get with the ‘default' technology for a specific type of extension, we consistently select the extension technology that is most effective and future proof for our purpose. This approach has resulted in some innovative and powerful Factory IDE features, such as integration with PowerShell and with the Confluence Wiki.

These are the main tools and templates in the Factory IDE:

Factory Product Wizard

In Factory terms, the thing that you build with one instance of the Factory is called a Factory Product. Since the architecture and the implementation structure of Factory products is standardized using the MAST standard, Factory products are also referred to as MAST products. Some examples of products that you can build with the Factory are:

For more details, see in this post under the heading “What can you build with the Factory?”.

Once you have a development machine installed with all required server products and applications, the next step when you start a new solution is to create its working folder. In addition to the source code, each working folder contains a copy of the Factory IDE. This allows integrated versioning of both the source and the development tools; it also makes product-specific Factory customizations easy to create and maintain. The Factory IDE is designed to run in multiple versions side-by-side, so independent versioning per Factory product is also possible in a shared development environment (see this post for more rationale on this).

The Factory Product Wizard lets you create a working folder for a new product, with (only) the tools and templates that you need for developing that product. Since the Wizard knows the MAST standardized working folder structure, which is designed in accordance to the convention over configuration principle, it requires very little user input to create a product working folder that has a consistent structure.

MAST Working Folder

Once the working folder for a Factory product has been created with the Factory Product Wizard, it will contain all the templates, tools and automation scripts that you need to start developing your product. A typical working folder created by the Wizard looks like this:

Working Folder - Source

Source – Product Dependency Folders

.NET 2 Product Dependency Folder

.NET 3 Template Solution

The templates in the MAST working folder are ready to be built and (automatically) deployed with a single action. This allows you to verify if your development machine is functioning end-to-end (using the automated system test suite included in the templates), even before you have written any code.

You can also automatically package and deploy the templates to other machines, e.g. a test machine. This means you get practically free deployment tests starting from day 1 of a project. No more ‘nasty surprises' that are common when you do your first deployment much later, e.g. right before your product must be tested in a test environment, only to discover too late that something in that environment was not installed or configured correctly.

Factory MSBuild extensions

When you build a MAST Visual Studio solution, post build scripts are run via an MSBuild task that hosts PowerShell. These PowerShell post build scripts perform a rapid (incremental) deploy of the current Visual Studio solution to the local machine. Deployment is done from the working folder (where files are edited under source control) to the runtime deployment locations that are configured for the local machine:

Factory Guide Visual Studio Extension

Developers spend most of their time working in Visual Studio. To automate development tasks and offer guidance to developers while they are working in Visual Studio, we created the Factory Guide.

The Factory Guide shows actions that are relevant in the context where the developer is working in. The context consists of elements such as the part of the product they are working on (the Functional Area) and the components that are present in that part. These actions are implemented in PowerShell scripts (the Guide hosts PowerShell). These scripts have direct access to the Visual Studio object model, the working folder, and the server products running on the development machine.

This provides a very powerful and flexible automation environment. Existing scripts can be modified easily and new scripts and actions can also be added easily. PowerShell offers excellent integration with command-line tools, web services and managed code assemblies, so it is easy to leverage any type of asset in the scripts. Modifying and extending the Factory Guide is simply done in the working folder of the product being built; no additional Factory source and no Factory development environment is required, and versioning is simple.

Each action in the Guide is preceded by icons that indicate whether approved and/or community guidance on that action is available (in the central Factory Wiki):

This way, up-to-date guidance availability information is visible at a glance for all actions in the Guide – no need to click through on each action to see whether there actually is any guidance. Any Factory user can add community guidance to any action with a single click. This means that when a developer (specialist) in any project adds guidance about an action, this becomes visible to all Factory users in all projects. Since guidance is initially sparse and then is created on the fly and as needed, this leads to guidance that is both valuable (there is a proven need) and effective (directly visible to all Factory users).

We call this guidance approach: community driven guidance in context.

MAST Solution Builder

First you've automatically created and installed your development machines with VMM and the Factory Machine Installer, and then you have created a working folder on your new development machine for your new Factory product with the Factory Product Wizard. Finally you have automatically deployed and tested the templates that you are going to use. Due to the level of automation in these steps an experienced user will spend less than an hour on them.

At this point you are ready to define the baseline for your new Factory product - meaning that you create all Visual Studio solutions and projects that will be needed (as far as is known at this point in the project). Once you have added all Visual Studio solutions and projects to the working folder, you can add it to source control and let the team start fleshing out the various components.

We created the MAST Solution Builder to let you quickly create new solutions and projects (or add projects to existing solutions). With a single click the Solution Builder takes the selected template projects, creates a new solution (if needed) and instantiates those projects in the new solution:

Since the Solution Builder understands the standard MAST structure of solutions and projects in the Factory, it can automate many tedious and error-prone tasks when creating new projects:

It creates the solution and project folders with the correct name

It copies all selected template projects while removing example code

It sets all namespaces and project properties to the correct names

It sets all correct project and assembly references, and updates linked project items

It creates an FxCop project file with all project assemblies added as target and with the MAST validation rules added (see MAST Validator below)

It updates comment headers with the correct product information.

Since the MAST templates live in the same working folder as the product source, under single source control, the templates can easily be customized and maintained for a specific project. The Solution Builder will instantiate the customized templates when creating new Visual Studio projects.

The MAST Solution builder requires very little user input to do its work; you only need enter the Functional Area (~ VS Solution) name and select the project templates to add.

By applying the principle of convention over configuration, the MAST standard allows to automatically translate a logical design (consisting of machine roles, product dependencies, functional areas and component types) into an implementation structure (consisting of product- and machine- config files and Visual Studio solutions and projects).

All the above makes it possible to create a complete implementation baseline, starting from a logical (whiteboard) product design, in a matter of hours – even for complex, large solutions. At that point, the entire baseline will build and deploy automatically without errors. The team can get from logical design to fleshing out the baseline in one day, while using continuous deployment across development and test environments.

MAST Validator

MAST standardizes the logical design and many details of the implementation structure. But even though this offers a lot of value, it is not realistic to expect developers to know all details of the standard by heart, much less to apply the standard consistently. The MAST templates and the MAST Solution builder provide you with a MAST compliant baseline, but once you start fleshing out that baseline, how will you make sure that you will remain compliant to the standard?

This concern is even more pressing because the Factory IDE tools rely on MAST compliance. If your source is not MAST compliant, there is no guarantee that the Factory IDE tools (e.g. automatic deployment) will work correctly.

To address this concern we created the MAST Validator. As the name implies, the MAST Validator validates the source in the working folder against the MAST standard. We implemented the MAST Validator as an FxCop rule library:

The MAST Validator analyzes much more than typical FxCop rule libraries, which usually only analyze compiled assemblies. The MAST Validator also analyzes source files (including Visual Studio project files) and even databases.

Where applicable, MAST rules adjust themselves automatically to customized templates. E.g. in the example above, the project references in the template solution (which can be customized just as any other solution in a Factory product) are interpreted by the rule as the set of all potential valid project references. If a non-template solution contains any other project reference, the rule reports this as an error. This allows for automatically detecting unwanted dependencies, much like the Layering Diagram in VS 2010 will provide.

In case you wonder why we integrated the MAST rule library in FxCop instead of directly in Visual Studio, this is because FxCop provides us with additional UI that supports the audit process. In FxCop you can exclude rule messages with a reason, and you can see all exclusion reasons at a glance. When you want to do a quick audit of a product, you run FxCop, verify that no active MAST messages exist, and whether you agree with the rationale provided in the notes of excluded messages:

The Factory IDE will only create MAST deployment packages if there are no active MAST error messages; this effectively enforces MAST compliance throughout the application's lifetime.

The MAST Validator helps you to keep your source MAST compliant, and to audit on MAST compliancy, with little effort. At the same time flexibility is preserved: you can ignore rule messages if you record a satisfying rationale to do so.

Factory Local Build Server

The Factory fully supports Visual Studio Team Foundation Server build servers. However as an alternative for smaller projects, a lightweight build server is also included in the Factory IDE:

Factory Vision, Part 1: Introducing the Macaw Solutions Factory
Tue, 07 Apr 2009 10:57 GMT
This is the start of a story about a three-year companywide Software Factory effort. The result is not your common code-generating or process-focused factory. In our effort to realize maximum value, we focused on solving common challenges that existing factories do not address.

We created a Factory to automate and standardize development and deployment on top of Microsoft Server products, targeting shared infrastructures with .NET, SQL Server, SQL BI, BizTalk and SharePoint:

Interested? Read on…

History

For the past three years it has been my job to move Macaw, a Microsoft Systems Integrator, to a software factory approach.

Macaw specializes in realizing IT solutions with the latest Microsoft technologies. While Macaw is organized in several Solution Centers that each specialize in specific types of IT Solutions, it is common for Macaw's clients to run these different solution types on a shared infrastructure (deployed on the same set of machines) and a common Microsoft technology stack (deployed on the same Microsoft server product installations, e.g. SharePoint). We created a Factory to support this.

Three years ago, based on experience gained at Macaw's (then) largest customer, I proposed a vision to create a single, durable software factory to support 80% of Macaw's work as a Systems Integrator. The vision was recognized and adopted by Maarten Sikkema, co-founder and CTO of Macaw, and with his support things were set in motion. I became member of a dedicated Factory core team, which was tasked with designing, building and embedding the Macaw Solutions Factory company-wide.

I convinced Macaw's key technology experts and development community leaders to participate in creating the core Factory standard – the Microsoft .NET Architecture Style (MAST). The entire Factory initiative is driven by voluntary participation (building the Factory) and adoption (using the Factory), as opposed to a mandatory approach. Although this did take a lot of extra effort and discussion to get people on-board, the resulting commitment was strong and durable because people really believed in the initiative and they felt it was also their factory. Durable community commitment is key for a durable factory approach; this is a necessary and worthwhile effort.

After the first year, Serge van den Oever joined the core Factory team. As Macaw's resident SharePoint guru he started out adding SharePoint support to the Factory, but since then Serge continued to add many key innovative elements to the Factory (PowerShell integration, the Factory Wiki, and the Factory Guide Visual Studio extension).

As Factory usage became more and more widespread, an extended team of Factory Champions was created, with members from all solution centers. The Factory Champions were tasked with continually improving Factory usage and usability for their respective centers. Finally a strategy was put into place to integrate results of Macaw innovation projects into the Factory.

During this time, we consciously maintained an internal focus, supporting and strengthening Macaw's capabilities as a Microsoft Systems Integrator.

Today

The Factory has been adopted throughout the company and has been sold under license for use by non-Macaw personnel. The people and processes to continually improve the Factory are in-place. We are integrating the latest Microsoft technologies as they are applied and we capture best practices as we learn them, across all solution centers. Now we feel it is time to share what we have learned and achieved.

In several factory blog post series, Maarten, Serge and I will cover the Factory vision, overview, features, usage and customer cases.
You can use this feed to easily track Factory posts by all contributors.

Vision

Ok, enough history. Since this is the first installment in the Factory Vision series, let me outline the key elements of our vision on a Software Factory for a Microsoft Systems Integrator.

Note: for this introductory post I will stick to the high-level view; it is long enough as it is. However we intend to quickly zoom in on the key details in the next posts, right down to the ‘bare metal'.

Design goals

When designing the Factory, we set these goals:

Fit at least 80% of Macaw's work as a systems integrator, specialized in the latest Microsoft technologies.

Be effective for very small teams (1 – 2 persons) as well as large teams (15 persons).

Support multiple, independently versioned solutions running on a shared infrastructure. Large projects or programs are often organized into multiple teams of 5-8 persons each, where all teams work on the same chain of DTAP (development, test, acceptation, production) environments. This means that the Factory must be versionable per solution; different factory versions must be able to live side-by-side in a shared development environment.

Be effective for short, simple projects (a simple web site) as well as long, complex ones (a SOA).

Deliver “Tomorrow's IT, Today”,

company-wide. Allow the knowledge and experience of Macaw's top

innovators who are working on projects to be captured with minimal overhead, and then distributed company-wide, quickly. This requires easy factory customizability in an ongoing project, frequent factory releases, and easy factory upgrades (also in ongoing projects).

Be durable, meaning that the effort in maintaining the factory – integrating new technologies and practices as they are learned by Macaw - should remain constant, while the total generated value of the factory keeps increasing. This means embedding Factory maintenance and innovation in the organization, and partitioning the Factory components across the underlying Microsoft server products and across the different versions of those server technologies.

Overall vision

Overall we envisioned a durable, company-wide Factory approach that would relieve people from the same tedious, error-prone, repetitive tasks that they encounter on many projects. Instead, they are able to focus on what is specific and innovative for the current project, while the Factory provides them up-to-date, combined knowledge and experience of their best colleagues at their fingertips. This approach will bring every team closer to the level of the company's very best combined; a dream team.

Core Factory value

Key to the value of the Factory is the value of Macaw's best innovators, who are leaders in their respective fields, working in innovative projects. The Factory amplifies this value (making it available for other teams) and also makes it more durable (even after the expert leaves the company, the captured knowledge and experience remains available). In a market where top IT talent remains scarce, hard to come by and hard to retain, this Factory approach provides a significant strategic advantage for an innovative systems integrator.

Customers benefit from consistent high levels of productivity and expertise that - across the board - are simply unattainable for traditional systems integrators who still use a manual, one-off approach to development. When customers license the Factory for use by non-Macaw personnel, they boost those developers with Macaw's packaged expertise.

What can you build with the Factory?

The first step in defining a Factory is describing what types of solutions you can build with it – what the targeted software product family is. We observed a large overlap across different solution types with respect to what Microsoft server products are used and what best practices are applied. In addition to that, different solution types often coexist on a single shared infrastructure.

So instead of building a separate factory for each type of IT solution, we decided to define a single Factory that can build:

“Solutions composed from a combination of custom .NET code and zero or more Microsoft server products”.

The phrase “zero or more” indicates that we target solutions ranging from the ‘traditional' fully custom solutions to solutions based on standard server products. The latter type of solutions has become predominant for Macaw's clients; as much as possible of the functionality is bought with standard server products. The remaining functionality is then realized by customizing and extending the standard products:

Examples of custom components are web pages, console UI or windows UI, business components, workflows and business entities.

Product extensions are specific types of components that Microsoft has defined to allow extending the functionality of their server products through code, such as web parts for SharePoint or pipeline components for BizTalk.

Product configurations are product-specific settings and data that specify how to use the product's existing functionality, such as configuration of web sites and web applications in IIS, site definitions for Moss 2007 and custom databases for SQL Server.

The Factory supports building small or simple solutions equally well as building large or complex solutions. The Factory is automatically configured to only include the tools and templates that you actually need for the solution at hand. If the solution is ‘light', the factory will also be ‘light', e.g. to build:

What does the Factory do for you?

Like most factories, this Factory increases developer productivity and quality by automating repetitive, error-prone development tasks. When building the above types of solutions, this is how the Factory addresses the most common big challenges for you:

Challenge

Strategy

Install development machines
For every person working in a development environment, a project-specific development machine must be installed with all required server products in a consistent configuration. This often involves many manual, error-prone steps; it is quite common to spend several working days installing and configuring a development machine correctly. Inconsistent installations also cost time downstream due to incomplete functionality, local defects or inability to reproduce defects from other environments.

Automated Factory Machine Installer
Virtual development and test machines are created fully automated, providing multiple OS choices in 32 and 64 bits variations. From a single selection menu all required server products are installed fully automatic on each machine in a single scripted step, with all the latest service packs and patches.

Deployment

Deployment needs to be done many times during the entire application lifecycle, throughout different environments. However extensions and configurations for Microsoft Server products require a wide variety of deployment tools and many error-prone, manual steps.

Automated deployment
Fully automated deployment across all DTAP (Development, Test, Application and Production) environments.

Maintainability
Maintaining a clear, consistent, stable source structure over time and across solutions for a wide variety of component types while developers, functionality and technology change.

Standard implementation structure
The Microsoft .NET Architecture Style (MAST) standard provides a standards-based, durable, stable yet flexible implementation structure that is consistent across Microsoft server products and technologies. MAST offers separation of concerns, alignment with underlying server products, and cost-effective automation through convention over configuration.

Specialist knowledge
How to make all required specialist knowledge of the underlying server products available in each development team, while specialist fields are narrow and scarce specialists are distributed across the organization.

Community driven guidance in context
Like most factories, specialist knowledge and best practices are packaged in templates and tools.
Going beyond that static approach, we also provide community driven guidance in context. Within Visual Studio the actions that are relevant in the current context are displayed. Guidance on each action is available in a single central Wiki – both approved guidance and community content. When any community member enters some guidance, it instantaneously becomes visible and available for all Factory users, when they are working in a relevant context.

New world of work
We provide the infrastructure to enable all employees to work together from any location, ad-hoc. They can work together on the same machine screen while talking to each other, allowing them to work together as least as effective as when they were physically working on the same machine.

What's in the box?

So now we have seen what types of solutions you can build with the Factory, and which common challenges the Factory helps you address. Now it's time to look at what exactly the Factory itself is. Here is our governing definition of the Factory:

As you can see this is quite a broad definition for a factory. We explicitly include not only the development tools and templates, but also all infrastructure that enables the Factory users to work (together) from any location. One of the main pillars of our factory is to support the “New World of Work”. This is part of our strategy to handle the challenge of distributed specialist knowledge.

How do we ensure sustained, increasing Factory value?

From the outset durability was a design goal for our entire factory approach. The main steps we took to realize this were:

We embedded Factory usage and usability in the organization by creating the Factory Champions team and working closely together, while keeping Factory usage a choice.

We proposed the Macaw Innovation Board, as a central place to provide overview and direction on all innovation projects. This allows to integrate and align innovation projects and the Factory to each other.

We created a Factory Improvement Forum with voting, open to all Factory users, to allow community-driven feature proposal and prioritization. This ensures that we are always working on the right features, that are the most wanted and valuable for the project teams.

We partitioned the Factory components across the underlying Microsoft server products and across the different versions of those server technologies, so we can distribute our maintenance effort proportionally to how much value is currently generated with a specific technology. Obsolete (versions of) technology can be removed in newer Factory versions very easily, while older Factory versions that still use that technology can continue to live side-by-side with newer Factory versions.

We made the Factory implementation very open and easily modified and extended. We implemented a large part of the Factory functionality in open PowerShell scripts to allow quick customization without the need for separate Factory source code or a Factory development environment. Our solution templates are customized with the same ease as when coding in regular Visual Studio Solutions. This way the innovation can be driven by projects on the go, modifying their factory instance as they learn.

We provide on-call consultancy from the core Factory team to help innovators extend the Factory within their project context in such a way that the result is suitable for other projects, and for integration in new Factory releases.

We provide one-click functionality to any Factory user to share guidance on any Factory action, or to post remarks or requests to clarify guidance. The community contributions are then used to create or improve the approved guidance. This way we prevent a central documentation bottleneck and we prevent work on guidance that is not needed.

The Factory IDE is versioned with each product that is built with it; multiple Factory versions can live side-by-side on the same machine. This way, factory upgrades and customizations can be done independently for each solution, even when the solutions are developed in shared DTAP environments. The impact of Factory upgrades is much less, because you only upgrade for projects if and when the benefits are significant in the current lifecycle stage of the solution.

Closing remarks

I hope this post gives you an impression of where we have come from and where we are heading with our Factory approach. For me it is a privilege and a joy to be realizing this vision.

I would like to take this opportunity to thank everyone who helped to make this possible. In particular I would like to thank Maarten Sikkema for his unwavering belief and support to let us realize this vision, and his personal, hands-on involvement. Also many thanks to Serge van den Oever for the great teamwork and his many innovative ideas. Thanks to the Factory Champions team who encouraged and helped their colleagues to use the Factory and who provide us with invaluable feedback and direction. Thanks to the innovators who volunteered to integrate their learnings in the Factory, and last but not least thanks to all Macawers who believe in and use the Factory.
Moving forward, realizing tomorrow's IT, today, the road ahead beckons brightly. In this article I outlined the vision that resulted in the Factory as it exists today. More recent developments in Microsoft IT have inspired us to expand our vision. Specifically VS Team System 2010, Microsoft Azure, GoGrid, ElasticHosts, and textual and graphical modeling with M and Quadrant offer great potential for future Factory versions, to raise the abstraction level, target cloud solutions and offer factories in the cloud, and to target more team roles and application lifecycle management (ALM) aspects. But more on that at a later time…

Vincent Hoogendoorn

Self-Service deployment with VMM and the Microsoft Deployment Toolkit
Thu, 08 Jan 2009 08:03 GMT
What is the best way to organize and maintain a software development infrastructure? At Macaw (and I think most software development environments), virtualization has become an essential part of the way we do our work. For most projects, we "clone" a standard development image for each team member and each integration, test and build server. Team members run their virtual machine using Virtual PC or Virtual Server on their notebook or desktop, often using external hard drives for better performance. The project servers are hosted in our centrally managed Virtual Server environment.
But this approach has many disadvantages: security is an issue with laptops getting stolen and USB drives getting lost, reliability is an issue with external hard drives crashing, performance is sluggish because of memory constraints and disk performance of laptops. We use Vista x64 on all our laptops with bitlocker enabled, this is quite secure but virtual machine performance is not very good, even on very powerful laptops with 4 Gigs of ram, especially when you run "heavy" virtual machines with Visual Studio, SQL Server and MOSS installed.
So we had to come up with a better solution. This solution was found in Microsoft Virtual Machine Manager (VMM) combined with Windows 2008 Terminal Services Gateway (TSG) for remote access. In this setup we run all virtual machines in a centrally managed server environment, using Remote Desktop or RemoteApp to access the development machines. This infrastructure provides better performance because virtual machines run on fast servers with quadcore processors and fast disk systems. And there is no project data on remote laptops anymore, so security and reliability are much, much higher. At Macaw, we initially built our Virtual Machine environment on VMM 2007 and Virtual Server 2005 R2, last month we upgraded this environment to VMM 2008 and Hyper-V Server 2008 on all virtual machine hosts. This infrastructure has quickly become very popular within Macaw, especially TSG works really well.
However, there is always another problem to solve. Now, we were creating and managing much more virtual machines in the central environment. Because most of our project-related infrastructure was now hosted centrally that meant extra work for our internal IT staff. Setting up a new development environment for a small team, with a few development machines and some build and test servers, costs a lot of work. Also, our one-size-fits-all development image, based on Windows 2003 Enterprise with Visual Studio 2005, SQL Server 2005 and MOSS, had inherent drawbacks. Cloning an image that has MOSS and SQL Server installed is not without issues. We have solved these, but it is unsupported and never felt 100% "right". The image also began to show its age: more and more projects were asking for 64 bits hardware, Windows 2008, Visual Studio 2008 and SQL Server 2008. The number of permutations with the 2005 and 2008 stack of software is large, so the cloning approach was no longer a viable solution.
For this reason we are now introducing the Factory Installer. What we created is a self-service model for completely installing development machines that can be used by any project team in our organization and Solutions Factory customer (we will sell this solution as as part of our Solutions Factory). After approval by a manager, the team receives VMM "quota points" and can go to the VMM Self Service Portal (a standard web interface for management of virtual machines, provided by VMM) and start creating virtual machines based on Virtual Machine templates. Currently we have 4 Operating System templates available: Windows 2003 x86 and x64, Windows 2008 x86 and x64. After computer name and admin password have been given, the new machine is created, deployed to a server host and started.

VMM 2008 Self Service Portal
Create new virtual machine based on template

At first login, the user is presented the Factory Setup Wizard. This wizard lets you install the complete software application suite for this new virtual machine in one single process. We currently support:

.NET Framework 3.5 Sp1
Visual Studio 2005, 2008
SQL Server 2005, 2008, with Analysis, Integration, and Reporting Services as separate options
WSS3 and MOSS 2007
Biztalk 2006 R2
Popular utilities like Firefox, 7Zip, Adobe Reader, Flash and Silverlight
Below are some images from the Installation Wizard.

Select applications to be installed

Confirmation Dialog: after this screen the unattended installation will run for up to 4 hours, depending on selected applications
After selection of the applications needed for the project, the Installer gets to work. Each software package is installed silently, with reboots in between. All applications come preconfigured with the most recent updates and service packs and installed according to our guidelines, and the latest security updates are installed automatically. Installation of a "loaded" server can take up to 4 hours time, but you don't have to wait for that: no manual intervention is required during the process.

During the install sequence
We created the Factory Installer using the Microsoft Deployment Toolkit. This freely available toolkit was originally created for deployment of desktop computers, but now also supports installation of server operating systems and custom task sequences. We have loaded the toolkit with modified instances of Windows 2003 and 2008 OSses, Virtual Server 2005 and Hyper-V drivers and created unattended installation scripts for those server and client applications that we use most often on our machines. There is a lot of technology at work here, you have to go through a lot of steps, but the whole mechanism works beautifully. Getting some applications (Visual Studio is an example) to install properly and silently takes a lot of magic; it cost us a lot of time to get every combination working correctly. We are still hard at work getting the final wrinkles out. But now that we have this whole mechanism working, from now on it will be very easy to start supporting a new operating system such as Windows 2008 R2 or add an application such as PerformancePoint to the wizard. To add an OS, we would create a customized unattended install of the new OS using vLite and the System Image Editor, add it in the Deployment Workbench, deploy it to an empty virtual machine, create a VMM Template from it and add this template to self service policies. Users can then start using the new OS for experimentation, and install all of the existing applications on it without extra effort. This will save us a lot of time and effort.

The Microsoft Deployment Workbench
Personally I think this solution is interesting not only to us, but for any organization that regularly deploys (virtual) servers with a variation of application software. Using the method I described, it is not too hard to create a deployment system like this in your own environment, especially if you have some prior experience with automated Windows deployment (i.e. WDS, RIS, System Center Configuration Manager, Altiris). But you will find that, although it is not hard to get the basic process working, it is hard to get a complete set of hardware environments, OSses and applications working really well in combination. It takes a lot of testing and that costs a lot of time.
For those who don't want to spend that time, we will sell this solution as a separate module of the Macaw Solutions Factory. It must be mentioned that this is not shrink-wrapped software, customization will always be needed because for many of the supported applications there is licencing involved, and the customer has to provide valid product keys for each OS and application that goes into the solution.
Finally, it is nice to know that this solution works just as well for physical server deployment and even cloud-based deployment: create a CD/DVD deployment point, upload the very large ISO file that Deployment Workbench will generate to cloud-based storage, attach the ISO file to a newly created cloud-based server (with Windows installed or even completely empty), boot it, and off it goes!