Thunderbird 3.1 breaks our customizations

It appears that a Thunderbird 3.1 release (probably 3.1.1) introduced a change that has caused at least one entry in our UVM ISP Hook file to become invalided.  Two alert distributed support staff recently reported that the SMTP Server that our installer configures now is set to perform plain-text authentication over an unencrypted channel rather than using STARTTLS.

We know that this setting was correct in the past, and we see that it still is set in our ISP profile… Thunderbird simply is ignoring the setting.

Fortunately, there is a workaround for this setting.  We can modify the “mailnews.js” file in the installer to have the default value for “try_ssl” set to “2” instead of the original default of “0”.  All I had to do was add a line of “sed” work to our build script:

..binsed.exe --binary "s/try_ssl", 0)/try_ssl", 2)/"  .buildnonlocalizeddefaultsprefmailnews_new.js
MOVE /Y .buildnonlocalizeddefaultsprefmailnews_new.js .buildnonlocalizeddefaultsprefmailnews.js

This “sed” command searches for the phrase try_ssl", 0 and substitutes try_ssl", 2 . After repackaging, we find that the first SMTP server account created after install (which is therefore the default SMTP account) will be set to use “STARTTLS” instead of “clear text”.
I worry about future unannounced retirements of ISP Hook settings.

Office Web Apps and SharePoint 2010

Office web apps are here!

Where do I get them? From Microsoft Volume Licensing of course:
You need to look in an unexpected place, though. Under the “Office” section, select “Office Professional Plus 2010”, then select the 32-bit edition (not 64-bit… weird!). Take note of the Web Apps product key in the product selection page.

Installation instructions can be found here:
And they are pretty good, with the exception of one PowerShell syntax error…

When you get to the stage of installing Service Application Proxies, if you choose to use the PowerShell method, you are given a small script that starts with the following command:

$appPool = Get-SPServiceApplicationPool -Name "SharePoint Web Services Default"

This command will not work as posted. “-Name” should be “-Identity”. A more reliable command might read:

$appPool = Get-SPServiceApplicationPool | where-object {$_.Name -match "SharePoint Web Services Default"}

Preparing the SharePoint 2007/WSS3 Database for Upgrade

  1. Run pre-upgrade check:
    stsadm –o preupgradecheck
    (to be run on the WSS 3 server… requires WSS 3.0 SP2 and October 2008 CU).
    Now, repair any problems that were reported…
  2. Delete orphaned sites:
    1. run: stsadm -o enumallwebs
      – find sites with “InSiteMap” set to “false”.
    2. take note of the SiteID GUID, then run:
      stsadm -o deletesite -force -siteid [siteid] -databaseserver [dbserver] -databasename [dbname]
    3. Delete references to missing web parts:
  3. The upgrade check report tells us:
    “ The following web part(s) are referenced by the content, but they are not installed on the web server Id = dcdbbbd0-8dd6-1ecb-a3b2-12d30061d482, Type = Unknown, Reference = 7, Status = Missing …”
    but there is no particular advice on how to fix the problem. I wasted a few hours trying to enumerate web parts in use by a site using PowerShell, and ended up digging around in the Content database using TransactSQL… here is what works:

    1. Run the following against the content database:
      use [contentDatabaseName]
      select DirName,LeafName from dbo.AllDocs where id in
      (select tp_PageUrlID from dbo.WebParts where
      OR (tp_WebPartTypeID='d5101cfe-e315-c578-cd06-1966f283e3ed')
      OR (tp_WebPartTypeID='602e7431-ac3e-75b9-c8e0-57533bdab161'))
    2. Access the page returned by the above query, appending “?Contents=1”.  Delete any web parts reporting errors.
  4. We are informed that various features are referenced by content, but that the features are not available on the server.  We are given only feature IDs.  Back to SQL:
    • use STSContentDBPrime
      select FullUrl from [Webs] where Id in (
      select WebId from [Features] where (FeatureId = 'bbe9def7-2fe9-a0b1-d712-aa128c837ebe')
      OR (FeatureId='bbe9def7-2fe9-a0b1-d7bb-aa128c837ebe')
    • This returns the sites that are using these “bad features”.  But what to do about it?
    • suggests the use of “WSSAnalyzeFeatures” and “WSSRemoveFeatureFromSite“, both of which work to purge the evil old CKS:Enhanced Blog edition feature from the one site that was using it… sheesh!
  5. We are warned that “The following site definition(s) are referenced by the content, but they are not installed on the web server”. The name of the missing site definition is “Unknown”. We are given only a “template id” (11003) and a Language Code (1033). Finding where this definition is being used can be accomplished with a very small sql query:
    use SharePoint_Content_1 
    SELECT Title, MasterURL, WebTemplate FROM dbo.Webs where WebTemplate='11003'
  6. We are warned that “setup files” are referenced in the database that are not present on the server. We are given the names and paths of the files that are missing, but not the web sites that reference them. The offending site can be tracked down quickly using “stsadm”. Run this command:
    stsadm -o enumAllWebs -includesetupfiles > allWebs.txt
    Then search the resultant file for one or more of the filenames listed in the pre-upgrade check report. In this case, I was able to locate a site that used “SharePoint Learning Kit” components. This site was completely broken, since we removed the Learning Kit over a year ago. I got approval from the site owner, and deleted the offending site.
  7. We are warned that the feature “f374a3ca-f4a7-11db-827c-8dd056d89593” is referenced by over 500 sites, but that the feature is no longer present on the server.  This feature is the “RadEditor for MOSS for IE”, which in fact has been phased out.  Lets use a handy “for” loop.
    1. First we export the SQL query we used above as a CSV file (this time searching for the “f374a…” feature instead).
    2. We use that csv as fuel for our “for” loop:
      for /f %i in (H:BadFeatureSites_2010.csv) do WssRemoveFeatureFromSite.exe -scope site -url -featureid f374a3ca-f4a7-11db-827c-8dd056d89593 -force
    3. Oh but look… WssRemoveFeatureFromSite can’t deal with sites that have spaces in the URL (why would you have a space in your site name, right?).  dang!  Let’s try “SharePoint Feature Administration and Clean Up Tool“… it works!
  8. We are told that we are missing some additional template files such as:
    Path = [C:Program FilesCommon FilesMicrosoft SharedWeb Server Extensions12Template1033MPS..stslistswplibdwpMSContentEditor.dwp], Reference = [2], Status = [Missing]

    • Some searching discovered KB839877, which informs us that an earlier version of the “Workspace Meeting” template contained an accidental double period when a single period was intended.  But how do we find the site that is using this bad template?
    • The excellent tool “SharePoint Manager” from Carsten Keutman allows us to explore the SharePoint object model to more quickly find Site Template data that we need in a site.
    • The really really excellent tool “PowerGUI Script Editor” was used to quickly develop this script:
      # Discovers Meeting Workspaces from all "Webs" in the SharePoint web application defined in the "webAppUrl" variable.
      # Outputs discovered data to file "sitelist.csv
      # Greg Mackinnon, 2010-02-05
      [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") | Where-Object { $_.GetType().FullName -ne "System.Reflection.Assembly" }
      $webAppUrl = ""
      $wa = [Microsoft.Sharepoint.Administration.SPWebApplication]::Lookup($webAppURL)
      #[string]$out = "List of sites using the Meeting Workspace template"
      Remove-Item -Path .sitelist.csv -Force
      foreach ($site in $wa.sites) {
      foreach ($web in $site.AllWebs) {
      Write-Output "Checking web: " + $web.Url
      if ($web.WebTemplate -match "MPS") {
      Out-File .sitelist.csv -InputObject $web.ID.Guid -NoClobber -Append -Encoding Unicode
      #Out-File .sitelist.csv $out -Force
    • The script returns a list of “web” object GUIDs.  We feed this into SQL to discover where in the content database the sites that are using these web parts are implemented:
      use STSContentDBPrime
      --select DirName,LeafName,SetupPath from dbo.AllDocs where WebId='abee2623-56d2-43d6-8a9c-7b362fbd323b'
      select DirName,LeafName,SetupPath
      from dbo.AllDocs
      where ((WebId in ('siteGuid1','siteGuid2','...')) AND (SetupPath LIKE '%MSContentEditor.dwp%'))
    • Finally, the SQL query finds the “Meeting Workspace” sites that are using this corrupted web part.  We probably could just delete the web parts in question from the web part gallery, but I chose instead to use the stsadm.exe “export”, “deletesite”, “createsite”, and “import” commands to re-create the sites.  The export utility filtered out the corrupt content for us.  Once restored, the discovered sites no longer contained corrupted web parts.   Phew!
  9. The WSS3 instance has been prepped… Clone existing content database, and proceed with upgrade testing!

Adding Drivers to the built-in Windows Recovery Environment

Windows 7 and Windows 2008 R2 feature an out-of-box installation of the very useful Windows Recovery Environment (WinRE).  WinRE can save your buttocks… but what if your system is using storage drivers that are not available in the out-of-box WinRE environment?  Such as the VMware Paravirtual SCSI driver (PVSCSI)?

Fortunately, WinRE is just a modified WinPE image, so you can add drivers using DISM.exe, right?  Sure… if you can find the WinPE image that is used by WinRE!  Fortunately, there is a tool for this.

Open a command prompt on your Server 2008 R2 system, and run “REAgentC.exe /info”… the output will tell you where to find the image:

Recovery Environment: \?GLOBALROOTdeviceharddisk0partition2Recoverybb338b68-0d2c-11df-be64-84e1223bd0bb
BCD Id: bb338b68-0d2c-11df-be64-84e1223bd0bb

So, on the second partition of the first disk (also known as the “C:” drive, according to “Diskpart”), you will find a hidden “recovery” directory, with subdirectory “bb338b68-0d2c-11df-be64-84e1223bd0bb”.  Within here is “winre.wim”.

Now there is simply the matter of injecting the drivers. First, place all the drivers you wish to inject into an easily accessible directory (such as c:localtemp, in our example), and then run the following commands:

mkdir c:wimtemp
dism /mount-wim /WimFile:c:recoverybb338b68-0d2c-11df-be64-84e1223bd0bbwinre.wim /index:1 /mountdir:c:wimtemp
dism /image:c:wimtemp /add-driver /driver:C:localtemp /recurse
dism /unmount-wim /mountdir:c:wimtemp /commit
rmdir /q c:wimtemp

Et voila! We press “f8” on next reboot, select the recovery environment, and suddenly we have full access to the local disk.

ADFS 2 and SharePoint 2010 Integration

Here is a quick entry on ADFS2 and SharePoint 2010 integration. It is not an implementation guide or end-to-end walkthough… that comes later, if we decide to implement this thing.

At present, I am most interested in the model of SharePoint->ADFS2->Shibboleth, where the SP-STS trusts tokens from ADFS2.  ADFS2 is part of a chained federation with our Shib service.  ADFS will consume Shib tokens, then transform them for the benefit of SharePoint.  However, I have no idea how to implement this solution at this time.

There are a few-too-many blog entires out there detailing how to configure ADFS2 and SharePoint 2010 for integration.  Trouble is, many of the step-by-step guides present contradictory configuration steps.  I guess there is no substitute for a deep, working knowledge of ADFS 2, SAML, and other Federation topics.

Here are some of the claims setup guides I have been working with:

Here are additional configuration posts on the process of upgrading an existing SharePoint Web Application from “Windows” authentication to “Claims” authentication.  The common denominators?

  1. You must add a valid new user to your claims-aware web app before migrating existing users, or the web application will be inaccessible after migration (or indeed, even before migration!)
  2. To trigger migration of users, you must invoke the “Migrate Users” method on your web app, E.g.:
    $wa = get-SpWebApplication "https://webappurl"

The things here that seem very unclear to me are:  What exactly is being done when you invoke the “MigrateUsers” method on the Web Application object?  How does SharePoint map legacy “Windows” users to new “Claims” users?  Anyway, here are the links:

Pages containing information that I have found useful while contemplating how to pull off SharePoint 2010:

Many of these links, as it turns out, were already discovered by members at  Doh…

ADFS 2 Setup and Configuration

The following is a work in progress… Everything done so far works, but this is not yet a complete end-to-end deployment guide…

ADFS 2: new… improved?

There were lots of setup and configuration guides for ADFS 1.0, even though the product was nauseatingly difficult to understand.  Along comes ADFS 2… new, more powerful, better, more standards compliant… documentation reads like scrawl on a napkin.

Some setup quirks:

  • Don’t forget that no current version of Windows Server includes ADFS 2.0… not even Server 2008 R2.  If you want to install ADFS 2.0 you must download it from MS and install.  DO NOT add the out-of-box ADFS role on Server 2008 R2.  It will confuse you, because it does not disclose which version of ADFS that it is.
  • Since we are planning a farm, generate a farm SSL certificate on one of the ADFS servers, then export the certificate in PFX format (that is, with the private key), and restore the cert to the second ADFS server.
  • It is considered desirable to put the ADFS database on a reliable external database server, but documentation on this options is limited to command line help.  Here is what I did:
    1. On one of the ADFS servers, run:
      Fsconfig.exe GenerateSQLScripts /ServiceAccount [account] /ScriptDestinationFolder [destination folder]
    2. Install the appropriate version of the SQL Native Client on the ADFS servers.
    3. On the SQL server, run the two scripts that were generated by the above command.  This will create two databases… AdfsConfiguration and AdfsArtifactStore, and set permissions for the service account specified.
    4. Make sure that any firewalls on the SQL server will allow the ADFS servers to connect to it.
    5. Since we use SQL mirroring, set up mirrors for these databases.  Add the service account to the list of accepted logins on the mirror server, since this will not be done automatically.
    6. Generate certificates that will be used for the ADFS web site, token signing, and token decryption.  Put the certificates in the Windows certificate store of the local computer.
    7. Preconfigure binding of the SSL certificate to the default IIS web site.
    8. Configure the first server in the farm using the following syntax:
      Fsconfig.exe CreateSQLFarm /ServiceAccount [Service Account]
      /ServiceAccountPassword [password] /SQLConnectionString "Initial Catalog=AdfsConfiguration; Data Source=spellbound; failover partner=rearwindow; integrated security=SSPI; Network Library=DBMSSOCN" /FederationServiceName /CleanConfig /CertThumbprint "[thumbprint]" /SigningCertThumbprint "[thumbprint]" /DecryptCertThumbprint "[thumbprint]"
      Note that the thumbprint can be obtained by viewing the properties of the certificate in Windows explorer.
    9. Note that the ADFS “Artifact Store” database should get configured automatically, but you can ceck on this by doing the following:
      1. Launch PowerShell with admin privs
      2. Run the command “Add-PSSnapin microsoft.adfs.powershell”
      3. Run “get-adfsproperties | select artifactdbconnection”
      4. Use “set-adfsproperties -artifactdbconnection” to change the string if necessary.
      5. See this resource for more details:

Of course, an ADFS server with no Federation partners does not do us much good.  Unfortunately, I don’t have any other ADFS servers that I want to integrate with, either.  What interests me more are Shibboleth services (such as our own “”), and federation with other “InCommon” partners.  I also am interested in “user centric” identity providers such as OpenID and “Windows Live ID”.  Here are some links to get us started down this path.  Unfortunately, I cannot find anything that details exactly what we want: