Find large files through Search in SharePoint 2010

How to get a list of large files in a (very large) SharePoint 2010 Farm, sorted by file size?

I discussed about this requirement with my friend and teammate Riccardo Celesti and we ended up with the same answer: use search!

Unfortunately, since the customer’s environment is still running on top of SharePoint Server 2010, the solution was not that immediate: we would need to add a metadata property mapped to the file size (and schedule a full crawl afterwards, which may be not feasible).

Or… it’ SharePoint 2010, right? We still have the FullText SQL Query Language available.

Typically, the SQL Query Language should not be an option, since this feature has been removed in SharePoint 2013 and was already considered obsolete in SharePoint 2010 (take a look at the “tip” at the beginning the MSDN page).

Anyway, this was a “fire and forget” requirement, so here’s what we ended up with:

Add-PSSnapin Microsoft.SharePoint.PowerShell
$site = Get-SPSite http://the-url-of-a-site
$query = new-Object Microsoft.Office.Server.Search.Query.FullTextSqlQuery($site);
$query.ResultTypes = "RelevantResults"
$query.RowLimit = 1000
$query.EnableStemming = $false
$query.TrimDuplicates = $false
$query.QueryText = "SELECT URL, filename, size, path FROM SCOPE() WHERE IsDocument=1 ORDER BY size DESC"
$results = $query.Execute()
$table = New-Object System.Data.DataTable
$table.Load($results["RelevantResults"], "OverwriteChanges")
$table | ogv

 

Hope useful 🙂

PowerShell Profiles and Processor Architectures

I’m using PowerShell profiles quite extensively, especially because I’m typically working on several (virtual) machines and profiles are definitely easy to keep settings in sync.

Profiles are independent from the Processor Architecture, which means that you should be careful when loading snap-ins or invoking features that rely on a specific architecture.

One way to ensure that your profile scripts run smoothly on both x86 and x64 processes is to apply conditional logics based on the PROCESSOR_ARCHITECTURE environment variable ($env:PROCESSOR_ARCHITECTURE).

I would suggest you to perform this check even if you think that you will only be executing scripts with the x64 Shell.

You may think “no, I’ll never be using the x86 version of PowerShell”… you’re probably right, until you try to launch a script from within the Visual Studio IDE 🙁

Uffa

Migration to SharePoint 2013: “The Controls collection cannot be modified because the control contains code blocks”

The background

If you have ever written some ASP.NET code making use of server side code blocks, you may have occasionally seen errors like this one:

The Controls collection cannot be modified because the control contains code blocks

This is typically a symptom of some “mess” with code blocks (i.e. <% … %> expressions) and dynamically generated control trees.

You can find a detailed explanation in this post by Rick Strahl.

If you want to quickly reproduce this issue, consider for example the ASPX page below:

<%@ Page Language="C#" AutoEventWireup="true" CodeBehind="Default.aspx.cs" Inherits="WebApplication2._Default" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
  <title></title>
</head>
<body>
  <form id="form1" runat="server">
    <div>
      <asp:Panel ID="mainContent" runat="server">
        <% Response.Write("Code Block"); %>
      </asp:Panel>
    </div>
  </form>
</body>
</html>

Here, a container control (the Panel with ID “mainContent”) contains a code block that just writes a few characters to the Response output.

Everything is fine, as long as you do not explicitly modify the Panel’s control tree: in the code snippet below, for example, a Literal control is being added dynamically as a child control of the “mainContent” Panel, which will throw an Exception at runtime for the reasons described above.

using System;
using System.Collections.Generic;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

namespace WebApplication2
{
  public partial class _Default : System.Web.UI.Page
  {
    protected override void OnInit(EventArgs e)
    {
      base.OnInit(e);

      // The line below will throw an exception.
      this.mainContent.Controls.Add(new Literal { Text = "Added dynamically" });

      // The line below will run correctly.
      this.Form.Controls.Add(new Literal { Text = "Added dynamically" });
    }
  }
}

Also, notice that adding the Literal control as a child of the Form control would run just fine: the Form control itself does not contain code blocks, so there’s no clash with dynamically generated control trees.

You can try this yourself: just comment out the second and the third line of the OnInit method respectively and see the results:

image

image

The issue

That said… how is all this related to the Migration from SharePoint 2010 to SharePoint 2013?

Well, it turns out that a small change has been introduced in the behavior of the WebPartPage class, that sits behind most of the out-of-the-box site pages (List Forms, List Views, etc.), and is often used as the base class for custom Site Pages provisioned through Module features.

Since SharePoint 2010, users are allowed to insert Web Parts inside Rich Text areas, as opposed to the classical Web Part Zones.

This does not imply that web parts can really be inserted outside of a web part zone (unless you use a web part as a standard web control, of course, which is a completely different story whatsoever).

Indeed, the Web Part Page creates a hidden web part zone, that is dynamically added to the page controls in order to host web parts inserted into Rich Text areas. The HTML markup that corresponds to these web parts is then moved in the correct position at runtime by the means of a bunch of Javascript code.

If you take a look at how the WebPartPage class accomplishes this, you may notice a private method called EmitHiddenWebPartZone.

In SharePoint 2010, the implementation of this method adds a Panel control as a direct child control of the Form (see below):

private void EmitHiddenWebPartZone()
{
    if (ScriptManager.GetCurrent(this.Page) != null)
    {
        SPWebPartManager sPWebPartManager = this.SPWebPartManager;
        WebPartZone webPartZone = new WebPartZone {
            ID = "wpz",
            PartChromeType = PartChromeType.TitleOnly
        };
        if (((this.Context == null) || (this.Context.Request == null)) || ((this.Context.Request.Form == null) || (string.IsNullOrEmpty(this.Context.Request.Form["wpcmVal"]) && string.IsNullOrEmpty(this.Context.Request.Form["_wpcmWpid"]))))
        {
            webPartZone.AllowLayoutChange = false;
        }
        sPWebPartManager.Register(webPartZone);
        this.wpz = webPartZone;
        Literal child = new Literal();
        Literal literal2 = new Literal();
        child.Text = "<div style='display:none' id='hidZone'>";
        literal2.Text = "</div>";
        UpdatePanel panel = new UpdatePanel {
            ID = "panelZone",
            UpdateMode = UpdatePanelUpdateMode.Conditional
        };
        panel.ContentTemplateContainer.Controls.Add(child);
        panel.ContentTemplateContainer.Controls.Add(webPartZone);
        panel.ContentTemplateContainer.Controls.Add(literal2);
        this.Page.Form.Controls.Add(panel);
        this.panelZone = panel;
        Literal literal3 = new Literal();
        string valueToEncode = string.Empty;
        if (((this.Context != null) && (this.Context.Request != null)) && (this.Context.Request.Form != null))
        {
            valueToEncode = this.Context.Request.Form["wpcmVal"];
        }
        if (valueToEncode == null)
        {
            valueToEncode = string.Empty;
        }
        literal3.Text = "<input type='hidden' id='_wpcmWpid' name='_wpcmWpid' value='' /><input type='hidden' id='wpcmVal' name='wpcmVal' value='" + SPHttpUtility.HtmlEncode(valueToEncode) + "'/>";
        this.Page.Form.Controls.Add(literal3);
    }
}

In SharePoint 2013, the behavior is “almost” identical:

private void EmitHiddenWebPartZone()
{
    if (ScriptManager.GetCurrent(this.Page) != null)
    {
        SPWebPartManager sPWebPartManager = this.SPWebPartManager;
        WebPartZone webPartZone = new WebPartZone {
            ID = "wpz",
            PartChromeType = PartChromeType.TitleOnly
        };
        if (((this.Context == null) || (this.Context.Request == null)) || ((this.Context.Request.Form == null) || (string.IsNullOrEmpty(this.Context.Request.Form["wpcmVal"]) && string.IsNullOrEmpty(this.Context.Request.Form["_wpcmWpid"]))))
        {
            webPartZone.AllowLayoutChange = false;
        }
        sPWebPartManager.Register(webPartZone);
        this.wpz = webPartZone;
        Literal child = new Literal();
        Literal literal2 = new Literal();
        child.Text = "<div style='display:none' id='hidZone'>";
        literal2.Text = "</div>";
        UpdatePanel panel = null;
        if (SPUtility.ContextCompatibilityLevel < 15)
        {
            panel = new UpdatePanel {
                ID = "panelZone",
                UpdateMode = UpdatePanelUpdateMode.Conditional
            };
            panel.ContentTemplateContainer.Controls.Add(child);
            panel.ContentTemplateContainer.Controls.Add(webPartZone);
            panel.ContentTemplateContainer.Controls.Add(literal2);
        }
        Control control = string.IsNullOrEmpty(base.MainContentID) ? null : this.Page.Form.FindControl(base.MainContentID);
        if (control != null)
        {
            if (SPUtility.ContextCompatibilityLevel >= 15)
            {
                control.Controls.Add(child);
                control.Controls.Add(webPartZone);
                control.Controls.Add(literal2);
            }
            else
            {
                control.Controls.Add(panel);
            }
        }
        else if (SPUtility.ContextCompatibilityLevel >= 15)
        {
            this.Page.Form.Controls.Add(child);
            this.Page.Form.Controls.Add(webPartZone);
            this.Page.Form.Controls.Add(literal2);
        }
        else
        {
            this.Page.Form.Controls.Add(panel);
        }
        if (SPUtility.ContextCompatibilityLevel < 15)
        {
            this.panelZone = panel;
        }
        string hiddenFieldInitialValue = string.Empty;
        if (((this.Context != null) && (this.Context.Request != null)) && (this.Context.Request.Form != null))
        {
            hiddenFieldInitialValue = this.Context.Request.Form["wpcmVal"];
        }
        if (hiddenFieldInitialValue == null)
        {
            hiddenFieldInitialValue = string.Empty;
        }
        SPPageContentManager.RegisterHiddenField(this.Page, "_wpcmWpid", "");
        SPPageContentManager.RegisterHiddenField(this.Page, "wpcmVal", hiddenFieldInitialValue);
    }
}

The most significant difference (at least for the purpose of this post) are the lines below:

Control control = string.IsNullOrEmpty(base.MainContentID) ? null : this.Page.Form.FindControl(base.MainContentID);
if (control != null)
{
    if (SPUtility.ContextCompatibilityLevel >= 15)
    {
        control.Controls.Add(child);
        control.Controls.Add(webPartZone);
        control.Controls.Add(literal2);
    }
    else
    {
        control.Controls.Add(panel);
    }
}

Which means “look for a control with an ID of [base.MainContentID] and, if you find it, add the Web Part Zone as a child of that control, otherwise, add the control as a child of the Form control”.

Now, the base class in this case is DeltaPage, and its MainContentID property is set to “PlaceHolderMain” in the DeltaPage instance constructor.

The primary (maybe the only?) reason for this modification is to support the new Minimal Download Strategy feature: the WebPartPage class adds dynamic controls to the PlaceHolderMain, rather than to the Form control, so that they become part of the Delta content associated with it.

As you can see, though, this may introduce some side effects

Let’s try so sum everything up:

  1. Server-side code blocks cannot be inserted inside a control, if the control tree of that control is changed by some other code (i.e. code behind of the page)
  2. In SharePoint 2010, the WebPartPage class manipulates the Form control tree
  3. In SharePoint 2013, the WebPartPage class may, under some circumstances, manipulate the control tree of the PlaceHolderMain control

This means that if:

  1. You have site pages with code blocks messing the page markup
  2. Code blocks are allowed for that page:
    1. Either your page is running in ghosted mode (the page and the master page are both ghosted)…
    2. … or you have excluded the page from Safe Mode execution

Then:

  1. Your page runs fine in SharePoint 2010
  2. Your page will break once migrated to SharePoint 2013 (no matter of the compatibility level the site is running under)

The solution(s)

If you have been so patient to read so far, I guess you deserve some hint about a possible solution to this issue.

I have some, indeed.

#1: Surround the code blocks with a server-side control (i.e. a <div runat=”server” />, a Panel, whatever). This way the control tree of the PlaceHolderMain will not be changed, since the code blocks have been moved to an inner control (btw, credits to Peppe for having found and applied this workaround on a project we have been working together some weeks ago). This is probably the better approach, although it may be a time consuming solution, since you are required to modify the markup of every single page that has code blocks, possibly in several different places.

#2: Do not inherit from WebPartPage, if you don’t need to. This may seem more a trick than a real solution (and… yes, it is), but it’s a quick and dirt approach that you may take into account when you have several pages and when none of these need to inherit from WebPartPage 🙂

#3: Set the MainContentId property to some nonexisting identifier (yes, another trick)

#4: Substitute code blocks with expressions, if possible (taking this to a larger extent, you may even write an expression builder that executes code blocks, but think that it’s a little bit too… expensive :))

And… I have to say #5: Get rid of code blocks. Period.

I would definitely vote for #5 (see below).

The conclusion

You may say that this issue is quite uncommon.

Yes, hopefully this is quite uncommon… but I have seen this a couple of times so far 🙁

I’m saying hopefully because, IMHO, messing the page markup with code blocks is not a nice approach even in plain ASP.NET implementations.

And more, talking about SharePoint, code blocks are not allowed for pages running in Safe Mode, which means all customized pages.

You can relax this constraint, but you open up to big security concerns, so this should be definitely discouraged 🙂

Migration to SharePoint 2013: Unable to change the Master Page of a Publishing Portal

A few days ago I was supporting a customer during the migration of the Corporate Intranet (SharePoint 2010) to SharePoint 2013.

One of the “manual configurations” that we needed to apply after the attach&mount step was to change the master page of a bunch of Team Sites, all having the Publishing Features activated.

Strangely enough, this operation succeeded on most of these sites but one, where we got the dreaded “Unexpected error”.

This error went away after performing a Version Upgrade of the site collection.

Mmhh…. I started investigating 🙂

I was just curious, so I opened up the page markup with the intention to ensure that the behavior of this Application Page was the one I expected.

Now, the “magic” of Compatibility Levels relies on having two versions of the SharePoint Root Hive (14 and 15), and on the SharePoint modules to pick up the correct version of pages and resources based on the Compatibility Level of the site collection.

Therefore, I started comparing the two versions of ChangeSiteMasterPage.aspx.

If you browse the14 version of this page, you should see something similar to the picture below:

image

Whereas this is the corresponding “15” version:

image

As you may notice, these pages are a little bit different:

  1. The 15 version supports Device Channels, which the 14 versions obviously does not (hence the difference in the Site Master Page and the System Master Page sections)
  2. The 15 version supports the propagation of Theme settings to the child webs,  which the 14 versions obviously does not (hence the new Theme section)

See, for example, the following code snippet (representing the Theme section) that is defined only in the 15 version of the page:

<!-- Theme inheritance section -->
            <wssuc:InputFormSection Title="<%$Resources:cms,areachromesettings_themeinheritance_header%>"
                Description="<%$Resources:cms,areachromesettings_themeinheritance_description%>"
                Collapsible="true"
                Collapsed="true"
                runat="server">
                <Template_InputFormControls>
                    <wssuc:InputFormControl runat="server">
                        <Template_Control>
                            <table>
                                <wssawc:InputFormCheckBox ID="inheritThemeCheckbox" LabelText="<%$Resources:cms, areachromesettings_themeinheritance_inherittheme_checkboxtext%>" runat="server"/>
                            </table>
                            <table>
                                <wssawc:InputFormCheckBox ID="resetThemeSubSitesCheckBox" LabelText="<%$Resources:cms, areachromesettings_themeinheritance_resetsubsite_checkboxtext%>" runat="server"/>
                            </table>
                        </Template_Control>
                    </wssuc:InputFormControl>
                </Template_InputFormControls>
            </wssuc:InputFormSection>

 

Now… this is perfectly fine, as long as any logic in the code behind takes the Compatibility Level into account when performing operation on the UI (i.e. referencing controls). This is typically done by checking the CompatibilityLevel property of the SPSite object:

private void InitThemeInheritanceControls(bool isInheriting, bool isRoot, CheckBox inheritCheckbox, CheckBox resetSubSitesCheckbox)
{
    if (base.Site.CompatibilityLevel >= 15)
    {
        if (isRoot)
        {
            isInheriting = false;
            inheritCheckbox.Enabled = false;
        }
        inheritCheckbox.Checked = isInheriting;
        resetSubSitesCheckbox.Checked = false;
    }
}

 

But when no check is performed against the compatibility level, the code behind should rely only on features/controls that are defined in both version of the page, right?

Now, take a look at the OnLoad event handler of the AreaChromeSettingsPage, which is the code behind class for ChangeSiteMasterPage.aspx (see Microsoft.SharePoint.Publishing.Internal.CodeBehind.AreaChromeSettingsPage in
Microsoft.SharePoint.Publishing, Version=15.0.0.0):

protected override void OnLoad(EventArgs e)
{
    base.OnLoad(e);
    this.EnsureChildControls();
    if (!this.Page.IsPostBack)
    {
        this.LoadValues();
        if (base.Web.Webs.Count == 0)
        {
            this.resetSystemMasterPageSubSitesCheckBox.Visible = false;
            this.resetSubSitesCheckBox.Visible = false;
            this.resetAlternateCssSubSitesCheckBox.Visible = false;
            this.resetThemeSubSitesCheckBox.Visible = false;
        }
    }
    base.ConfigureCancelButton(this.BtnCancel);
}

 

See what I mean?

This snippet is supposed to disable a bunch of controls (the Check Boxes that allow you to apply the new settings to all child webs) if the context web has no child web at all.

Unfortunately, there’s no CompatibilityLevel check, so the following line:

this.resetThemeSubSitesCheckBox.Visible = false;

will fail if:

  1. The Site is in 14 mode
  2. The Site has child webs

That was exactly my case.

You can easily reproduce this behavior:

  1. Create a new, out-of-the-box Publishing Portal on a SP2010 box
  2. Migrate it to a SP2013 farm
  3. Do not perform any Version Upgrade and try to navigate to the “Change Master Page” page. It should work fine, since the default SP2010 Publishing Portal has a couple of subwebs (Search and Press Releases)
  4. Delete both subwebs and try to load the page again. The page should be broken now

Funny, isnt’t it? 🙁