20 July 2014

301 Redirect Handling

In previous post we saw that we could create a Custom Field and store values we wanted in it. In our case we took a simple example to store data for a 301 redirect. We saw the use of a button on the field to regenerate the data whenever we wanted... Also for that example we chose to store the data in Json format. This was totally arbitratory and you can choose to store your data in whatever format you need. Well let's try to finish the code to see how we can integrate that plus pipeline action (like here) to handle the 301 redirects.

1- We need to define our templates

We will need 3 templates
  • The Redirection Status: 301, 302... this will help us to define the status code and the status text to pass to the Response. The template will looks like:


  • The Domain Redirection Item: This item will store the Json list objects as per our previous post as well as the domain we want the redirection to be applicable. Here we are thinking about 301 for multi-sites. Indeed, if your sitecore instance has multiple sites and you are setting up a redirection for /applications, this might be a valid  redirection for Site 1 but /applications could exist on Site 2... So better to plan for it. Now, when we had a look at the Domain Dictionary, we saw that in the configuration of your site you will have the attribute "dictionaryDomain". well we will use the same attribute to find our site domain for redirection. The template should look like the following (note the second field is our custom field):

  • Finally we will need a Redirect Item template. This template will have 3 fields: the requested URL (single line text), Redirect To: droptree that can point to a media or content item... and Status Code: droptree pointing at our Redirection Status folder as per above image...\

 2- Custom Field

well we already saw that part... under our "Redirections", we will have our Domain Item that will store the domain and the Json Data... Under this domain item, we will define all the redirection items.
Our Custom Field, will have a button which will trigger an action to get all the children and build the Json List of objects... You can follow this post to setup the field.

So we now that the data we are going to use on the HTTP Request Begin pipeline is stored on the domain item, so theoretically, we just need to retrieve this item, get the data, deserialise and get the relevant redirect if any... then proceed with redirection

3- Configuration

Easy part, we just need a processor on the HTTPRequestBegin Piepline:

      < httpRequestBegin>
        < processor type="MySite.Business.Sitecore.Pipelines.PermanentRedirect, MySite.Business" patch:after="processor[@type='Sitecore.Pipelines.HttpRequest.ItemResolver, Sitecore.Kernel']" />
      < /httpRequestBegin>

4- Code

Now we have our data, we have our pipeline configure, let's intercept the request and handle some 301 shall we?
Create your class PermanentRedirect based on the HttpReqiestProcessor...

namespace MySite.Business.Sitecore.Pipelines
{
    /// 
    /// This processor will check for 301 redirects
    /// 
    public class PermanentRedirect : HttpRequestProcessor
    {
        public override void Process(HttpRequestArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            if (Context.Database == null || args.Url.ItemPath.Length == 0 || Context.Site == null)
                return;
        }
    }
}

Now let's think about what we will need:

we will need a method to set the Response: status, status code, header for the location to redirect to...
So let's create that method. Don't forget, you can modify the response from the Args.Context:

        /// 
        /// Set the Header for the reponse
        /// 
        /// 
        /// 
        /// 
        /// 
        private void SetResponse(string redirectTo, string status, int statusCode, HttpRequestArgs args)
        {
            args.Context.Response.Status = status;
            args.Context.Response.StatusCode = statusCode;
            args.Context.Response.AddHeader("Location", redirectTo);
            args.Context.Response.End();
        }

The next thing we will need is a URL resolver. If you remember on our Json object we will store the RedirectTo value as Sitecore ID (to either a content or a media). So we would need a method to get the item and get either the content URL or a media URL if it is pointing at a document... Notes, that we could have tried to store the nice URL on the Json object but the jSon object is generate from the Custom Field button which would have the Sitecore Context Site as Shell so you would have to pass your website context as UrlOptions. Also you want to check that the item you are redirecting to exsits in your delivery site so it is always better to resolve this item on the pipeline action...

        /// 
        /// This will resolve either the media URL or the Content URL
        /// 
        /// 
        /// 
        private string GetSitecoreUrl(Item redirectToItem)
        {
            string mediaLibraryPath = Settings.GetSetting(SitecoreSettings.MediaPath).ToLower();

            // if the item is not a media item
            if (!redirectToItem.Paths.Path.StartsWith(mediaLibraryPath))
            {
                return LinkManager.GetItemUrl(redirectToItem);
            }

            // media item
            var mediaItem = (MediaItem)redirectToItem;
            var mediaUrl = MediaManager.GetMediaUrl(mediaItem);
            return StringUtil.EnsurePrefix('/', mediaUrl);

        }


So now we have our methods, we can start with the main business logic. The first thing we want is to get the root item for our redirection. This root item is the parent of all the Domain Redirection Items. This will allow us to go retrieve all Domain Redirection Items, and select only the ones matching our context.Site.DictionaryDomain:

            String domain = Context.Site.DictionaryDomain;
            if (string.IsNullOrEmpty(domain))
                return;

            // Root for the redirect folder
            Item rootFolder = Context.Database.GetItem(new SC.Data.ID(ItemId.System.Redirection.RedirectionRootFolder));
            if (rootFolder == null)
                return;

            if(!rootFolder.HasChildren)
                return;

            var validDomainItems = rootFolder.Children.Where(x => x.TemplateID.ToGuid() == new Guid(TemplateIds.System.RedirectionDomain)
                                                && x[FieldIds.System.RedirectionDomain.SiteDomain].ToLower() == domain.ToLower());


            if (!validDomainItems.Any())
                return;


Finally we can now loop through all the valid Domain Redirect Item, Get the Json Data field, Deserialize and find the relevant item to redirect to:

                    List redirectItems = (List)Newtonsoft.Json.JsonConvert.DeserializeObject(dataField, typeof(List));
                    if(redirectItems.Any(x=> x.redirectFrom.ToLower().Contains(requestedPath.ToLower())))
                    {
                        RedirectItem item = redirectItems.First(x =>  x.redirectFrom.ToLower().Contains(requestedPath.ToLower()));

                        // Only get the URL at this point to ensure permission, site context...
                        var redirectToItem = SC.Context.Database.GetItem(item.redirectTo);
                        if (redirectToItem == null)
                            continue;

                        string redirectToUrl = this.GetSitecoreUrl(redirectToItem);

                        SetResponse(redirectToUrl, item.status, item.statusCode, args);
                        return;
                    }



So all together, that could looks like something:
namespace MySite.Business.Sitecore.Pipelines
{
    /// 
    /// This processor will check for 301 redirects
    /// 
    public class PermanentRedirect : HttpRequestProcessor
    {
        public override void Process(HttpRequestArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            if (Context.Database == null || args.Url.ItemPath.Length == 0 || Context.Site == null)
                return;

            if (!Settings.GetSetting(SitecoreSettings.ValidSites).ToLower().Contains(Context.Site.Name))
                return;

            String domain = Context.Site.DictionaryDomain;
            if (string.IsNullOrEmpty(domain))
                return;

            // Root for the redirect folder
            Item rootFolder = Context.Database.GetItem(new SC.Data.ID(ItemId.System.Redirection.RedirectionRootFolder));
            if (rootFolder == null)
                return;

            if(!rootFolder.HasChildren)
                return;

            var validDomainItems = rootFolder.Children.Where(x => x.TemplateID.ToGuid() == new Guid(TemplateIds.System.RedirectionDomain)
                                                && x[FieldIds.System.RedirectionDomain.SiteDomain].ToLower() == domain.ToLower());


            if (!validDomainItems.Any())
                return;

            // get the actual request
            var requestedUrl = HttpContext.Current.Request.Url.ToString();
            var requestedPath = HttpContext.Current.Request.Url.AbsolutePath;

            // Get XML from the field... 
            foreach (var redirectDomainItem in validDomainItems)
            {
                string dataField = redirectDomainItem[FieldIds.System.RedirectionDomain.RedirectionData];
                if (string.IsNullOrEmpty(dataField))
                    continue;

                try
                {
                    //Deserialize the List
                    List redirectItems = (List)Newtonsoft.Json.JsonConvert.DeserializeObject(dataField, typeof(List));
                    if(redirectItems.Any(x=> x.redirectFrom.ToLower().Contains(requestedPath.ToLower())))
                    {
                        RedirectItem item = redirectItems.First(x =>  x.redirectFrom.ToLower().Contains(requestedPath.ToLower()));

                        // Only get the URL at this point to ensure permission, site context...
                        var redirectToItem = SC.Context.Database.GetItem(item.redirectTo);
                        if (redirectToItem == null)
                            continue;

                        string redirectToUrl = this.GetSitecoreUrl(redirectToItem);

                        SetResponse(redirectToUrl, item.status, item.statusCode, args);
                        return;
                    }
                }
                catch (Exception ex)
                {
                    SC.Diagnostics.Log.Warn("Could not deserialize the Json for redirection Object - ex: " + ex.StackTrace , this);
                }
 
            }
        }

        /// 
        /// Set the Header for the reponse
        /// 
        /// 
        /// 
        /// 
        /// 
        private void SetResponse(string redirectTo, string status, int statusCode, HttpRequestArgs args)
        {
            args.Context.Response.Status = status;
            args.Context.Response.StatusCode = statusCode;
            args.Context.Response.AddHeader("Location", redirectTo);
            args.Context.Response.End();
        }

        /// 
        /// This will resolve either the media URL or the Content URL
        /// 
        /// 
        /// 
        private string GetSitecoreUrl(Item redirectToItem)
        {
            string mediaLibraryPath = Settings.GetSetting(SitecoreSettings.MediaPath).ToLower();

            // if the item is not a media item
            if (!redirectToItem.Paths.Path.StartsWith(mediaLibraryPath))
            {
                return LinkManager.GetItemUrl(redirectToItem).Replace("/site content", "");
            }

            // media item
            var mediaItem = (MediaItem)redirectToItem;
            var mediaUrl = MediaManager.GetMediaUrl(mediaItem);
            return StringUtil.EnsurePrefix('/', mediaUrl);

        }
    }
}



You can now try to call your redirect URL and see the redirection happening...

7 July 2014

Custom FIeld


Today I just wanted to show how you can create a custom field. Even if you will not really use this everyday. I found it quite handy from time to time. You can use custom fields for lots of reason: getting data from third party (Youtube, Google Map...) and store it in the field. You can also use it to display your field value in a certain way: preview your video... I guess it all depends on where your imagination can take you.

So on this example, I did not wanted to tackle something too tricky and connecting to third party... So what I was thinking about doing something like having this field to store a list of Json Object that could be used for a 301 redirect... Let say for example that you want to create a 301 redirect system that would look like:


Ok, so obviously when you will create your pipeline action to intercept the HttpRequestBegin to resolve the 301, you could just loop through all the items and find the item relevant to your request. Well, it is totally possible. Also considering a full site rebuild with a large amount of pages you would need to loop through 1000 links (I made up the number...). Anyway for our purpose today, the approach we will take is to define the list of JSon objects that will represent all our redirect items and store it on the parent item in our custom field. In our pipeline action we will then need to get the field value and deserialize the data and find the relevant redirect item. Here is how the field would appear


Well as you can image, this list will not be updated unless a content editor add a new item... That is why in that case a custom field is quite handy... especially with a nice little button at the top to regenerate the data when you will need to.

1- Defining the field type.

 To define the field type, you need to go to the CORE DB in the desktop. Under the System folder, you need to locate the item: Field Type. Create a new folder underneath which you can call Custom Field Types.
Under this newly created folder, you can create an Item from template Template field type. You can see the value to the control field: myfields:JsonField


2- Defining the menu item: button appearing at the top of the field

To define the menu item, you need to create a new folder under the field type you just created. The template to use is /Common/Folder. Under this folder, you will need to create a anew item from the template Menu item. The Display name field will be used for the text that will be displayed on the button. Also note the value placed on the Message field: myfields:regeneratejson.



3- Configurations

As you may have noticed we created some reference to controls: myfields:...
We now need to map it to the code we will write. So create a new config file in the Include folder. In this configuration file, we are going to define the Control to point it to a .net class that we will create.

    < controlsources>
      < source assembly="MySite.Business" mode="on" namespace="MySite.Business.Sitecore.Fields" patch:after="source[@namespace='ComponentArt.Web.UI']" prefix="myfields" />
    < /controlsources>


As you can see the myfields prefix will be mapping to the Namespace MySite.Business.Sitecore.Fields. Now, I am sure you are wondering where is the class defined. Well "myfields:JsonField", your .NET class will need to be called JsonField.

4- our code

OK, now we have the config in place, we can create our class... Namespace is defined above and the class name: JsonField. The field will be quite simple: we just need a free text on which will store Text... If you look into the namespace: Sitecore.Shell.Applications.ContentEditor you will some base control that you can base your control on. In our case we will choose to Sitecore.Shell.Applications.ContentEditor.Text as base type. Our first draft of the code will be similar to

namespace MySite.Business.Sitecore.Fields
{
    public class JsonField : SC.Shell.Applications.ContentEditor.Text
    {
        protected override bool LoadPostData(string value)
        {
            value = SC.StringUtil.GetString(new string[1]
            {
                value
            });

            if (this.Value == value)
                return false;

            this.Value = value;

            return true;
        }
    }
}


you can verify that you field display as a text box and the value you place in the field will be saved correctly.

5- Button action

The button action. Well in the definition of the menu item (button), we defined the Message field: myfields:regeneratejson
If you decompile the code for the base control: Sitecore.Shell.Applications.ContentEditor.Text, you will see that there are a few method you can override. One of them is the HandleMessage. This will allow you to intercept the regeneratejson coming from the button and redirect your code to your own method to hande the action:

        public override void HandleMessage(Message message)
        {
            base.HandleMessage(message);
            switch (message.Name)
            {
                case "myfields:regeneratejson":
                    this.RegenerateData();
                    break;
            }
        }


In our case, you can note that we wanted to loop through all children and serialised the data to build our Json list of redirect item... So first, we need to ensure we can access the actual current Item. This can be done as per the following:

        public string ItemID
        {
            get
            {
                return base.GetViewStateString("ItemID");
            }
            set
            {
                Assert.ArgumentNotNull(value, "value");
                base.SetViewStateString("ItemID", value);
            }
        }
Now we have the item we can build our method to loop through children and build our serialisation. Here is the full code:

namespace MySite.Business.Sitecore.Fields
{
    public class JsonField : SC.Shell.Applications.ContentEditor.Text
    {
        public string ItemID
        {
            get
            {
                return base.GetViewStateString("ItemID");
            }
            set
            {
                Assert.ArgumentNotNull(value, "value");
                base.SetViewStateString("ItemID", value);
            }
        }

        public override void HandleMessage(Message message)
        {
            base.HandleMessage(message);
            switch (message.Name)
            {
                case "myfields:regeneratejson":
                    this.RegenerateData();
                    break;
            }
        }

        protected void RegenerateData()
        {
            var masterDB = SC.Data.Database.GetDatabase("master");
            if (masterDB == null)
            {
                SheerResponse.ShowError("Could not get master DB", "Error: Could not retrieve the Master Database");
            }

            var item = masterDB.GetItem(ItemID);
            if (item == null)
            {
                SheerResponse.ShowError("Could not resolve the current Item", "Error: Could not resolve the current Item");
            }

            if(!item.HasChildren)
                return;

            List list = new List();
            foreach (var child in item.Children)
            {
                RedirectItem rItem = new RedirectItem((Item)child);
                if (rItem.isValid)
                {
                    list.Add(rItem);
                }
            }

            string serialization = Newtonsoft.Json.JsonConvert.SerializeObject(list);

            this.Value = serialization;
        }

        protected override bool LoadPostData(string value)
        {
            value = SC.StringUtil.GetString(new string[1]
            {
                value
            });

            if (this.Value == value)
                return false;

            this.Value = value;

            return true;
        }
    }
}


You can now try to view the field and click on the button:


3 July 2014

SPIF error when synchronising documents


In a recent project, we had a Sitecore/Sharepoint integration task. After looking at the different options, we have decided to use the SPIF. Looking at the documentation, that would definitely do what we needed and provided even more API possibilities.

So we started the implementation and everything looked quite nice:
  • Connecting to the Sharepoint server
  • Create the Sharepoint connector item in Sitecore
  • And even retrieving the documents.
Unfortunately, we encountered an error:

For one of the library we were trying to sync, none of the document were getting into sitecore. After looking at the log file  we had the following exception:

ManagedPoolThread #10 00:00:00 ERROR Sharepoint Provider can't process tree.
Integration config item ID: {23C8604D-6761-4FD8-B3AD-A0E3BB880AD8}, Web:http://sharepoint.test.com/ List: {7C8A645E-A085-46DE-A059-168B8F480641}
Exception: System.InvalidOperationException
Message: There is an error in XML document (44, 2760).
Source: System.Xml
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Sitecore.Sharepoint.Data.WebServices.SharepointLists.Lists.GetListItems(String listName, String viewName, XmlNode query, XmlNode viewFields, String rowLimit, XmlNode queryOptions, String webID)
   at Sitecore.Sharepoint.ObjectModel.Connectors.ItemCollectionConnector.GetItems(BaseList list, ItemsRetrievingOptions options)
   at Sitecore.Sharepoint.ObjectModel.Entities.Collections.ItemCollection.GetEntities()
   at Sitecore.Sharepoint.Data.Providers.SharepointProvider.ProcessTree(ProcessIntegrationItemsOptions processIntegrationItemsOptions, SynchContext synchContext)
   at Sitecore.Sharepoint.Data.Providers.SharepointProvider.ProcessTree(ProcessIntegrationItemsOptions processIntegrationItemsOptions, Item integrationConfigDataSource)
 
Nested Exception
 
Exception: System.Xml.XmlException
Message: '[1]', hexadecimal value 0x02, is an invalid character. Line 44, position 2760.
Source: System.Xml
   at System.Xml.XmlTextReaderImpl.Throw(String res, String[] args)
   at System.Xml.XmlTextReaderImpl.ParseNumericCharRefInline(Int32 startPos, Boolean expand, StringBuilder internalSubsetBuilder, Int32& charCount, EntityType& entityType)
   at System.Xml.XmlTextReaderImpl.ParseNumericCharRef(Boolean expand, StringBuilder internalSubsetBuilder, EntityType& entityType)
   at System.Xml.XmlTextReaderImpl.HandleEntityReference(Boolean isInAttributeValue, EntityExpandType expandType, Int32& charRefEndPos)
   at System.Xml.XmlTextReaderImpl.ParseAttributeValueSlow(Int32 curPos, Char quoteChar, NodeData attr)
   at System.Xml.XmlTextReaderImpl.ParseAttributes()
   at System.Xml.XmlTextReaderImpl.ParseElement()
   at System.Xml.XmlTextReaderImpl.ParseElementContent()
   at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace)
   at System.Xml.XmlLoader.ReadCurrentNode(XmlDocument doc, XmlReader reader)
   at System.Xml.XmlDocument.ReadNode(XmlReader reader)
   at System.Xml.Serialization.XmlSerializationReader.ReadXmlNode(Boolean wrapped)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderLists.Read30_GetListItemsResponse()
   at Microsoft.Xml.Serialization.GeneratedAssembly.ArrayOfObjectSerializer53.Deserialize(XmlSerializationReader reader)
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)

That kept us going for days. Sitecore support and Sitecore team help us debugging the issue and provided us with a fix. Basically what happen is, there are some invalid chars (for XML parsing) on the documents we are trying to retrieve from Sharepoint. This was causing the XML parsing to fail and none of the documents were getting created in Sitecore. You can find the issue with the resolution in the knowledge base item.

Solution: 1- download the dll patch from the knowledge base item 2- Modify the web.config file by adding the following configuration into the < system.web> section:
< webservices> 
  < soapextensiontypes> 
    < add group="0" priority="1" type="Sitecore.Support.SharePoint.XmlCleanupSoapExtension, Sitecore.Support.411647" /> 
  < /soapextensiontypes> 
< /webservices>