SPSocialFeedManager.GetFeed: Exception: System.UriFormatException: Invalid URI: The hostname could not be parsed.

Today I fixed the second part of one of the craziest SharePoint 2013 issues of my year.

This issue to begin with only exists until January CU 2017 of SharePoint 2013. Since we are rapidly approaching 2019 I hope most of your farms are already past this update level.

However I have one customer that is still on a 2016 patch level: 15.0.4815.1000 (April 2016 CU).

As usual, we are currently in the process of updating and QA and UAT are already patched. However I was able to get my hands on a machine with the same patch level as production and was able to reproduce the issue.

The issue is extremely rare and to my shame I didn’t find it, a colleague did. Take note of the fixed issues list of KB3141477 (January CU 2016 for SharePoint Server 2013).

When you create a new newsfeed post that includes an app URL with an incorrect protocol (http:// instead of https://), the newsfeed page is broken and it doesn’t display news. This update ensures any problem in one of the news item should not block from rendering the newsfeed page.

What this means as an example: Somebody installs a sharepoint-hosted app (add-in) in your environment. Somebody copies the address of the app (https://app-%5BGUID%5D.%5Bappdomain%5D) or any link starting with this into your SharePoint 2013 microfeed. But the important part is, he/ she removes the s from the ssl-protocal before pasting. This works in the other direction as well of course (you have an app host with http:// and he/ she adds an “s” after “http”).

The occurring issue is that your user will receive a message that this didn’t work as expected “we’ve hit a snag”.

The user will click on “OK” and think he/ she is done with it. However far from the truth. Refreshing the page will show a corrupted microfeed with a cryptic message (which is not so cryptic, if you know what’s up…but more later…)

Something went wrong
TECHNICAL DETAILS
SharePoint returned the following error: Invalid URI: The hostname could not be parsed. Contact your system administrator for help in resolving this problem.

So now what?

The ULS logs will tell you a bit more of course, but not sure if you will make the connection to delete the item from the microfeed list.

Request for app scheme does not match the webapp's scheme for this zone. Request Uri: http://app-a5ced8ff740661.apps-...
Request for app scheme does not match the webapp's scheme for this zone. Request Uri: http://app-a5ced8ff740661.apps-...
SPSocialFeedManager.GetFeed: Exception: System.UriFormatException: Invalid URI: The hostname could not be parsed.     at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind)...

So you see this and you don’t know what it means. Well as mentioned above it has to do with the fact that somebody added a link in the microfeed and used the wrong protocol for the app domain. So you will go to the microfeed list and see that only the farm administrator account can actually delete items there with the right permission of “Social Data” in the User Profile Service. You can use this script to delete them via powershell (if you have server access):

Add-PSSnapin Microsoft.SharePoint.Powershell -ea 0;

$site = "";
$itemIds = @();

$spweb = get-spweb $site;
$microfeed = $spweb.lists["MicroFeed"];
$itemId = $null;
foreach( $itemId in $itemIds ) {
    if( $itemId -ne $null ) {
        Write-Host ("ItemId: " + $itemId);
        $item = $null;
        $item = $microfeed.GetItemById($itemId);

        $item.Delete();
    }
}
$spweb.Dispose();

Once you delete this item, you will find the feed on the site itself is fine (after a refresh!). However if in the meantime you have been on the mysite host and checked your stream and you have been following this particular site, where the person added the url you will find that feed broken as well. You might be lucky like me and it’s not. But some users might still be affected. You can stop following that room but that will probably not be a solution for users productively using SharePoint.

Why is the error still there even though the item has already been deleted? Caching. If you want to delete your browser cache and try again you will find this doesn’t help. You will then try to clear the timer cache or do an iisreset. Still no dice. What actually helps is to clear the DISTRIBUTED CACHE (appfabric).

You can do that via the following script, which targets specifically only the activity feed container:

Add-PSSnapin Microsoft.SharePoint.Powershell -ea 0;
Clear-SPDistributedCacheItem -ContainerType DistributedActivityFeedCache;

That will help. You can find more information on this command here. What didn’t help for me which was a red herring were Update-SPRepopulateMicroblogFeedCache and Update-SPRepopulateMicroblogLMTCache. Btw the required parameters for the call Update-SPRepopulateMicroblogFeedCache change between April 2016 CU and July 2018 CU.

So in summary:

If you find you have this similar issue: Don’t waste your time. Check your farm version, delete the list item, flush the cache. Plan to update to a recent update, asap. I hope two professional days of my life turn out to be just minutes for you.

Advertisements

Renew Certificate in Provider Hosted Apps Scenario

With a certain customer of mine I recently had an issue, where in the span of a month all of the certificates for the Provider Hosted Apps Domain (PHA) had to be renewed for four staging environments (including PROD).

I was lucky to be the successor of somebody who made the same mistake as the service provider a month later, so I was prepared and could save the day. In hopes of saving you the time it took for me to force the service provider’s hand (around 10 hours telco time) I want to give you a brief overview of how to tackle this, the full list of reference articles and a script to set the sharepoint part (trust).

First a short introduction.

Certificates. Certificates are often used to encrypt data communication between machines. This is done to make sure that two parties can communicate without a third party listening. Also this is done to verify the identity of somebody initiating communication.

In the scenario of SharePoint and PHA we have two parties. We have the PHA Server Farm and the SharePoint Server Farm. Usually each farm consists of at least 2 servers for redundancy/ high availability reasons.

When HTTP communication is done via SSL each WebSite in IIS has a binding on port 443, which uses a certificate for encrypting the data he site responds with to requests.

Any admin can swap the certificate in IIS. All you need to do is check the certificate that exists and request a new certificate either self-signed, internally trusted or externally trusted with the correct SAN (Subject Alternative Name).

As an example, let’s suggest the following setup:
SharePoint has a wildcard certificate, like *.apps.mycompany.com. The PHA environment has a certificate corresponding to this in apps.mycompany.com. This may be the same certificate, if you request the big kahuna, i.e. a multi-san, wildcard certificate. Usually this is not the case, and is not necessary.

The PHA IIS will have the apps.mycompany.com certificate, and SharePoint will have the wildcard certificate. However how does SharePoint make sure, that PHAs are not added to different server and this server has different code and pretends to be the PHA server? There is a trust between these servers on the SharePoint side. In essence this article has one message: “Don’t forget this trust!”

The underlying process of replacing the apps.mycompany.com certificate is based on four easy steps, all of them are necessary:

  1. Replace the apps.mycompany.com certificate in the IIS of each PHA server

    This is a no-brainer. Request the certificate, get the response, use certmgr.msc to import the certificate into the Personal Store of the Machine Account. Make sure to have a private key for the certificate. This can be self-signed, internally trusted or externally trusted (depending on your scenario, if you externalize your farm or not).

  2. Export the apps.mycompany.com certificate as pfx (with private key)

    Export it with private key (and password) and put it into the location, where the web.config of each Provider Hosted App can access it. Usually this certificate is stored in a central location on each IIS PHA Server.

  3. Export the apps.mycompany.com certificate as cer (without private key)

    Export it without private key and put it into a location on a SharePoint server, where you can access it from the SharePoint Powershell script in the next step.

  4. Replace the SharePoint trust via script

    The certificate (cer) is referenced in two locations in SharePoint (SPTrustedRootAuthority, STSTrustedSecurityTokenIssuer). You can set it in the SPTrustedRootAuthority by updating the object and by deleting the STSTrustedSecurityTokenIssuer object and recreating this with the correct IssuerName and RegisteredIssuerName ([Issuer GUID]@[Realm]). See Script below.

EDIT: This image differs from the code below. A crucial parameter is missing. line 29 must have the flag “-IsTrustBroker” as seen below. I wrote a specific article on this topic here


param (
[string] $CertificateSubjectAlternativeName = "apps.mycompany.com"
, [string] $CertificatePathLocation = "[MyDrive]:\[MyPath]\apps.mycompany.com.cer"
)

asnp microsoft.sharepoint.powershell -ea 0

$certificate = $null;
$certificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($CertificatePathLocation);

if($certificate -ne $null) {
$tra = $null;
$tra = Get-SPTrustedRootAuthority | ? { $_.Certificate.Subject.Contains(${CertificateSubjectAlternativeName}) }

if( $tra -ne $null ) {
$tra.Certificate = $certificate;
$tra.Update();
} else {
Write-Host -ForegroundColor Red “Error: No Certificate with SAN ‘${CertificateSubjectAlternativeName}’ found in Root Authority Store.”;
}

$sci = $null;
$sci = Get-SPTrustedSecurityTokenIssuer | ? { $_.SigningCertificate.Subject.Contains(${CertificateSubjectAlternativeName}) }

if( $sci -ne $null ) {
$regIssuerName = $sci.RegisteredIssuerName;
$issuerName = $sci.DisplayName;
$sci.Delete();
New-SPTrustedSecurityTokenIssuer -Name “${issuerName}” -RegisteredIssuerName “${regIssuerName}” -Certificate $certificate -IsTrustBroker;
} else {
Write-Host -ForegroundColor Red “Error: No Certificate with SAN ‘${CertificateSubjectAlternativeName}’ found in Trusted Security Token Issuer Store.”;
}
} else {
Write-Host -ForegroundColor Red “Error: Certificate not found at location ‘${CertificatePathLocation}’.”;
}

The last step, which is not mandatory, but we had to do it was on the IIS Servers of the PHA environment. The certificate gets cached by the UserProfile of the User running the app pool. Thus once you replace it is no longer able to find the file. This will be broadcasted by an ugly error like: ‘CryptographicException: The system cannot find the file specified.’

This is how to fix that: open IIS –> ApplicationPools –> DefaultAppPool –> “Right Click” –> Advanced Settings –> Load User Profile | set this value to “true”.

It seems a bit absurd to change this setting since it did not have to be set when configuring the PHA connection in the first place, but it does the trick.

Links:

Read more of this post

Unexpected Response from Server when updating SharePoint ListItem via JSOM

These days I am working a lot on the client-side of things. So a couple of months ago I started writing my first lines of JavaScript/ JSOM (Javascript (Client)-Side Object Model).

I wrote a small method to create list items in a list (listTitle) based on a collection/ list of properties (properties) and their respective values. Here it is:

function createListItem(listTitle, properties) {
    var d = $.Deferred();
    try {
        var ctx = SP.ClientContext.get_current();
        var oList = ctx.get_web().get_lists().getByTitle(listTitle);

        var itemCreateInfo = new SP.ListItemCreationInformation();
        oListItem = oList.addItem(itemCreateInfo);

        for (var i = 0; i < properties.length; i++) {
            var prop = properties[i];
            oListItem.set_item(prop.Key, prop.Value);
        }

        oListItem.update();
        ctx.load(oListItem);
        var o = { d: d, ListItem: oListItem, List: listTitle };
        ctx.executeQueryAsync(
        function () {
            o.d.resolve(o.ListItem);
        },
        function (sender, args) {
            o.d.reject("Could not create list item in list " + o.List + " - " + args.get_message());
        });
    } catch (Exception) {
        console.log(Exception.message);
    }
    return d.promise();
}

function updateListItem(listTitle, properties) {
  var d = $.Deferred();
  try {
        var d = $.Deferred();
        var ctx = SP.ClientContext.get_current();
        var oList = ctx.get_web().get_lists().getByTitle(listTitle);

        oListItem = oList.getItemById(id);

        for (var i = 0; i < properties.length; i++) {
            var prop = properties[i];
            try {
                oListItem.set_item(prop.Key, prop.Value);
            } catch (Exception) {
                console.log(prop.Key + ' ' + prop.Value);
                console.log(Exception.message);
            }
        }
        oListItem.update();
        //ctx.load(oListItem);
        var o = { d: d, ListItem: oListItem, List: listTitle, p: properties };
        ctx.executeQueryAsync(
            Function.createDelegate(o, 
        function () {
            o.d.resolve(o.ListItem);
        }),
        Function.createDelegate(o, 
        function (sender, args) {
            o.d.reject("Could not update list item in list " + o.List + " - " + args.get_message());
        }));
    } catch (Exception) {
        console.log(Exception.message);
    }
  return d.promise();
}

So this is what happened when I had this code execute on editing another item in a different list…

When I debugged using Chrome (my browser of choice when writing JavaScript – never used it before that, interestingly…) I received the error “unexpected response from the server”.

I figured out that there are two key lines in this code that can be the cause of this.

var ctx = SP.ClientContext.get_current();

and

ctx.load(oListItem);

In my case the first line was actually not responsible for the error message. For your reference if you use var ctx = new SP.ClientContext(url); you may encounter this error message. So make sure to check that.

You should always use the current client context, when using JSOM, similar to the best practice guidelines for opening webs on server-side (SSOM [Server-Side Object Model]).

In my case the second line was the cause for the issue.

When creating an item I need to load the item into the context afterwards (or the error will show up even if the item is created correctly).

When updating an item the item may not be loaded into the context afterwards (or the error will show up even if the item is updated correctly).

It kind of makes sense, because when creating an item you are actually sending an SP.ListItemCreationInformation to the server. When updating an item I already have my listitem object. Why would I need to load all the other information afterwards?

So once I removed the line from the update method the code no longer evaluated to fail and the error message disappeared.

So for the experts among you this may be old news, but I actually needed to think about this for a few minutes before I figured it out, so I thought it was well worth blogging about. Especially since I haven’t blogged for quite some time.

AppManagement and SubscriptionSettings Services, Multiple Web Applications and SSL

So currently I am setting up four environments of which one is production, 2 are staging and another is was a playground installation.

My staging environments (TEST, QA, PROD) are multi-server, multi-farm systems (multi-farm because the 2013 Farm publishes Search and UPA to existing 2010 Farms).
They are running SPS2013 Standard with March PU 2013 + June CU 2013. They will be using App Pool Isolation and App Management and SubscriptSettings Services have their own account (svc_sp{t, q, p}_app, i.e. svc_spt_app, svc_spq_app and svc_spp_app).

I have three web applications of which all are secured by SSL making a wildcard certificate necessary for the app domain. Each has their own account (svc_sp{t, q, p}_{col, tws, upa}). The reason for this is that I will be using Kerberos Authentication and for the SPNs I need dedicated accounts for each Application URL.

My playground was once a 4 server farm, but now 3 servers have been removed. It does not run the March PU 2013 nor June CU 2013. There app pool isolation wihtout SSL is used.

On the playground the app management worked well. I actually encountered my problem on my test first and tried to replicate on the playground, but couldn’t. But I am getting ahead of myself. The system was setup via autospinstaller and the necessary certificates and IPs involved were requested and implemented. The AD Team did the domain setup for me. I didn’t setup my environment following this article, but it is a good one to read. I also got the idea of creating a separate dummy web application for attaching my IIS Bindings and Certificate from it, which makes a lot of sense, because of security considerations and kerberos.

The first article to read to get an overview of what is necessary and what’s trying to be achieved can be found here.

So I set up everything and still it wasn’t working. What does that mean? I will explain. When I subscribe to an app, download it from the store and add it in a site collection of my choosing I get to click on it once it is finished installing. The link then leads me to my app domain. With SSL only when I was using the same application pools I could actually get anywhere, otherwise I say the below.

This is what I wanted to see:
Expected

This is what I saw on any of the web applications with SSL and that had a different app pool account than the one I was using for my dummy web application.
Actual

So this blank page is actually exactly what you see when you leave the request management service running on the frontends without doing any topology configuration.

So I tried to work with the user policy from the web application management page in hopes of giving the users permissions on the content databases. This was actually not happening as I found out later, but which was actually exactly what was needed. I had to manually add the account of the app pool for the app domain to the SPDataAccess Group of the content databases. Then it also works with SSL. I actually set up three web applications WITHOUT SSL on the Test Staging Environment with the same users as the SSL Web Applications and this worked like a charm, but for any SSL web application I needed to explicitly give permissions to the content database. This is a nightmare to maintain. For my migration of 20 databases from 2010 to 2013 I need to do this again and again and for each new content database I will create in the future. Just imagine you create a new content database and forget to do this. Now for any site collection in this content database the above issue will show up. Hard to debug later on.

Not sure what Microsoft is thinking here, but I am happy that it only took me 4 days to figure this one out.

web.config

Okay, so I have my dev machine, and I have the typical error: please change your setting of customError in the web.config to RemoteOnly or Off so you can see the error.

I tried a lot, but this is what finally did the trick:
change the setting in the 14 hive web.config:
C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\LAYOUTS\web.config

– I checked to make sure the spelling was right (it is case-sensitive).
– I added it at different parts of the config file (no difference, you will see there is only one correct place to put it).
– Changed it from Off to RemoteOnly to Off (no difference, if you are accessing locally)
– Changed it via web.config manually and via IIS (the settings seem to be synced, but in special cases it seems they are not).

Creating a UPA Proxy via PowerShell – technet article typo

http://technet.microsoft.com/en-us/library/ff607737.aspx

In the example it says:

$app = Get-SPServiceApplication -Name PartitionedUserProfileApplication. New-SPProfileServiceApplicationProxy -Name PartitionedUserProfileApplication_Proxy -ProfileServiceApplication $app -PartitionMode

But the Member is -ServiceApplication rather than -ProfileServiceApplication.

Tried it, worked. Keep in mind.

If they had a chance to add a comment, I would have done it there.

CheckedIn / Added events on libraries

So I had to do another event receiver for a wiki pages library. There are different versions depending on the development state – they are different in that the library either has a force checkout or not. See below image and then the lower of the two red rectangles shows the setting that is either yes or no. The other one defines if a simple approval workflow is used to manage versions.
Versioning Settings

So, what am I doing in my eventreceiver? I am adding the title, if it was not set (watch out, there is a difference between the dialog for a new page via the site settings menu and the new item in the pages library. The other thing I am doing is changing the settings of XsltListViewWebParts (change the view, set the row limit to at least 100 and filter the view using the title and a column I use called ‘page’.

These are the two interesting parts of the code:

This one sets the title. That means I need a web opened as an administrator and then check to see if the title is the way I want it to be…

So I get the list and the item and then check the title field.
if the title is null, then I set it, item.Title does not have a set-method so there we go…quick question what is faster? GetItemById, or GetItemByUniqueId? I would expect the first option…I did try it once though, and the id returned 33 while the number of items was 13 (I deleted a lot of pages) and the method returned an error, so I was playing safe here.

      private void SetTitle(SPItemEventProperties properties)
        {
            SPListItem item = properties.ListItem;
            if (properties.Web != null)
            {
                SPSecurity.RunWithElevatedPrivileges(
                    delegate
                    {
                        using (SPSite site = new SPSite(properties.Web.Url))
                        using (SPWeb web = site.OpenWeb())
                        {
                            SPList list = web.Lists[properties.ListId];
                            SPListItem admItem = 
                                           list.GetItemByUniqueId(item.UniqueId);
                            if (string.IsNullOrEmpty(
                                admItem[SPBuiltInFieldId.Title] as string)
                            )
                            {
                                title = admItem.Name.Replace(".aspx", "");
                                admItem[SPBuiltInFieldId.Title] = title;
                            }
                            else 
                            {
                                title = item.Title;
                            }
                            EventFiringEnabled = false;
                            site.AllowUnsafeUpdates = true;
                            web.AllowUnsafeUpdates = true;
                            admItem.SystemUpdate(false);
                            web.AllowUnsafeUpdates = false;
                            site.AllowUnsafeUpdates = false;
                            EventFiringEnabled = true;
                        }
                    });
            }
        }

The event firing and the updating is just fine right there. Watch for the system update, which is crucial.

Problem right now is still, that the check-in will overwrite the modified-by user that did the check-in that triggered the event, that I will solve with <a href="this.

     private void CheckView(SPWeb web, XsltListViewWebPart wp)
        {
            Guid oGuid = new Guid(wp.ViewGuid);
            SPList list = web.Lists[wp.ListId];
            SPView oWebPartView = list.Views[oGuid];
            int limit = 100;
            bool changed = false;
            if (oWebPartView.RowLimit < limit)
            {
                oWebPartView.RowLimit = 
                        Math.Max(limit, oWebPartView.RowLimit);
                changed = true;
            }
            string query = "" + title + "";
            if(!query.Equals(oWebPartView.Query))
            {
                oWebPartView.Query = query;
                changed = true;        
            }
            if(changed)
                oWebPartView.Update();
        }

In this method I simply take the webpart, which was already casted, then I pick up the Guid and get the view from the list. Initially I thought: what if somebody has an XsltListViewWebPart referring to a list in a sub-web…well that’s not possible – I checked – at least not ootb.

So now that I developed these two functions for the check-in event I thought: what if there is no force-checkout? Well that I still need to implement, adding an added-event and having it work only if there is a valid item to work on, because the properties.ListItem member does not exist after the added event when force-checkout is in-place.
So you got to watch that!

        private SPLimitedWebPartManager GetManagerFromItem(SPListItem item)
        {
            if (item == null || item.File == null) return null;
            SPFile itemFile = item.File;
            // open file...
            // GetLimitedWebPartManager from SPFile object
            SPLimitedWebPartManager wpmgr = 
                         itemFile.Web.GetLimitedWebPartManager(
                         itemFile.ServerRelativeUrl, 
                         PersonalizationScope.Shared);
            return wpmgr;
        }
        ...
        SPLimitedWebPartCollection collection = wpmngr.WebParts;
        List consumers = new List();
        foreach (var wp in collection)
        {
          if (wp != null)
          {
            if (wp is XsltListViewWebPart)
            {
              XsltListViewWebPart view = (XsltListViewWebPart) wp;
              CheckView(web, view);
              consumers.Add(view);
            }
          }
        } 
        ...                        

So what happens here is just that I get the limited webpart manager from the spfile object’s relativeurl and then iterate over the webpartcollection getting only the xsltlistviewwebparts…this is also discussed at many other places, you can google that, if you need more in-depth information.