Renew Certificate in Provider Hosted Apps Scenario

With a certain customer of mine I recently had an issue, where in the span of a month all of the certificates for the Provider Hosted Apps Domain (PHA) had to be renewed for four staging environments (including PROD).

I was lucky to be the successor of somebody who made the same mistake as the service provider a month later, so I was prepared and could save the day. In hopes of saving you the time it took for me to force the service provider’s hand (around 10 hours telco time) I want to give you a brief overview of how to tackle this, the full list of reference articles and a script to set the sharepoint part (trust).

First a short introduction.

Certificates. Certificates are often used to encrypt data communication between machines. This is done to make sure that two parties can communicate without a third party listening. Also this is done to verify the identity of somebody initiating communication.

In the scenario of SharePoint and PHA we have two parties. We have the PHA Server Farm and the SharePoint Server Farm. Usually each farm consists of at least 2 servers for redundancy/ high availability reasons.

When HTTP communication is done via SSL each WebSite in IIS has a binding on port 443, which uses a certificate for encrypting the data he site responds with to requests.

Any admin can swap the certificate in IIS. All you need to do is check the certificate that exists and request a new certificate either self-signed, internally trusted or externally trusted with the correct SAN (Subject Alternative Name).

As an example, let’s suggest the following setup:
SharePoint has a wildcard certificate, like *.apps.mycompany.com. The PHA environment has a certificate corresponding to this in apps.mycompany.com. This may be the same certificate, if you request the big kahuna, i.e. a multi-san, wildcard certificate. Usually this is not the case, and is not necessary.

The PHA IIS will have the apps.mycompany.com certificate, and SharePoint will have the wildcard certificate. However how does SharePoint make sure, that PHAs are not added to different server and this server has different code and pretends to be the PHA server? There is a trust between these servers on the SharePoint side. In essence this article has one message: “Don’t forget this trust!”

The underlying process of replacing the apps.mycompany.com certificate is based on four easy steps, all of them are necessary:

  1. Replace the apps.mycompany.com certificate in the IIS of each PHA server

    This is a no-brainer. Request the certificate, get the response, use certmgr.msc to import the certificate into the Personal Store of the Machine Account. Make sure to have a private key for the certificate. This can be self-signed, internally trusted or externally trusted (depending on your scenario, if you externalize your farm or not).

  2. Export the apps.mycompany.com certificate as pfx (with private key)

    Export it with private key (and password) and put it into the location, where the web.config of each Provider Hosted App can access it. Usually this certificate is stored in a central location on each IIS PHA Server.

  3. Export the apps.mycompany.com certificate as cer (without private key)

    Export it without private key and put it into a location on a SharePoint server, where you can access it from the SharePoint Powershell script in the next step.

  4. Replace the SharePoint trust via script

    The certificate (cer) is referenced in two locations in SharePoint (SPTrustedRootAuthority, STSTrustedSecurityTokenIssuer). You can set it in the SPTrustedRootAuthority by updating the object and by deleting the STSTrustedSecurityTokenIssuer object and recreating this with the correct IssuerName and RegisteredIssuerName ([Issuer GUID]@[Realm]). See Script below.

EDIT: This image differs from the code below. A crucial parameter is missing. line 29 must have the flag “-IsTrustBroker” as seen below. I wrote a specific article on this topic here


param (
[string] $CertificateSubjectAlternativeName = "apps.mycompany.com"
, [string] $CertificatePathLocation = "[MyDrive]:\[MyPath]\apps.mycompany.com.cer"
)

asnp microsoft.sharepoint.powershell -ea 0

$certificate = $null;
$certificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($CertificatePathLocation);

if($certificate -ne $null) {
$tra = $null;
$tra = Get-SPTrustedRootAuthority | ? { $_.Certificate.Subject.Contains(${CertificateSubjectAlternativeName}) }

if( $tra -ne $null ) {
$tra.Certificate = $certificate;
$tra.Update();
} else {
Write-Host -ForegroundColor Red “Error: No Certificate with SAN ‘${CertificateSubjectAlternativeName}’ found in Root Authority Store.”;
}

$sci = $null;
$sci = Get-SPTrustedSecurityTokenIssuer | ? { $_.SigningCertificate.Subject.Contains(${CertificateSubjectAlternativeName}) }

if( $sci -ne $null ) {
$regIssuerName = $sci.RegisteredIssuerName;
$issuerName = $sci.DisplayName;
$sci.Delete();
New-SPTrustedSecurityTokenIssuer -Name “${issuerName}” -RegisteredIssuerName “${regIssuerName}” -Certificate $certificate -IsTrustBroker;
} else {
Write-Host -ForegroundColor Red “Error: No Certificate with SAN ‘${CertificateSubjectAlternativeName}’ found in Trusted Security Token Issuer Store.”;
}
} else {
Write-Host -ForegroundColor Red “Error: Certificate not found at location ‘${CertificatePathLocation}’.”;
}

The last step, which is not mandatory, but we had to do it was on the IIS Servers of the PHA environment. The certificate gets cached by the UserProfile of the User running the app pool. Thus once you replace it is no longer able to find the file. This will be broadcasted by an ugly error like: ‘CryptographicException: The system cannot find the file specified.’

This is how to fix that: open IIS –> ApplicationPools –> DefaultAppPool –> “Right Click” –> Advanced Settings –> Load User Profile | set this value to “true”.

It seems a bit absurd to change this setting since it did not have to be set when configuring the PHA connection in the first place, but it does the trick.

Links:

Read more of this post

Advertisements

Unexpected Response from Server when updating SharePoint ListItem via JSOM

These days I am working a lot on the client-side of things. So a couple of months ago I started writing my first lines of JavaScript/ JSOM (Javascript (Client)-Side Object Model).

I wrote a small method to create list items in a list (listTitle) based on a collection/ list of properties (properties) and their respective values. Here it is:

function createListItem(listTitle, properties) {
    var d = $.Deferred();
    try {
        var ctx = SP.ClientContext.get_current();
        var oList = ctx.get_web().get_lists().getByTitle(listTitle);

        var itemCreateInfo = new SP.ListItemCreationInformation();
        oListItem = oList.addItem(itemCreateInfo);

        for (var i = 0; i < properties.length; i++) {
            var prop = properties[i];
            oListItem.set_item(prop.Key, prop.Value);
        }

        oListItem.update();
        ctx.load(oListItem);
        var o = { d: d, ListItem: oListItem, List: listTitle };
        ctx.executeQueryAsync(
        function () {
            o.d.resolve(o.ListItem);
        },
        function (sender, args) {
            o.d.reject("Could not create list item in list " + o.List + " - " + args.get_message());
        });
    } catch (Exception) {
        console.log(Exception.message);
    }
    return d.promise();
}

function updateListItem(listTitle, properties) {
  var d = $.Deferred();
  try {
        var d = $.Deferred();
        var ctx = SP.ClientContext.get_current();
        var oList = ctx.get_web().get_lists().getByTitle(listTitle);

        oListItem = oList.getItemById(id);

        for (var i = 0; i < properties.length; i++) {
            var prop = properties[i];
            try {
                oListItem.set_item(prop.Key, prop.Value);
            } catch (Exception) {
                console.log(prop.Key + ' ' + prop.Value);
                console.log(Exception.message);
            }
        }
        oListItem.update();
        //ctx.load(oListItem);
        var o = { d: d, ListItem: oListItem, List: listTitle, p: properties };
        ctx.executeQueryAsync(
            Function.createDelegate(o, 
        function () {
            o.d.resolve(o.ListItem);
        }),
        Function.createDelegate(o, 
        function (sender, args) {
            o.d.reject("Could not update list item in list " + o.List + " - " + args.get_message());
        }));
    } catch (Exception) {
        console.log(Exception.message);
    }
  return d.promise();
}

So this is what happened when I had this code execute on editing another item in a different list…

When I debugged using Chrome (my browser of choice when writing JavaScript – never used it before that, interestingly…) I received the error “unexpected response from the server”.

I figured out that there are two key lines in this code that can be the cause of this.

var ctx = SP.ClientContext.get_current();

and

ctx.load(oListItem);

In my case the first line was actually not responsible for the error message. For your reference if you use var ctx = new SP.ClientContext(url); you may encounter this error message. So make sure to check that.

You should always use the current client context, when using JSOM, similar to the best practice guidelines for opening webs on server-side (SSOM [Server-Side Object Model]).

In my case the second line was the cause for the issue.

When creating an item I need to load the item into the context afterwards (or the error will show up even if the item is created correctly).

When updating an item the item may not be loaded into the context afterwards (or the error will show up even if the item is updated correctly).

It kind of makes sense, because when creating an item you are actually sending an SP.ListItemCreationInformation to the server. When updating an item I already have my listitem object. Why would I need to load all the other information afterwards?

So once I removed the line from the update method the code no longer evaluated to fail and the error message disappeared.

So for the experts among you this may be old news, but I actually needed to think about this for a few minutes before I figured it out, so I thought it was well worth blogging about. Especially since I haven’t blogged for quite some time.

AppManagement and SubscriptionSettings Services, Multiple Web Applications and SSL

So currently I am setting up four environments of which one is production, 2 are staging and another is was a playground installation.

My staging environments (TEST, QA, PROD) are multi-server, multi-farm systems (multi-farm because the 2013 Farm publishes Search and UPA to existing 2010 Farms).
They are running SPS2013 Standard with March PU 2013 + June CU 2013. They will be using App Pool Isolation and App Management and SubscriptSettings Services have their own account (svc_sp{t, q, p}_app, i.e. svc_spt_app, svc_spq_app and svc_spp_app).

I have three web applications of which all are secured by SSL making a wildcard certificate necessary for the app domain. Each has their own account (svc_sp{t, q, p}_{col, tws, upa}). The reason for this is that I will be using Kerberos Authentication and for the SPNs I need dedicated accounts for each Application URL.

My playground was once a 4 server farm, but now 3 servers have been removed. It does not run the March PU 2013 nor June CU 2013. There app pool isolation wihtout SSL is used.

On the playground the app management worked well. I actually encountered my problem on my test first and tried to replicate on the playground, but couldn’t. But I am getting ahead of myself. The system was setup via autospinstaller and the necessary certificates and IPs involved were requested and implemented. The AD Team did the domain setup for me. I didn’t setup my environment following this article, but it is a good one to read. I also got the idea of creating a separate dummy web application for attaching my IIS Bindings and Certificate from it, which makes a lot of sense, because of security considerations and kerberos.

The first article to read to get an overview of what is necessary and what’s trying to be achieved can be found here.

So I set up everything and still it wasn’t working. What does that mean? I will explain. When I subscribe to an app, download it from the store and add it in a site collection of my choosing I get to click on it once it is finished installing. The link then leads me to my app domain. With SSL only when I was using the same application pools I could actually get anywhere, otherwise I say the below.

This is what I wanted to see:
Expected

This is what I saw on any of the web applications with SSL and that had a different app pool account than the one I was using for my dummy web application.
Actual

So this blank page is actually exactly what you see when you leave the request management service running on the frontends without doing any topology configuration.

So I tried to work with the user policy from the web application management page in hopes of giving the users permissions on the content databases. This was actually not happening as I found out later, but which was actually exactly what was needed. I had to manually add the account of the app pool for the app domain to the SPDataAccess Group of the content databases. Then it also works with SSL. I actually set up three web applications WITHOUT SSL on the Test Staging Environment with the same users as the SSL Web Applications and this worked like a charm, but for any SSL web application I needed to explicitly give permissions to the content database. This is a nightmare to maintain. For my migration of 20 databases from 2010 to 2013 I need to do this again and again and for each new content database I will create in the future. Just imagine you create a new content database and forget to do this. Now for any site collection in this content database the above issue will show up. Hard to debug later on.

Not sure what Microsoft is thinking here, but I am happy that it only took me 4 days to figure this one out.

web.config

Okay, so I have my dev machine, and I have the typical error: please change your setting of customError in the web.config to RemoteOnly or Off so you can see the error.

I tried a lot, but this is what finally did the trick:
change the setting in the 14 hive web.config:
C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\LAYOUTS\web.config

– I checked to make sure the spelling was right (it is case-sensitive).
– I added it at different parts of the config file (no difference, you will see there is only one correct place to put it).
– Changed it from Off to RemoteOnly to Off (no difference, if you are accessing locally)
– Changed it via web.config manually and via IIS (the settings seem to be synced, but in special cases it seems they are not).

Creating a UPA Proxy via PowerShell – technet article typo

http://technet.microsoft.com/en-us/library/ff607737.aspx

In the example it says:

$app = Get-SPServiceApplication -Name PartitionedUserProfileApplication. New-SPProfileServiceApplicationProxy -Name PartitionedUserProfileApplication_Proxy -ProfileServiceApplication $app -PartitionMode

But the Member is -ServiceApplication rather than -ProfileServiceApplication.

Tried it, worked. Keep in mind.

If they had a chance to add a comment, I would have done it there.

CheckedIn / Added events on libraries

So I had to do another event receiver for a wiki pages library. There are different versions depending on the development state – they are different in that the library either has a force checkout or not. See below image and then the lower of the two red rectangles shows the setting that is either yes or no. The other one defines if a simple approval workflow is used to manage versions.
Versioning Settings

So, what am I doing in my eventreceiver? I am adding the title, if it was not set (watch out, there is a difference between the dialog for a new page via the site settings menu and the new item in the pages library. The other thing I am doing is changing the settings of XsltListViewWebParts (change the view, set the row limit to at least 100 and filter the view using the title and a column I use called ‘page’.

These are the two interesting parts of the code:

This one sets the title. That means I need a web opened as an administrator and then check to see if the title is the way I want it to be…

So I get the list and the item and then check the title field.
if the title is null, then I set it, item.Title does not have a set-method so there we go…quick question what is faster? GetItemById, or GetItemByUniqueId? I would expect the first option…I did try it once though, and the id returned 33 while the number of items was 13 (I deleted a lot of pages) and the method returned an error, so I was playing safe here.

      private void SetTitle(SPItemEventProperties properties)
        {
            SPListItem item = properties.ListItem;
            if (properties.Web != null)
            {
                SPSecurity.RunWithElevatedPrivileges(
                    delegate
                    {
                        using (SPSite site = new SPSite(properties.Web.Url))
                        using (SPWeb web = site.OpenWeb())
                        {
                            SPList list = web.Lists[properties.ListId];
                            SPListItem admItem = 
                                           list.GetItemByUniqueId(item.UniqueId);
                            if (string.IsNullOrEmpty(
                                admItem[SPBuiltInFieldId.Title] as string)
                            )
                            {
                                title = admItem.Name.Replace(".aspx", "");
                                admItem[SPBuiltInFieldId.Title] = title;
                            }
                            else 
                            {
                                title = item.Title;
                            }
                            EventFiringEnabled = false;
                            site.AllowUnsafeUpdates = true;
                            web.AllowUnsafeUpdates = true;
                            admItem.SystemUpdate(false);
                            web.AllowUnsafeUpdates = false;
                            site.AllowUnsafeUpdates = false;
                            EventFiringEnabled = true;
                        }
                    });
            }
        }

The event firing and the updating is just fine right there. Watch for the system update, which is crucial.

Problem right now is still, that the check-in will overwrite the modified-by user that did the check-in that triggered the event, that I will solve with <a href="this.

     private void CheckView(SPWeb web, XsltListViewWebPart wp)
        {
            Guid oGuid = new Guid(wp.ViewGuid);
            SPList list = web.Lists[wp.ListId];
            SPView oWebPartView = list.Views[oGuid];
            int limit = 100;
            bool changed = false;
            if (oWebPartView.RowLimit < limit)
            {
                oWebPartView.RowLimit = 
                        Math.Max(limit, oWebPartView.RowLimit);
                changed = true;
            }
            string query = "" + title + "";
            if(!query.Equals(oWebPartView.Query))
            {
                oWebPartView.Query = query;
                changed = true;        
            }
            if(changed)
                oWebPartView.Update();
        }

In this method I simply take the webpart, which was already casted, then I pick up the Guid and get the view from the list. Initially I thought: what if somebody has an XsltListViewWebPart referring to a list in a sub-web…well that’s not possible – I checked – at least not ootb.

So now that I developed these two functions for the check-in event I thought: what if there is no force-checkout? Well that I still need to implement, adding an added-event and having it work only if there is a valid item to work on, because the properties.ListItem member does not exist after the added event when force-checkout is in-place.
So you got to watch that!

        private SPLimitedWebPartManager GetManagerFromItem(SPListItem item)
        {
            if (item == null || item.File == null) return null;
            SPFile itemFile = item.File;
            // open file...
            // GetLimitedWebPartManager from SPFile object
            SPLimitedWebPartManager wpmgr = 
                         itemFile.Web.GetLimitedWebPartManager(
                         itemFile.ServerRelativeUrl, 
                         PersonalizationScope.Shared);
            return wpmgr;
        }
        ...
        SPLimitedWebPartCollection collection = wpmngr.WebParts;
        List consumers = new List();
        foreach (var wp in collection)
        {
          if (wp != null)
          {
            if (wp is XsltListViewWebPart)
            {
              XsltListViewWebPart view = (XsltListViewWebPart) wp;
              CheckView(web, view);
              consumers.Add(view);
            }
          }
        } 
        ...                        

So what happens here is just that I get the limited webpart manager from the spfile object’s relativeurl and then iterate over the webpartcollection getting only the xsltlistviewwebparts…this is also discussed at many other places, you can google that, if you need more in-depth information.

The trouble with editing items on synchronous add-events

As I have written once or twice (see here) before, I create a lot of functionality using event receivers.

Now when you are adding new items you might want to fill field values based on either the event or external data, so you will use a synchronous adding event (SPEventReceiverType.ItemAdding). Once you get the item (SPListItem item = properties.ListItem) and you want to edit the item you should use a disable-event-firing strategy (Eventfiring) if appropriate (trigger event on edit, not insert) – this is usually appropriate when you know conditions to be false on insertion or if you already call methods from insert events that would be duplicated by the edit-event.

Now the tricky part of the editing is that if you do it wrong, you will get cryptic messages (“Microsoft.SharePoint.SPException: Save Conflict Your changes conflict with those made concurrently by another user. If you want your changes to be applied, click Back in your Web browser, refresh the page, and resubmit your changes.”). The reason for this is that the user who creates the item is not the same as the context the event receiver is running in as long as you use the standard edit-and-update-methods available.

So let’s do it the right way:

SPListItem item = properties.ListItem;
item["TheInternalFieldName"] = "TheCalculatedValue";
item.SystemUpdate(false);

So this little piece of code took me a long time to figure out. Let’s check out why it’s good and then, what you could have done wrong:
The first step is basic: get the item from the properties.
The second step is editing the item. Easy! The internal field name must exist and the calculated value should be something that fitts into the column. Bamm! There you go. You could have done this by yourself so far.
Now, here is the important part: item.SystemUpdate(false).
It’s a system-user update with false as parameter.
If you check msdn you will find there are two SystemUpdate-methods. Web-Objects have the Update (non-overloaded: Update()), List-Objects have the method Update (overloaded: Update(), Update(boolean)) and items also have SystemUpdate (overloaded: SystemUpdate(), SystemUpdate(boolean)).
SystemUpdate(false) is exactly what you want as the explanation says: Updates the database with changes that are made to the list item without changing the Modified or Modified By fields, or optionally, the item version.
Okay. So you’re safe. One thing less to worry about. But why? Why? If it’s that simple why am I writing a whole article dedicated to this?
Well I have seen the error-messages. I need to reproduce them, because it has been a while since then but the essence is: usually you use update, as it is natural for lists and web and you use it without parameter, because usually you don’t have it (web) or you don’t need it (list – only when migrating data). So until you understand why SystemUpdate with boolean is important, a whole day can go by.

Basically I’m saying the same as Karine Bosch. Just found it while searching for error messages!