Automate Office 365 Health Status Monitoring Using PowerShell

[Update: This solution is now outdated as Microsoft retired the API used. You can now refer this article to use new Office 365 Communication APIs]

A few days ago, many of the users of our SharePoint Online environment complained about not being able to access the portal and were getting a blank page when accessing on Internet Explorer. Now, since the site was accessible perfectly fine with Chrome browser, there was no way to suspect any issue from SharePoint Online side itself.

First Thoughts

We had implemented a redirection from an On-premise IIS site to SharePoint Online site using Smart Links. This was because, our business users wanted to use a known URL like https://mycompany.com to access the portal site, instead of https://mycompany.sharepoint.com and still avoid the Microsoft login page. So, first we thought, something went wrong there.

But well, it was still working in Chrome!  and nothing was changed in that redirection code for months, so as expected couldn’t find anything.

Initial Troubleshooting

We started looking into browser related issue like cache and cookies. This article mentioned similar issue and suggestion to clear the cache, cookies & local storage. We did so and it worked for some users and not so for others. Well so, investigations continue.

Eureka Moment

Just when everyone in the team was looking for some clue, someone looked into Office 365 Admin Portal and saw this message posted 🙂

O365 Message

Well, what can I say, we should have looked at that first!
But anyway, it’s not very productive and efficient to keep looking into the message center frequently and I couldn’t find any option to configure notification mails in the portal (It does exist in Office 365 Admin App for mobile).

So, I decided to explore if we can do something quickly with PowerShell.

Solution

Quick search on internet gave me some good starting points like this and this.

Step 1: Record Last Run Time of the Script

This is to ensure our script runs in an incremental way and picks up events which happened after it was run last time. Otherwise, users will be flooded with old and duplicate mails about events.

I decided to use Windows Registry to keep this information. You can use any other logic. This is a separate PowerShell function, which takes care of this.

[code]
#Keep Last Run Time of the Script in Registry
Function GetLastRunTimeFromReg($LastRunTime)
{
$ExistingDateTime = $LastRunTime
if((Get-ItemProperty HKCU:\Software\O365Monitor -Name LastRunTime -ea 0).LastRunTime)
{
$ExistingDateTime = (Get-ItemProperty HKCU:\Software\O365Monitor -Name LastRunTime).LastRunTime
Set-ItemProperty -Path HKCU:\Software\O365Monitor -Name LastRunTime -Value $LastRunTime
}
else
{
New-Item -Path HKCU:\Software\O365Monitor -Force
Set-ItemProperty -Path HKCU:\Software\O365Monitor -Name LastRunTime -Value $LastRunTime
}
return $ExistingDateTime
}
[/code]

Step 2: Connect to the Office 365 Tenant

You will need to have at least Service Administrator role for the account to get all the health information.

Even though, this article mentions that after the first call, subsequent calls can be done without Get-Credentials, I could not make that work from within windows Task Scheduler. So, I changed to old and known way to doing so. I have added the credentials directly in my script, but you can use Stored Credentials concept. Github has the functions published here.

[code]

Write-Host “Connecting to Office 365…”
$username = “[email protected]
$password = ConvertTo-SecureString ‘xxxxxxx’ -AsPlainText -Force
$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $username, $password

#Prepare the Call
$jsonPayload = (@{userName=$cred.username;password=$cred.GetNetworkCredential().password;} | convertto-json).tostring()

$cookie = (invoke-restmethod -contenttype “application/json” -method Post -uri “https://api.admin.microsoftonline.com/shdtenantcommunications.svc/Register” -body $jsonPayload).RegistrationCookie

#preferredEventType 0 and 1 here are for Service Incidents and Maintenance Events
$jsonPayload = (@{lastCookie=$cookie;locale=”en-US”;preferredEventTypes=@(0,1)} | convertto-json).tostring()

#Call the REST API and get all the events under Service Incidents and Maintenance Events
$events = (invoke-restmethod -contenttype “application/json” -method Post -uri “https://api.admin.microsoftonline.com/shdtenantcommunications.svc/GetEvents” -body $jsonPayload)

[/code]

Step 3: Prepare some PowerShell Variables

This is to ensure, we have all variables separate from the logic.

[code]
#Name of the SMTP Server to send mails
$smtp = “SMTP Server Name”

#List of users’ valid mail IDs to send mails to
[string[]]$to = “[email protected]”,”[email protected]

#Any dummy account to appear under from. This need not be a valid mail ID.
from = “[email protected]

#Get the current date and convert to a format in which Office 365 stores date, so that it can compare with last runtime of the script
$CurrentDateTime = (Get-Date).ToUniversalTime()
$CurrentDateTime = $CurrentDateTime.ToString(“MM/dd/yyyy HH:MM:ss”)

#Call the function written in Step 1 to create the registry entry, if doesn’t exist, otherwise return the last run time and update the current run time back in the registry
$LastRunTimeOfThisScript = GetLastRunTimeFromReg $CurrentDateTime

#For some weird reasons, calling this method first when when the registry key gets created, I was not getting the correct time from the function, so I took the easy way out and called that again 🙂
$LastRunTimeOfThisScript = GetLastRunTimeFromReg $CurrentDateTime

[/code]

Step 4:

Now, we are ready to iterate through all the events and filter the ones we are interested in

[code]
#Variables to Extract Message Details
$MessageTitle = “”
$MessageUserImpact = “”
$ApplicationName = “”

Write-Host “Looking for any new Events…”
foreach($event in $events.Events)
{
#Get the Latest Message. Each event can have multiple messages. We are just interested in the latest one.
$message = $event.Messages[$event.Messages.Count-1]
#Filter only those messages which occurred after the last run of this script
if($message.PublishedTime -gt $LastRunTimeOfThisScript)
{
$msgText = $message.MessageText
$ApplicationName = $event.AffectedServiceHealthStatus.ServiceName
#Extract Title and Impacted Application Name to be passed as Subject
try
{
$MessageTitle = $msgText.Substring($msgText.IndexOf(“Title:”),$msgText.IndexOf(“User Impact:”))
}
catch
{
$MessageTitle = $event.Title
}
try
{
$MessageCurrentStatus = $msgText.Substring($msgText.IndexOf(“User Impact:”))
}
catch
{
$MessageCurrentStatus = $msgText
}
Write-Host $ApplicationName $MessageTitle.Trim() $message.PublishedTime.ToUniversalTime()

#Prepare and Send Updates on Mail
try
{
$subject = $ApplicationName + “: ” + $MessageTitle.Replace(“Title:”,””).Trim() + ” – ” + $event.Status
}
catch
{
$subject = $ApplicationName + ” – ” + $event.Status
}
$subject = $subject.Replace(‘`r’, ‘ ‘).Replace(‘`n’, ‘ ‘)
$body = $MessageCurrentStatus
send-MailMessage -SmtpServer $smtp -To $to -From $from -Subject $subject -Body $body -Priority high
}
}

Write-Host “Completed…”
[/code]

If you want to get event notifications only for specific Office 365 Services like Exchange Online or SharePoint Online, you can use if condition on $event.AffectedServiceHealthStatus.ServiceName in the above loop.

Step 5: Schedule in Windows Task Scheduler

Our script is now ready to be added to the task scheduler. Put all these in a single .ps1 file and Add that to the task scheduler and schedule it to run every hour! It will run every hour and if there are any events posted in Office 365 portal since it ran last, it will send the details in the mail. Obviously, you can also trigger this manually.

Task Scheduler

And we are done. Enjoy the mail notification for all such incidents, advisories and updates 🙂

Mailer

Enjoy,
Anupam

You may also like

19 comments

  1. Hi Anupam,

    Very informative here. We’ve just jumped on board the O365 wagon and we’re currently in hybrid mode so this alerting script would really come in useful for us however I seem to be stuck at the 1st hurdle and that’s the 1st part of your script that should record the last run time in the registry. I’m guessing this should create a new registry item under HKCU:\Software\… ? But it just fails to work for me and obviously the rest of the script fails as it would always look for this entry to compare date/time. If you have time, I’d really appreciate it if you can explain this part of the script to me in more detail and how you managed to get to work? Also as an alternative, is it possible to record the last run time in something as simple as a TXT file?

    Best regards,
    Billy

    1. Hi Billy,
      You can try adding manually adding the Key O365Monitor first using RegEdit under HKCU:\Software. The script should then create the Key LastRunTime.
      You can always use a text file to write and read back. It’s just I wanted to avoid the script’s dependency on any external files.

      Hope this helps.
      Anupam

      1. Thanks Anupam. I tried but am now getting the below error:

        Could not compare “02/05/2018 05:31:10″ to ”
        New-Item -Path HKCU:\Software\O365Monitor -Force
        Set-ItemProperty -Path HKCU:\Software\O365Monitor -Name LastRunTime -Value $LastRunTime
        02/05/2018 15:02:34″. Error: “Cannot convert the “System.Object[]” value of type “System.Object[]” to type “System.DateTime”.”
        At C:\O365 Scripts\O365-Monitoring.ps1:55 char:12
        + if($message.PublishedTime -gt $LastRunTimeOfThisScript)
        + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : ComparisonFailure

        1. Hi,
          Are you calling the function GetLastRunTimeFromReg with $CurrentDateTime converted to string format? See the line below

          #Get the current date and convert to a format in which Office 365 stores date, so that it can compare with last runtime of the script
          $CurrentDateTime = (Get-Date).ToUniversalTime()
          $CurrentDateTime = $CurrentDateTime.ToString(“MM/dd/yyyy HH:MM:ss”)

        2. I also realized, “else” was missing after the if condition in the function GetLastRunTimeFromReg. I have updated the script in the post. Use this, it should work now.

          Function GetLastRunTimeFromReg($LastRunTime)
          {
          $ExistingDateTime = $LastRunTime
          if((Get-ItemProperty HKCU:\Software\O365Monitor -Name LastRunTime -ea 0).LastRunTime)
          {
          $ExistingDateTime = (Get-ItemProperty HKCU:\Software\O365Monitor -Name LastRunTime).LastRunTime
          Set-ItemProperty -Path HKCU:\Software\O365Monitor -Name LastRunTime -Value $LastRunTime
          }
          else
          {
          New-Item -Path HKCU:\Software\O365Monitor -Force
          Set-ItemProperty -Path HKCU:\Software\O365Monitor -Name LastRunTime -Value $LastRunTime
          }
          return $ExistingDateTime
          }

  2. Hi Anupam Thanks for the script. It’s working fine. Just one query can we also include the ID of the incident in the email alert body?

    1. Yes easy. Use $event.ID. Each event can have multiple messages, so ID will be with event and not message.

      1. Thanks Anupam for your help. I was able to set the Event ID in the subject itself, so that it would be easy to search in O365 portal with the help of ID.

  3. One more strange thing I observed in :
    $CurrentDateTime = $CurrentDateTime.ToString(“MM/dd/yyyy HH:MM:ss”)

    This was not giving the correct Minutes after conversion. I changed the HH:MM:ss to HH:mm:ss (mm in small). Not it’s reflecting the correct time 🙂

  4. Thanks Anupam again. This script is working fine, I am running this from last 2-3 days. The only thing I have found that it’s missing to pick some updates from portal. Looked further into this and found that in some cases $message.PublishedTime is not giving the correct time as given in the portal from some event. There seems to be delay in some cases to get correct time with $message.PublishedTime. Are you also facing this issue?

  5. Hi Anupam,

    It is a wonderful script.

    Just wanted to ask can we only get notifications for “Incidents” only and Not for “Advisories”.

    Please let me if this possible. If yes then can you please provide what all changes needs to be done in the script.

    1. The filter used preferredEventTypes=@(0,1) gets the list of “Service Incidents and Maintenance Events”. Only other filter available is 2 which includes all messages. So, you can try updating preferredEventTypes=@(0,1,2)

  6. Hi, looks really good, do you have this saved as a .txt file? Can’t get it to write to the registry and would like to see how the script should be structured as a whole.

    1. You can save it to a .txt file easily. Entire script is in the article, just copy all the functions and step 4 calls those functions…

  7. Thank you for this. I’m trying to run the script in powershell and it seems to get to the following line:

    cookie = (invoke-restmethod -contenttype “application/json” -method Post -uri “https://api.admin.microsoftonline.com/shdtenantcommunications.svc/Register” -body $jsonPayload).RegistrationCookie

    the script prompts for the URI. I enter the same URI manually in the prompt (https://api.admin.microsoftonline.com/shdtenantcommunications.svc/Register) and I get the message:

    Service
    BODY { color: #000000; background-color: white; font-family: Verdana; margin-left: 0px; margin-top: 0px; } #content { margin-left: 30px; font-size: .70em; padding-bottom: 2em; } A:link
    { color: #336699; font-weight: bold; text-decoration: underline; } A:visited { color: #6699cc; font-weight: bold; text-decoration: underline; } A:active { color: #336699; font-weight: bold;
    text-decoration: underline; } .heading1 { background-color: #003366; border-bottom: #336699 6px solid; color: #ffffff; font-family: Tahoma; font-size: 26px; font-weight: normal;margin: 0em
    0em 10px -20px; padding-bottom: 8px; padding-left: 30px;padding-top: 16px;} pre { font-size:small; background-color: #e5e5cc; padding: 5px; font-family: Courier New; margin-top: 0px;
    border: 1px #f0f0e0 solid; white-space: pre-wrap; white-space: -pre-wrap; word-wrap: break-word; } table { border-collapse: collapse; border-spacing: 0px; font-family: Verdana;} table th {
    border-right: 2px white solid; border-bottom: 2px white solid; font-weight: bold; background-color: #cecf9c;} table td { border-right: 2px white solid; border-bottom: 2px white solid;
    background-color: #e5e5cc;}

    Would you have any ideas why this would happen. Appreciate your time in effort in creating this.

    cheers

    Matt

  8. Matt T, thanks for the feedback, i am facing the same issue and unable to proceed with the script.

    any response to Matt T’s issue Gents?

    1. This is strange.
      I just tested the code again and there is no prompt to input URI again and that’s how it should be!

      Are you getting values in earlier calls? Like do you get valid values in the variable $jsonPayload when you run this line $jsonPayload = (@{lastCookie=$cookie;locale=”en-US”;preferredEventTypes=@(0,1)} | convertto-json).tostring()

      1. Hi Anupam,

        thanks for the assistant, it was account role/permission issue.
        account used to connect to Office must have admin permission.
        🙂

Leave a Reply

Your email address will not be published. Required fields are marked *