Improving Autopilot Reporting

Oct 02, 2020

So, you have progressed through engineering an Autopilot build, conducted some initial testing in IT and are ready for your first business pilot. Your project manager comes to you and requests a report on whether the pilot users have successfully enrolled devices or not.

You could go and look at the devices enrolled by each user in the Microsoft Endpoint Manager console but that would get tedious quickly. Alternatively the Autopilot deployments report (under Devices\Monitor) seems to hold promise. You scan the results for devices you expect to see.

Initially the Autopilot Deployments report looks promising but you quickly realize three problems

Where is the export button?
Why do the results change when I hit refresh?
Why can I not see devices that I know have enrolled?

Your project manager wants a neat report in Excel of all the devices that enrolled. The the Autopilot Deployments report in the Endpoint Manager console does not provide an easy route to provide reliable reporting.

The Solution

The solution to providing a high quality report is directly querying data using Graph API. This post will show you how to produce a reliable report from the Graph API data.

You can download the script from this post from Github [LINK]. I have posted the complete code for reuse.

The script does the following

Connects to MS Graph
Queries a list of Autopilot events
Enriches the returned data with user data and completed application installations
Outputs a report in CSV format

Here are some screenshots of the Autopilot reporting script in operation. Device names and other organization specific data have been redacted because this script was run on a production environment

This is an example report, opened in Excel.

Breaking the Script Down

Posting scripts with zero explanation of the script execution is one of my pet hates. The reader learns little if the script code is not obvious or they lack the knowledge to understand. Hopefully this break down of the script flow will help someone else use the same techniques for a different problem.

Script Setup

The top few lines specify parameters that will be used in the script.

param (
    [Parameter(Mandatory)]
    [Int]$QuerySize,
    [Parameter(Mandatory)]
    [Int]$PageSize
)

Two parameters are specified

QuerySize
PageSize

These parameters are used to tune the Graph API response (explanation further on).

Next, a log file is created by starting a script transcript. It is likely that you will hand the script to an operations team, so having a log file will make troubleshooting easier if (when) the script fails.

$Logfile = "AutopilotEnrolment" + "_" + (get-date -Format "ddMMyyyyHHmm") + ".log"
Start-Transcript -Path .\$Logfile -append

Next, a data array is created to hold application configuration information. This could be performed by importing a config file but for simplicity a few lines of data array worked for the deployment that the script was used on.

The application array contains data about applications that are installed by users after the initial Autopilot deployment. The array provides a way of reporting on whether the users have followed your instructions for post-autopilot device setup.

#Create the application array
$Applications = New-Object System.Collections.ArrayList
#Add the application ID's to the Application arrays
$Applications += [pscustomobject] @{
    "AppID"= 'GUID'
    "AppName"="Office"
    "AppDetail" = "Office 365 ProPlus - Monthly Channel"
}
$Applications += [pscustomobject] @{
    "AppID"= 'GUID'
    "AppName"="Office"
    "AppDetail" = "Office 365 ProPlus with Visio - Monthly Channel"
}

$Applications += [pscustomobject] @{
    "AppID"= 'GUID'
    "AppName"="Citrix"
    "AppDetail" = "Citrix Receiver"
}

A similar data array is also created to hold metadata about Autopilot deployment types. The array allows an AutoPilot profile name to be matched to a deployment ring (See a future post of explanation) and device type

#Create the Device ring assignment array
$DeviceRings= New-Object System.Collections.ArrayList
$DeviceRings+= [pscustomobject] @{
    'AutopilotProfile' = 'Ring Zero'
    'DeviceType' = "Windows 10 vNext"
    'DeploymentRing' = "Ring Zero"
}
$DeviceRings+= [pscustomobject] @{
    'AutopilotProfile' = 'Ring One'
    'DeviceType' = "Windows 10 vNext"
    'DeploymentRing' = "Ring One"
}
$DeviceRings+= [pscustomobject] @{
    'AutopilotProfile' = 'Ring Two'
    'DeviceType' = "Windows 10 vNext"
    'DeploymentRing' = "Ring Two"
}
$DeviceRings+= [pscustomobject] @{
    'AutopilotProfile' = 'Ring Two Shared'
    'DeviceType' = "Windows 10 Shared"
    'DeploymentRing' = "Ring Two Shared"
}
$DeviceRings+= [pscustomobject] @{
    'AutopilotProfile' = 'Ring Three Shared'
    'DeviceType' = "Windows 10 Shared"
    'DeploymentRing' = "Ring Three Shared"
}

Querying Graph API for Autopilot Events

Before we connect to MS Graph, the Graph API Schema needs to be set to Beta because the Autopilot graph endpoint is a beta endpoint.

Update-MSGraphEnvironment -SchemaVersion beta

Then a Graph connection is initiated, which will prompt for authorization using an Azure AD login.

Connect-MSGraph

Next, we create array is created to store the output data

$EnrolledDevices = New-Object System.Collections.ArrayList

These status variables are used to control the query loop. The variables need to be initiated explicitly to ensure the values are set to a default value.

$AllResultsRetrieved = $False
$ResultsRetrieved = 0

The Autopilot event query uses a Do loop with a binary condition to exit the loop. In most scripts, a Do loop is a bad pattern because you can get stuck in a never ending loop, but with this script it’s a necessity

#Loop through the result set
do {
    ...................................
} until ($AllResultsRetrieved -eq $True)

The first step inside the loop sets the query URL if this is the first run of the loop

If ($ResultsRetrieved -eq 0) {
        #Set the Graph API URL for the first run
        $AutopilotEventInvokeURL = "deviceManagement/AutopilotEvents?     top=$PageSize&" + "$" + "count=true"
}

The PageSize parameter is critical to the operation of the script because queries to the AutopilotEvents endpoint will fail if you try and retrieve too much data in one query (I found that out the hard way during a migration). The ?top=$PageSize parameter in the URL allows us to return a subset of the data (I.E. the first 50 results) without overloading the query.

PageSize is a parameter because it allows the operator of the script to tune the number of results in each sub-query when Graph is having a bad day (failures are more frequent in US working hours).

Next, the Graph query is executed by invoking a Get request to the specified URL.

$AutopilotEventResult = Invoke-MSGraphRequest -HttpMethod GET -Url $AutopilotEventInvokeURL

Grabbing the count value from the result set allows decisions to be made on the next action in the loop.

$AutopilotEventCount = $AutopilotEventResult."@odata.count"

Next, the results from the current result set are stored in a holding variable. The autopilot events are an array variable retrieved from the Value property on the Graph API result object.

$AutopilotEvents = $AutopilotEventResult.Value

Parsing the Autopilot Events

The AutopilotEvents array can be processed with a For Each loop. Each Autopilot event is stored in a custom PSObject. A custom PSObject is used to allow custom field values to be specified in the results array.

foreach ($AutopilotEvent in $AutopilotEvents) {
        
        #Add the result to the results array
        $EnrolledDevice = New-Object -TypeName PSObject -Property @{
            'DeviceName' = $AutopilotEvent.managedDeviceName
            'SerialNumber' = $AutopilotEvent.deviceSerialNumber
            'DeviceType' = ""
            'DeploymentRing' = ""
            'EnrolmentState' = $AutopilotEvent.enrollmentState
            'DeploymentState' = $AutopilotEvent.deploymentState
            'AutopilotDeviceID' = $AutopilotEvent.deviceId
            'AutopilotEventDateTime' = $AutopilotEvent.eventDateTime
            'DeviceRegistrationDateTime' = $AutopilotEvent.deviceRegisteredDateTime
            'enrollmentStartDateTime' = $AutopilotEvent.enrollmentStartDateTime
            'deploymentStartDateTime' = $AutopilotEvent.deploymentStartDateTime
            'deploymentEndDateTime' = $AutopilotEvent.deploymentEndDateTime
            'enrollmentType' = $AutopilotEvent.enrollmentType
            'UPNID' = $AutopilotEvent.userPrincipalName
            'UPN' = ""
            'UserDisplayName' = ""
            'AutopilotProfile' = $AutopilotEvent.windowsAutopilotDeploymentProfileDisplayName
            'OSVersion' = $AutopilotEvent.osVersion
            'Office' = ""
            'Citrix' = ""
        }
        $EnrolledDevices.Add($EnrolledDevice) | Out-Null
}

At the end of each iteration of the For Each loop, the PSObject is added to the results array with this line.

$EnrolledDevices.Add($EnrolledDevice) | Out-Null

Processing the Next Step in Graph API Query

Script execution reverts to the Do loop once all results are processed by the For Each loop. The next step in the Do loop determines whether another Graph query is required to complete the retrieval of all the required events.

First, the total count of the events retrieved is incremented using the count from the most recent query.

$ResultsRetrieved = $ResultsRetrieved + ($AutopilotEventResult.Value).count

Then the number of returned events is checked against the page size. If the number of events returned is less than the page size then a further query will not return any more event (note the query will still return a event but it will be a duplicate).

If ($AutopilotEventCount -eq $PageSize) {
 ......
 } else {
        ....
        $AllResultsRetrieved = $True
}

If the number of returned results for the last query is equal to the page size then more results could potentially be returned. The next decision statement checks whether we have reached the maximum number of queries that the QuerySize parameter specifies.

If ($ResultsRetrieved -eq $QuerySize) {
   .......
} else {
   .....
   $AutopilotEventInvokeURL = $AutopilotEventResult."@odata.nextLink"
}

If there are more results still to be returned, then we set the URL for the next iterative Graph API query using the @odata.nextLink property of the returned query result.

$AutopilotEventInvokeURL = $AutopilotEventResult."@odata.nextLink"

If all the results have been returned, then the AllResultsRetrieved boolean is set to True to trigger the exit of the Do loop.

$AllResultsRetrieved = $True

Enriching the Returned Data

At this stage, the returned results may not contain human readable data for the username associated with the device. A For Each loop cycles through the results array to ensure that the UPN and User Display Name are populated in the results rather than a GUID value.

A graph query converts the returned UPN value from the Autopilot events into the human readable values.

foreach ($EnrolledDevice in $EnrolledDevices){
    Write-Host "Querying UPN for $($EnrolledDevice.DeviceName)"
    $Deviceupn = $EnrolledDevice.UPNID
    $DeviceUser = Invoke-MSGraphRequest -Url "https://graph.microsoft.com/v1.0/users/$Deviceupn"
    $EnrolledDevice.UPN = $DeviceUser.userPrincipalName
    $EnrolledDevice.UserDisplayName = $DeviceUser.displayName
}

Next, the DeviceRings configuration array is used to add details of the device type and deployment ring. This step adds polish to the report and is not strictly necessary but it does make the report easier to filter in Excel.

Foreach ($EnrolledDevice in $EnrolledDevices) {
    Write-Host "Querying Device Ring for $($EnrolledDevice.DeviceName)"
    $DeviceType = $DeviceRings | Where-Object {$_.AutopilotProfile -eq $EnrolledDevice.AutopilotProfile}
    If (!($null -eq $DeviceType)) {
        $EnrolledDevice.DeviceType = $DeviceType.DeviceType
        $EnrolledDevice.DeploymentRing = $DeviceType.DeploymentRing
    } else {
        $EnrolledDevice.DeviceType = "Unknown"
        $EnrolledDevice.DeploymentRing = "Unknown"
    }
}

A Where-Object query matches the AutopilotProfile name returned from the Autopilot events with the matching value in the ring.

$DeviceType = $DeviceRings | Where-Object {$_.AutopilotProfile -eq $EnrolledDevice.AutopilotProfile}

The matched values are stored back into the results array.

$EnrolledDevice.DeviceType = $DeviceType.DeviceType
$EnrolledDevice.DeploymentRing = $DeviceType.DeploymentRing

Querying Installed Applications

In addition to adding polish by reporting device types, the script will also track whether users have installed particular applications on their device.

A For Each loop cycles through the Applications configuration array

foreach ($Application in $Applications){
 .....
}

For each application GUID in the Applications configuration array, a Graph API query is performed to get the applications installation status for that application type. The deviceAppManagement/mobileApps endpoint appears more reliable than the AutopilotEvents endpoint so a single query can retrieve all the results, even of a large result set.

$AppInstallInvokeURL = "deviceAppManagement/mobileApps/" + $Application.AppID + "/deviceStatuses"
$AppInstallResult = Invoke-MSGraphRequest -HttpMethod GET -Url $AppInstallInvokeURL
    
$AppInstallState = $AppInstallResult.Value

The returned results for that application are processed using a For Each loop to determine whether the application is installed on one of the Autopilot devices returned from the results array.

foreach ($EnrolledDevice in $EnrolledDevices){
        $location = [array]::indexof($AppInstallState.DeviceName,$EnrolledDevice.DeviceName) #get location of device from appinstallstate (if returned -1 the device is not listed therefore not insalled)
        if ($location -ne '-1'){
            $EnrolledDevice.($Application.AppName) = $AppInstallState[$location].installState
        } elseif ($null -eq $EnrolledDevice.($Application.AppName)) {
            $EnrolledDevice.($Application.AppName) = "not installed"
        }
}

Exporting the Results

The final step of the script exports the results to a CSV file.

$filename = "EnrollmentandAppStatus" + " " + (get-date -Format "ddMMyyyy HHmm") + ".csv"
$EnrolledDevices | Select-Object DeviceName,SerialNumber,DeviceType,DeploymentRing,AutopilotProfile,EnrolmentState,UPN,UserDisplayName,OSVersion,Office,Citrix,DeploymentState,DeviceRegistrationDateTime,enrollmentStartDateTime,deploymentStartDateTime,deploymentEndDateTime | Sort-Object SerialNumber,enrollmentStartDateTime | Export-Csv .\$filename -NoTypeInformation

Specifying the columns in the export with a Select-Object statement will order the columns in the report and ensure any unnecessary columns are not exported.

The file name is specified using an auto-generated file name.

Future Improvements

This script was created and incrementally improved during a large scale Autopilot device deployment. An initial version was created then tactically improved to resolve issues that arose during the migration. As such the script is not perfect but it does produce good results.

If I was going to improve the script, then I would probably focus on the following

The logic that detects the Autopilot events end of available results will not work if the page size is equal to the number of available results in the query (I.E. 50=50 when there are no more results after 50 available)
Move the application and device type configuration to a config file
The app field names specified in the application configuration array need to be manually added to script code rather than dynamically added based on the array
The output file should be a parameter rather than auto-generating a file
I would add some logic to avoid querying the UPN for AAD Self Deploying Autopilot profiles

An update will be posted to GitHub, if improvements to the script are made for a future Autopilot deployment.

Acknowledgements

We often build on work borrowed from colleagues and the wider community. I think it is important to acknowledge where re-used work comes from and provide credit where credit is due.

This script was a collaborative effort. I came up with the idea for the Autopilot event reporting script and produced an initial version that dumped a simple list of recently enrolled devices. We used that version for the first IT pilot deployments of the new Autopilot build.

Greg Judge added the logic for the UPN and application queries and made a number of script improvements as we rolled into the business pilot and full deployment.

We then used the script for a few weeks before encountering the issues with querying Autopilot events. I made a series of tactical improvements to the script including adding the device types and fixing the Autopilot Graph query using a sequential query structure. At least five significant rewrites of the script occurred over a four month deployment window.

Jamie Wiltshire and Kevin Parker provided valuable feedback on the script during the migration. Their input fed into the requirements and helped make incremental improvements.

To The Cloud and Beyond

Discussion about this post