Troubleshooting Ivanti EPM IIS 404 Errors

Help others and share!

One of the tools available to administrators is Microsoft’s Log Parser Studio and Log Parser Studio GUI. Ivanti provides some details on how to install, but doesn’t seem to provide any details on how to go in depth with troubleshooting and proactive analysis of the log files  to identify and resolve issues. 

In this post, I am going to demonstrate how to find and resolve 404 errors specifically due to missing client policy XML files. I am not going to cover installing Log Parser or the GUI, please review the Ivanti link here for those instructions: 

The first thing I suggest you do, is to use the built in query “IIS: HTTP Status Codes by Count”. You want to get a baseline and understanding of all the codes currently going on in your system. 404.0 errors happened to be the second highest hit count on my servers and accounted for about 40% of all hits combined. 200.0 codes, thankfully, were the majority.

To identify what I was getting 404 hits on, I built a query and saved it as “IIS: 404 Distrinct Instances”. The goal here was to get a list of all distinct (no duplicates in my list) instances of where the 404 errors were coming from. The query below returns that list to a .csv file. You will need to make sure your log type is selected as IISW3CLOG. Obviously if IIS logging is turned off, you’ll need to turn it on first.

SELECT DISTINCT cs-uri-stem 
INTO '[OUTFILEPATH]\404_Distinct.csv'
FROM '[LOGFILEPATH]'
Where sc-status = 404
ORDER BY cs-uri-stem

Analyzing my output, I noticed that an overwhelming majority of 404 files were from the “/landesk/files/ClientPolicies/” folder. Each was an XML file with the format CP.0000.xml, where 0000 was the number of the Task ID in the console. These files correlate to scheduled tasks in EPM. In our case, there were 450 of these XML files missing. We believe the root cause was an in place upgrade that might have wiped the entire area, but we have yet to determine the exact root cause. 

Regardless, I checked with Ivanti, and there is no mechanism in place to monitor and ensure these files exist and\or are recreated should they go missing. You simply get a failure and the portal manager is unable to display the task to the end users. (You can vote for the ER here: https://ivanticore.uservoice.com/forums/904675-endpoint-manager/suggestions/35180251-monitor-and-recover-missing-cp-xml-files)

To get a distinct list of these tasks, I used log parser with the following code and saved it as “IIS: 404 Distinct Client Policies”.

SELECT DISTINCT cs-uri-stem 
INTO '[OUTFILEPATH]\404_Distinct_ClientPolicies.csv'
FROM '[LOGFILEPATH]'
Where sc-status = 404
AND cs-uri-stem like '/landesk/files/ClientPolicies/CP.%'
ORDER BY cs-uri-stem

The next step was to use the EPM console and track down each one of these tasks manually. Simply take the number found in the CP.0000.XML file and search for the task based on the Task ID. To make things easier and faster, I rearranged the columns so that the task ID showed first. 

Once you find the task, restart it and the CP file will get recreated. 

That’s it. Hope it was helpful. 

Help others and share!