How to review a Conditional Access policy

A repeatable review order, severity definitions, and the findings that recur most.
Published by Urbannerd Consulting · Reviewed 2026-07-29

On this page

Policy review rubric
Accepted input formats
Review order
Severity definitions
Standard findings library
Output shape

Policy review rubric

Applied when a user pastes a policy, a policy export or a description of one. Deterministic

order so two reviews of the same policy produce the same findings.

Accepted input formats

Format	Handling
Graph `conditionalAccessPolicy` JSON	Preferred. Full fidelity. Parse against
Portal screenshot described in prose	Workable. Confirm the fields the description omits before reviewing
A list of settings typed by the user	Workable. Explicitly list what was not stated — silence is not "not configured"
"Require MFA for all users" (one line)	Treat as an intent, not a policy. Ask for the actual configuration or review the intent and say so

The silence rule: an unstated field is unknown, never empty. A policy description

that does not mention exclusions does not mean there are none. The agent must say

"exclusions not stated — if there are none, this is Critical" rather than assuming either way.

Review order

Work these in sequence. Do not skip ahead — order is what makes the review reproducible.

1. State

state: enabled / disabled / enabledForReportingButNotEnforced. Everything below is

interpreted differently depending on this. A dangerous policy in report-only is a

finding, not an incident.

2. Emergency access

Are break-glass accounts excluded? If not stated, ask. If genuinely absent → Critical,

stop and lead with it. If excluded via a group, is the group role-assignable or PIM-managed?

3. User scope

includeUsers / excludeUsers / includeGroups / excludeGroups / includeRoles /

excludeRoles / includeGuestsOrExternalUsers.

All users? Which populations does that actually catch — service accounts, guests, sync

accounts, shared mailboxes?

Role-based scope in a PIM tenant applies only during activation.
Are exclusions documented, owned and dated?

4. Resource scope

includeApplications / excludeApplications / includeUserActions /

includeAuthenticationContextClassReferences.

For MFA policies, All resources is required (benchmark guidance/5.2.2.2) — anything narrower

leaves admin portals and Azure Service Management reachable.

Are app exclusions justified, or accumulated?

5. Conditions

clientAppTypes, platforms, locations, signInRiskLevels, userRiskLevels,

devices/device filter, authenticationFlows, insiderRiskLevels.

Risk conditions present → P2 required. Does the population hold it?
Location conditions → is this being used as a substitute for identity controls?

6. Grant controls — the highest-yield check

operator (AND / OR) and builtInControls, authenticationStrength, termsOfUse.

This is where the most valuable findings come from. Check specifically:

Multiple controls with operator OR — does one of them let a user bypass the others?

MFA OR compliant device means a compliant device needs no MFA.

Should this be two policies? Almost always yes when MFA and device state are combined.
Is block combined with anything? Block is exclusive.

7. Session controls

signInFrequency, persistentBrowser, applicationEnforcedRestrictions,

cloudAppSecurity, continuousAccessEvaluation, secureSignInSession,

disableResilienceDefaults.

Token protection on browser paths — unsupported, will break.
disableResilienceDefaults set to true removes the backstop that keeps users signed in

during an Entra outage. Rarely intended.

8. Interaction with the rest of the estate

Overlap and duplication with other policies the user has described.
Conflicting grants across policies — remember policies combine as AND, so a Block

anywhere wins.

Gaps: which of the benchmark guidance controls has no corresponding policy?

9. Licensing

Does anything here require P2, Intune or Defender for Cloud Apps that the stated licence

does not include? See .

10. Hygiene

Naming, description, ownership, review date.

Severity definitions

Severity	Meaning
Critical	Lockout path, or the policy provides materially less protection than it appears to
High	Significant population blocked, or a real bypass exists
Medium	Weakened control, operational complexity, troubleshooting difficulty
Low	Hygiene, naming, documentation

Standard findings library

Recurring findings with their standard severity. Not exhaustive — but if one of these

applies, use this severity for consistency.

Finding	Severity	Why
Break-glass accounts not excluded	Critical	Tenant lockout, no in-product recovery
Break-glass exclusion via non-role-assignable group	Critical	Group Administrators can bypass CA or silently remove the exclusion
MFA OR compliant device in one policy	Critical	Compliant device satisfies the policy without MFA — protection is illusory
Block, All users, All resources, no exclusions	Critical	Total lockout
Phishing-resistant strength on admins with no pre-registration	Critical	Registration deadlock
MFA policy scoped to selected apps, not All resources	High	Admin portals and Azure Service Management API remain reachable
Enforced directly, no report-only period	High	Unmeasured blast radius
All administrators excluded	High	Inverts the security model
Device compliance enforced, enrolment coverage unknown	High	Mass block of unenrolled devices
Register-security-info policy with no TAP path	High	New hires cannot onboard
Legacy auth blocked with no MFP survey	High	Scan-to-email and LOB integrations fail
Risk conditions used without P2 across the scoped population	High	Policy silently does not apply as intended
Guests in scope of device compliance, not excluded	High	Partner access outage
Permanent service account exclusion, no owner or date	High	Becomes a standing bypass
Duplicate/overlapping MFA policies	Medium	Troubleshooting complexity; unclear which applies
Token protection applied to browser paths	Medium	Unsupported — breaks
Sign-in frequency shorter than needed	Medium	Prompt fatigue increases phishing susceptibility
`disableResilienceDefaults` = true	Medium	Removes outage backstop
Role-scoped policy in PIM tenant, applicability misunderstood	Medium	Applies only during activation
No policy description or owner	Low	Operational confusion
Inconsistent naming	Low	Hard to reason about the estate

Output shape

Policy summary — restate what it does in one paragraph, so the user can confirm the

agent read it correctly.

Lockout risk — rating and mechanism (see 03-lockout-risk-model.md).
Findings table — severity / finding / risk / recommendation, ordered most severe first.
Licensing dependencies — including what needs verification.
Remediation plan — in execution order, not severity order. The order you do things

in is rarely the order of severity.

What could not be assessed — fields not supplied, and what tenant data would settle them.

Section 6 is not optional. A review that does not say what it could not see is overclaiming.