Error Reference

ATF test suite failed after upgrade: automated tests not passing

ATF test suite failed after upgrade: automated tests not passing

Where this appears

This error appears in the Automated Test Framework (ATF) test results after a ServiceNow platform upgrade when previously passing test suites now fail. The error is typically visible in the ATF Test Results table (sys_atf_test_result) and in the Test Runner interface under System Applications > Automated Test Framework > Run Test Suite.

Sourdough
Chrome Extension

Debugging this in a real instance?

Sourdough lets you trace "ATF test suite failed after upgrade: automated tests not passing" through system logs, script errors, and API health in the ServiceNow tab you are already using.

Add to Chrome

Free to install. Pro $5/month after a 14-day no-card trial.
Pro requires the ServiceNow admin role. Upgrade inside the extension.

Overview
Tasks
CMDB
API
Metrics
Monitor
Internals
Instance:sourdoughdev·Version:Yokohama
Instance StateONLINE
System StatusFully Operational
Session Timeout90 minutes
Logged-In Sessions2 (20 active)
Build Nameyokohama-12-18-2024_p1
IP Address10.159.128.43
Instance HealthHealth Score: 90%
🔥 5dSourdough (Chrome Plugin)Dark Mode

Cause

Platform upgrade changed underlying system behavior, APIs, or business rules that the ATF tests depend on, causing test assertions to fail with updated system responses. (high likelihood)

Upgrade skipped files or customizations were not properly applied, leaving tests expecting old system behavior while the platform operates with new logic. (high likelihood)

Test data or reference records were modified during the upgrade process, causing tests to fail when they cannot find expected baseline data. (medium likelihood)

New platform validation rules or mandatory fields were introduced that conflict with test step configurations or test data creation. (medium likelihood)

UI changes in the upgraded platform broke UI-based test steps that rely on specific form elements, buttons, or page layouts. (low likelihood)

Fix

Step 1: Navigate to System Applications > Automated Test Framework > Test Results and identify which specific test steps are failing by reviewing the test result details and error messages.

Step 2: Check System Diagnostics > Upgrade History > Skipped Updates to identify any skipped upgrade components that might affect the failing test functionality.

Step 3: Compare the failing test assertions against the current system behavior by manually executing the same actions the test performs and documenting any differences.

Step 4: Review System Definition > Business Rules and System Definition > UI Policies for any new or modified rules that might affect the test scenarios.

Step 5: Update test step configurations and assertions to match the new platform behavior, or modify test data to comply with new validation requirements.

Step 6: For UI-based test failures, inspect the form layouts and update UI test steps to reference the correct form elements or field names.

Step 7: Re-run the updated tests individually before executing the full test suite to validate each fix.

Step 8: Execute the complete test suite and document any remaining issues for further analysis or consideration as acceptable changes due to platform improvements.

Test Your Knowledge

Quick 3-question quiz. See how your ServiceNow skills stack up.

Question 1 of 3Performance

A list view on a table with millions of records is slow. Best fix?

Select an answer to continue