Safe Deployment and Rollback Pattern

advancedTested on Xanadu

The Safe Deployment and Rollback Pattern provides a structured approach to deploying ServiceNow changes with built-in safety mechanisms and quick recovery options. This pattern combines pre-deployment backups, controlled phased rollouts using domain separation, and pre-prepared rollback update sets to minimize risk during production deployments.

This pattern addresses the critical challenge of maintaining system stability while delivering continuous improvements in enterprise ServiceNow environments. It creates a comprehensive safety net that allows development teams to deploy confidently while maintaining the ability to quickly revert changes if issues arise. The pattern includes automated backup creation, deployment validation checks, and orchestrated rollback procedures that can be executed with minimal downtime.

Chrome Extension

Sourdough: ServiceNow Monitoring and Analytics

A Chrome extension for ServiceNow Admins and Developers with essential tools, analytics, graphs and monitoring features.

Instance HealthGraphs & ChartsAPI HealthDeveloper ToolsQuick SearchInstance Switcher

Add to Chrome

Free to install. Pro $5/month after a 14-day no-card trial.
Pro requires the ServiceNow admin role. Upgrade inside the extension.

Overview

Tasks

CMDB

API

Metrics

Monitor

Internals

Instance:sourdoughdev·Version:Yokohama

Instance StateONLINE

System StatusFully Operational

Session Timeout90 minutes

Logged-In Sessions2 (20 active)

Build Nameyokohama-12-18-2024_p1

IP Address10.159.128.43

Instance HealthHealth Score: 90%

🔥 5dSourdough (Chrome Plugin)Dark Mode

Use This When

Use this pattern for critical production deployments, major feature releases, or when deploying changes that affect core business processes or large user populations. This pattern is essential when your organization has strict SLA requirements, compliance obligations, or when the cost of deployment failures is high.

Avoid When

Avoid this pattern for simple configuration changes, emergency hotfixes where speed is critical, or in development environments where rollback complexity isn't justified. This pattern adds significant overhead and may slow down deployment cycles, making it inappropriate for organizations practicing rapid continuous deployment of low-risk changes.

Implementation

Server-side

var SafeDeploymentManager = Class.create();
SafeDeploymentManager.prototype = {
    initialize: function() {
        this.deploymentGr = new GlideRecord('x_custom_deployment_log');
        this.updateSetUtil = new UpdateSetUtils();
    },
    
    // Create pre-deployment backup and validation
    createDeploymentPlan: function(updateSetSysId, targetDomain, phaseConfig) {
        var deploymentId = this._generateDeploymentId();
        
        // Create deployment record
        this.deploymentGr.initialize();
        this.deploymentGr.deployment_id = deploymentId;
        this.deploymentGr.update_set = updateSetSysId;
        this.deploymentGr.target_domain = targetDomain || 'global';
        this.deploymentGr.phase_config = JSON.stringify(phaseConfig);
        this.deploymentGr.status = 'planned';
        this.deploymentGr.created_by = gs.getUserID();
        var deploymentSysId = this.deploymentGr.insert();
        
        // Create rollback update set
        var rollbackSetId = this._createRollbackUpdateSet(updateSetSysId, deploymentId);
        
        // Update deployment record with rollback set
        this.deploymentGr.get(deploymentSysId);
        this.deploymentGr.rollback_set = rollbackSetId;
        this.deploymentGr.update();
        
        gs.info('Deployment plan created: ' + deploymentId + ', Rollback set: ' + rollbackSetId);
        return deploymentId;
    },
    
    // Execute phased deployment with validation
    executePhaseDeployment: function(deploymentId, phase) {
        this.deploymentGr.get('deployment_id', deploymentId);
        if (!this.deploymentGr.isValidRecord()) {
            throw new Error('Deployment not found: ' + deploymentId);
        }
        
        var phaseConfig = JSON.parse(this.deploymentGr.phase_config.toString());
        var currentPhase = phaseConfig.phases[phase];
        
        // Set deployment status
        this.deploymentGr.status = 'deploying_phase_' + phase;
        this.deploymentGr.current_phase = phase;
        this.deploymentGr.phase_start_time = new GlideDateTime();
        this.deploymentGr.update();
        
        try {
            // Deploy to specific domain/scope
            if (currentPhase.domain) {
                this._deployToDomain(this.deploymentGr.update_set.toString(), currentPhase.domain);
            }
            
            // Wait for validation period
            if (currentPhase.validation_period_minutes > 0) {
                gs.info('Phase ' + phase + ' deployed. Validation period: ' + currentPhase.validation_period_minutes + ' minutes');
                this._scheduleValidationCheck(deploymentId, phase, currentPhase.validation_period_minutes);
            }
            
            // Update status
            this.deploymentGr.get('deployment_id', deploymentId);
            this.deploymentGr.status = 'phase_' + phase + '_active';
            this.deploymentGr.phase_end_time = new GlideDateTime();
            this.deploymentGr.update();
            
            return true;
        } catch (e) {
            gs.error('Phase deployment failed: ' + e.message);
            this._initiateRollback(deploymentId, 'phase_deployment_error');
            return false;
        }
    },
    
    // Initiate rollback procedure
    initiateRollback: function(deploymentId, reason) {
        return this._initiateRollback(deploymentId, reason);
    },
    
    // Private methods
    _createRollbackUpdateSet: function(originalSetId, deploymentId) {
        var rollbackSet = new GlideRecord('sys_update_set');
        rollbackSet.initialize();
        rollbackSet.name = 'Rollback_' + deploymentId + '_' + new GlideDateTime().getDisplayValue();
        rollbackSet.description = 'Automated rollback set for deployment: ' + deploymentId;
        rollbackSet.state = 'build';
        var rollbackSetId = rollbackSet.insert();
        
        // Capture current state of objects being modified
        this._captureCurrentState(originalSetId, rollbackSetId);
        
        return rollbackSetId;
    },
    
    _captureCurrentState: function(updateSetId, rollbackSetId) {
        var updateXml = new GlideRecord('sys_update_xml');
        updateXml.addQuery('update_set', updateSetId);
        updateXml.query();
        
        while (updateXml.next()) {
            var tableName = updateXml.table.toString();
            var recordId = updateXml.name.toString();
            
            // Create rollback entry for current state
            this._createRollbackEntry(tableName, recordId, rollbackSetId);
        }
    },
    
    _deployToDomain: function(updateSetId, targetDomain) {
        // Use domain-scoped deployment logic
        var domainUtil = new DomainUtils();
        return domainUtil.deployToSpecificDomain(updateSetId, targetDomain);
    },
    
    _initiateRollback: function(deploymentId, reason) {
        this.deploymentGr.get('deployment_id', deploymentId);
        var rollbackSetId = this.deploymentGr.rollback_set.toString();
        
        // Update deployment status
        this.deploymentGr.status = 'rolling_back';
        this.deploymentGr.rollback_reason = reason;
        this.deploymentGr.rollback_start_time = new GlideDateTime();
        this.deploymentGr.update();
        
        // Execute rollback update set
        var rollbackResult = this.updateSetUtil.commitUpdateSet(rollbackSetId);
        
        // Update final status
        this.deploymentGr.get('deployment_id', deploymentId);
        this.deploymentGr.status = rollbackResult.success ? 'rolled_back' : 'rollback_failed';
        this.deploymentGr.rollback_end_time = new GlideDateTime();
        this.deploymentGr.update();
        
        gs.info('Rollback ' + (rollbackResult.success ? 'completed' : 'failed') + ' for deployment: ' + deploymentId);
        return rollbackResult.success;
    },
    
    _generateDeploymentId: function() {
        return 'DEP_' + new GlideDateTime().getNumericValue() + '_' + gs.generateGUID().substring(0, 8);
    },
    
    type: 'SafeDeploymentManager'
};

Client-side

// Client-side deployment dashboard controller
function DeploymentDashboard() {
    this.deploymentManager = null;
    
    this.initiateSafeDeployment = function(updateSetId, phaseConfig) {
        var ga = new GlideAjax('SafeDeploymentManager');
        ga.addParam('sysparm_name', 'createDeploymentPlan');
        ga.addParam('sysparm_update_set_id', updateSetId);
        ga.addParam('sysparm_phase_config', JSON.stringify(phaseConfig));
        
        ga.getXML(function(response) {
            var answer = response.responseXML.documentElement.getAttribute('answer');
            var result = JSON.parse(answer);
            
            if (result.success) {
                g_form.addInfoMessage('Deployment plan created: ' + result.deployment_id);
                // Update UI to show deployment progress
                updateDeploymentStatus(result.deployment_id, 'planned');
            } else {
                g_form.addErrorMessage('Failed to create deployment plan: ' + result.error);
            }
        });
    };
    
    this.executePhase = function(deploymentId, phase) {
        if (!confirm('Execute phase ' + phase + ' of deployment ' + deploymentId + '?')) {
            return;
        }
        
        var ga = new GlideAjax('SafeDeploymentManager');
        ga.addParam('sysparm_name', 'executePhaseDeployment');
        ga.addParam('sysparm_deployment_id', deploymentId);
        ga.addParam('sysparm_phase', phase);
        
        ga.getXML(function(response) {
            var answer = response.responseXML.documentElement.getAttribute('answer');
            var result = JSON.parse(answer);
            
            if (result.success) {
                g_form.addInfoMessage('Phase ' + phase + ' deployment initiated successfully');
                updateDeploymentStatus(deploymentId, 'deploying_phase_' + phase);
            } else {
                g_form.addErrorMessage('Phase deployment failed: ' + result.error);
            }
        });
    };
    
    this.initiateRollback = function(deploymentId, reason) {
        if (!confirm('This will rollback deployment ' + deploymentId + '. Continue?')) {
            return;
        }
        
        var ga = new GlideAjax('SafeDeploymentManager');
        ga.addParam('sysparm_name', 'initiateRollback');
        ga.addParam('sysparm_deployment_id', deploymentId);
        ga.addParam('sysparm_reason', reason || 'Manual rollback');
        
        ga.getXML(function(response) {
            var answer = response.responseXML.documentElement.getAttribute('answer');
            var result = JSON.parse(answer);
            
            if (result.success) {
                g_form.addInfoMessage('Rollback completed successfully');
                updateDeploymentStatus(deploymentId, 'rolled_back');
            } else {
                g_form.addErrorMessage('Rollback failed: ' + result.error);
            }
        });
    };
}

The server-side SafeDeploymentManager class orchestrates the entire deployment lifecycle with automatic rollback set creation, phased deployment execution, and comprehensive error handling. The client-side DeploymentDashboard provides a user interface for managing deployments with confirmation dialogs and real-time status updates. Key features include automated backup creation before deployment, domain-specific deployment capabilities, and structured rollback procedures with detailed logging.

Alternatives

Manual Update Set Backup Strategy

Simpler to implement and requires less infrastructure but relies heavily on manual processes and human discipline. Use when you have a small team and infrequent deployments but need some rollback capability.

Blue-Green Deployment with Instance Cloning

Provides complete environment isolation and instant rollback but requires additional ServiceNow instances and significant infrastructure costs. Best for organizations with large budgets and zero-downtime requirements.

Feature Toggle Pattern with Conditional Logic

Allows runtime rollback without redeployment and enables gradual feature exposure but adds code complexity and technical debt. Ideal for application features that can be safely toggled on/off without data structure changes.

Test Your Knowledge

Quick 3-question quiz — see how your ServiceNow skills stack up.

Question 1 of 3Performance

A list view on a table with millions of records is slow. Best fix?

Select an answer to continue