Bulk Record Operation Pattern - Safe…

intermediateTested on Xanadu

The Bulk Record Operation Pattern provides a structured approach to safely updating large numbers of records in ServiceNow without hitting transaction quotas or causing performance issues. ServiceNow imposes limits on database operations per transaction (typically 10,000 database operations), and naive approaches to mass updates can quickly exceed these limits, causing script failures.

This pattern solves the problem by breaking large update operations into manageable chunks, using efficient update methods like GlideRecord.updateMultiple() when possible, and leveraging background processing contexts like Scheduled Jobs or Script Actions to avoid user session timeouts. The pattern includes proper error handling, progress tracking, and rollback capabilities.

The core technique involves using GlideRecord with setLimit() and offset calculations to process records in batches, combined with transaction management to ensure data consistency. This approach allows you to update tens of thousands of records reliably while maintaining system performance and user experience.

Chrome Extension

Sourdough: ServiceNow Monitoring and Analytics

A Chrome extension for ServiceNow Admins and Developers with essential tools, analytics, graphs and monitoring features.

Instance HealthGraphs & ChartsAPI HealthDeveloper ToolsQuick SearchInstance Switcher

Add to Chrome

Free to install. Pro $5/month after a 14-day no-card trial.
Pro requires the ServiceNow admin role. Upgrade inside the extension.

Overview

Tasks

CMDB

API

Metrics

Monitor

Internals

Instance:sourdoughdev·Version:Yokohama

Instance StateONLINE

System StatusFully Operational

Session Timeout90 minutes

Logged-In Sessions2 (20 active)

Build Nameyokohama-12-18-2024_p1

IP Address10.159.128.43

Instance HealthHealth Score: 90%

🔥 5dSourdough (Chrome Plugin)Dark Mode

Use This When

Use this pattern when you need to update more than 1,000 records at once, perform data migrations or cleanup operations, or execute bulk updates that might take more than a few seconds to complete. It's essential for scheduled maintenance tasks, data correction scripts, and migration utilities.

Avoid When

Avoid this pattern for small-scale updates under 100 records where simple GlideRecord loops are sufficient, or when updates need to trigger complex business rules and workflows for each record individually. Don't use for real-time user-initiated operations where immediate feedback is required, as the chunked approach introduces processing delays.

Implementation

Server-side

var BulkUpdateProcessor = Class.create();
BulkUpdateProcessor.prototype = {
    initialize: function(tableName, encodedQuery) {
        this.tableName = tableName;
        this.encodedQuery = encodedQuery;
        this.chunkSize = 500; // Adjust based on operation complexity
        this.totalProcessed = 0;
        this.errorCount = 0;
        this.maxErrors = 10;
    },
    
    // Main processing method - use in Scheduled Job or Fix Script
    processRecords: function(updateFields) {
        gs.info('Starting bulk update for table: ' + this.tableName);
        
        var totalRecords = this._getTotalRecordCount();
        gs.info('Total records to process: ' + totalRecords);
        
        var offset = 0;
        var hasMoreRecords = true;
        
        while (hasMoreRecords && this.errorCount < this.maxErrors) {
            try {
                var processedInChunk = this._processChunk(offset, updateFields);
                
                if (processedInChunk === 0) {
                    hasMoreRecords = false;
                } else {
                    this.totalProcessed += processedInChunk;
                    offset += this.chunkSize;
                    
                    // Progress logging
                    var percentComplete = Math.round((this.totalProcessed / totalRecords) * 100);
                    gs.info('Processed ' + this.totalProcessed + ' of ' + totalRecords + ' (' + percentComplete + '%)');
                    
                    // Brief pause to prevent overwhelming the database
                    gs.sleep(100);
                }
            } catch (ex) {
                gs.error('Error processing chunk at offset ' + offset + ': ' + ex.message);
                this.errorCount++;
                offset += this.chunkSize; // Skip problematic chunk
            }
        }
        
        gs.info('Bulk update completed. Processed: ' + this.totalProcessed + ', Errors: ' + this.errorCount);
        return {
            totalProcessed: this.totalProcessed,
            errorCount: this.errorCount
        };
    },
    
    _getTotalRecordCount: function() {
        var gr = new GlideRecord(this.tableName);
        if (this.encodedQuery) {
            gr.addEncodedQuery(this.encodedQuery);
        }
        gr.query();
        return gr.getRowCount();
    },
    
    _processChunk: function(offset, updateFields) {
        var gr = new GlideRecord(this.tableName);
        if (this.encodedQuery) {
            gr.addEncodedQuery(this.encodedQuery);
        }
        
        // Use chooseWindow for efficient pagination
        gr.chooseWindow(offset, offset + this.chunkSize - 1);
        gr.query();
        
        var recordsInChunk = 0;
        var sysIds = [];
        
        // Collect sys_ids for updateMultiple approach
        while (gr.next()) {
            sysIds.push(gr.getUniqueValue());
            recordsInChunk++;
        }
        
        if (sysIds.length > 0) {
            this._updateRecordsBatch(sysIds, updateFields);
        }
        
        return recordsInChunk;
    },
    
    _updateRecordsBatch: function(sysIds, updateFields) {
        var gr = new GlideRecord(this.tableName);
        gr.addQuery('sys_id', 'IN', sysIds.join(','));
        
        // Set update fields
        for (var field in updateFields) {
            gr.setValue(field, updateFields[field]);
        }
        
        // Use updateMultiple for efficiency - single database operation
        gr.updateMultiple();
    },
    
    type: 'BulkUpdateProcessor'
};

// Example usage in Scheduled Job or Fix Script:
// var processor = new BulkUpdateProcessor('incident', 'stateIN1,2,3');
// var updateFields = {
//     'u_bulk_updated': 'true',
//     'u_processed_date': new GlideDateTime().getValue()
// };
// processor.processRecords(updateFields);

The code implements a reusable BulkUpdateProcessor class that handles chunked processing using chooseWindow() for efficient pagination and updateMultiple() for batch updates. Key features include error handling with maximum error thresholds, progress tracking with percentage completion, and brief sleep intervals to prevent database overload during large operations.

Alternatives

Direct updateMultiple() without chunking

Faster for moderate datasets (under 5,000 records) but risks hitting transaction limits on larger operations. Use when you're certain the record count won't exceed quotas and you need maximum speed.

Individual GlideRecord updates with business rule execution

Necessary when you need business rules, workflows, and audit trails to execute for each record, but significantly slower and more resource-intensive. Use for data corrections where business logic must be preserved.

REST API bulk operations

Better for cross-instance operations or when integrating with external systems, but adds network overhead and complexity. Use when updating records from external applications or between ServiceNow instances.

Test Your Knowledge

Quick 3-question quiz — see how your ServiceNow skills stack up.

Question 1 of 3Performance

A list view on a table with millions of records is slow. Best fix?

Select an answer to continue