Scheduled Job Distribution Pattern for…

intermediateTested on Xanadu

The Scheduled Job Distribution Pattern prevents database performance issues by spreading scheduled job execution times across the day rather than allowing them to cluster during peak hours. Many organizations inadvertently schedule critical jobs at obvious times like midnight or during business hours, creating resource contention that can degrade system performance.

This pattern uses randomized start times, intelligent offset calculations, and queue depth monitoring to ensure jobs run when system resources are available. It maintains job dependencies while distributing load, preventing the classic problem of dozens of jobs trying to execute simultaneously at round-hour intervals.

The pattern works by calculating optimal execution windows based on job priority, duration estimates, and current system load. It includes mechanisms to detect and redistribute jobs that have clustered over time, ensuring long-term load distribution as new jobs are added to the system.

Chrome Extension

Sourdough: ServiceNow Monitoring and Analytics

A Chrome extension for ServiceNow Admins and Developers with essential tools, analytics, graphs and monitoring features.

Instance HealthGraphs & ChartsAPI HealthDeveloper ToolsQuick SearchInstance Switcher

Add to Chrome

Free to install. Pro $5/month after a 14-day no-card trial.
Pro requires the ServiceNow admin role. Upgrade inside the extension.

Overview

Tasks

CMDB

API

Metrics

Monitor

Internals

Instance:sourdoughdev·Version:Yokohama

Instance StateONLINE

System StatusFully Operational

Session Timeout90 minutes

Logged-In Sessions2 (20 active)

Build Nameyokohama-12-18-2024_p1

IP Address10.159.128.43

Instance HealthHealth Score: 90%

🔥 5dSourdough (Chrome Plugin)Dark Mode

Use This When

Use this pattern when you have multiple scheduled jobs that could run at flexible times, when experiencing database performance issues during peak hours, or when managing large ServiceNow instances with dozens of scheduled jobs. This is especially valuable in environments where job timing isn't strictly dictated by business requirements.

Avoid When

Avoid this pattern when jobs have strict business timing requirements that cannot be adjusted, when you have very few scheduled jobs that don't create resource contention, or when jobs have complex interdependencies that require precise sequencing. Don't use this for time-sensitive integrations or jobs that must run immediately after business events.

Implementation

Server-side

// Script Include: JobDistributionManager
var JobDistributionManager = Class.create();
JobDistributionManager.prototype = {
    initialize: function() {
        this.PEAK_HOURS_START = 8; // 8 AM
        this.PEAK_HOURS_END = 17;  // 5 PM
        this.MAX_CONCURRENT_JOBS = 5;
    },

    // Distribute job execution times to avoid clustering
    redistributeJobs: function() {
        var jobGR = new GlideRecord('sysauto_script');
        jobGR.addQuery('active', true);
        jobGR.addQuery('run_type', 'periodically');
        jobGR.query();
        
        var jobs = [];
        while (jobGR.next()) {
            jobs.push({
                sysId: jobGR.getUniqueValue(),
                name: jobGR.getValue('name'),
                currentTime: jobGR.getValue('run_start'),
                priority: this._calculateJobPriority(jobGR)
            });
        }
        
        // Sort by priority and redistribute
        jobs.sort(function(a, b) { return b.priority - a.priority; });
        this._applyDistributionSchedule(jobs);
        
        gs.info('JobDistributionManager: Redistributed ' + jobs.length + ' scheduled jobs');
    },
    
    // Calculate optimal start time with randomized offset
    calculateOptimalStartTime: function(jobName, preferredHour) {
        var baseTime = new GlideDateTime();
        baseTime.setDisplayValue(baseTime.getDate().getDisplayValue() + ' 00:00:00');
        
        // Add preferred hour with random offset to avoid clustering
        var randomOffset = Math.floor(Math.random() * 240) - 120; // ±2 hours
        var finalHour = preferredHour + (randomOffset / 60);
        
        // Avoid peak hours if possible
        if (finalHour >= this.PEAK_HOURS_START && finalHour <= this.PEAK_HOURS_END) {
            finalHour = (finalHour < 12) ? finalHour - 4 : finalHour + 4;
        }
        
        // Ensure hour is within 24-hour range
        finalHour = Math.max(0, Math.min(23, finalHour));
        
        baseTime.addSeconds(Math.floor(finalHour * 3600));
        
        // Add additional minute randomization
        var minuteOffset = Math.floor(Math.random() * 60);
        baseTime.addSeconds(minuteOffset * 60);
        
        gs.debug('JobDistributionManager: Calculated start time for ' + jobName + ': ' + baseTime.getDisplayValue());
        return baseTime;
    },
    
    // Check current job queue depth to prevent overload
    checkJobQueueDepth: function() {
        var runningJobs = new GlideAggregate('sys_trigger');
        runningJobs.addQuery('state', 'ready');
        runningJobs.addQuery('trigger_type', 'run_script');
        runningJobs.addAggregate('COUNT');
        runningJobs.query();
        
        var queueDepth = 0;
        if (runningJobs.next()) {
            queueDepth = parseInt(runningJobs.getAggregate('COUNT'));
        }
        
        var shouldDefer = queueDepth > this.MAX_CONCURRENT_JOBS;
        gs.info('JobDistributionManager: Current queue depth: ' + queueDepth + ', Deferring new jobs: ' + shouldDefer);
        
        return {
            queueDepth: queueDepth,
            shouldDefer: shouldDefer,
            recommendedDelay: shouldDefer ? Math.floor(queueDepth * 5) : 0 // 5 min per queued job
        };
    },
    
    _calculateJobPriority: function(jobGR) {
        var priority = 50; // Base priority
        
        // Higher priority for system maintenance jobs
        if (jobGR.getValue('name').indexOf('System') === 0) priority += 20;
        
        // Lower priority for reporting jobs
        if (jobGR.getValue('name').indexOf('Report') >= 0) priority -= 10;
        
        // Consider frequency - less frequent jobs get higher priority
        var runPeriod = parseInt(jobGR.getValue('run_period'));
        if (runPeriod > 86400) priority += 10; // Daily or less frequent
        
        return priority;
    },
    
    _applyDistributionSchedule: function(jobs) {
        var timeSlots = this._generateTimeSlots(jobs.length);
        
        for (var i = 0; i < jobs.length; i++) {
            var job = jobs[i];
            var newTime = timeSlots[i];
            
            var jobGR = new GlideRecord('sysauto_script');
            if (jobGR.get(job.sysId)) {
                jobGR.setValue('run_start', newTime.getDisplayValue());
                jobGR.update();
                gs.debug('JobDistributionManager: Updated job ' + job.name + ' to run at ' + newTime.getDisplayValue());
            }
        }
    },
    
    _generateTimeSlots: function(jobCount) {
        var slots = [];
        var hoursInDay = 24;
        var nonPeakHours = hoursInDay - (this.PEAK_HOURS_END - this.PEAK_HOURS_START);
        var interval = (nonPeakHours * 60) / jobCount; // Minutes between jobs
        
        var currentTime = new GlideDateTime();
        currentTime.setDisplayValue(currentTime.getDate().getDisplayValue() + ' 00:00:00');
        
        for (var i = 0; i < jobCount; i++) {
            var slotTime = new GlideDateTime(currentTime);
            var minutesToAdd = i * interval;
            
            // Skip peak hours
            var hour = Math.floor(minutesToAdd / 60) % 24;
            if (hour >= this.PEAK_HOURS_START && hour < this.PEAK_HOURS_END) {
                minutesToAdd += (this.PEAK_HOURS_END - this.PEAK_HOURS_START) * 60;
            }
            
            slotTime.addSeconds(Math.floor(minutesToAdd * 60));
            slots.push(slotTime);
        }
        
        return slots;
    },
    
    type: 'JobDistributionManager'
};

The code provides a comprehensive job distribution system with three main functions: redistributeJobs() analyzes all active scheduled jobs and spreads them across non-peak hours, calculateOptimalStartTime() uses randomization to prevent clustering while avoiding business hours, and checkJobQueueDepth() monitors system load to prevent job queue overflow. The pattern includes priority-based scheduling and intelligent time slot generation to maintain optimal system performance.

Alternatives

Fixed Offset Scheduling

Simple implementation using predetermined time intervals between jobs without randomization. Easier to predict and debug but less effective at preventing clustering when new jobs are added over time.

Load-Based Dynamic Scheduling

Jobs automatically reschedule themselves based on real-time system metrics like CPU and database response times. More responsive to actual system conditions but requires additional monitoring infrastructure and can be unpredictable for operations teams.

Manual Peak Hour Avoidance

Administrative policy requiring all scheduled jobs to run outside business hours without automated distribution. Simpler to implement and understand but requires manual coordination and doesn't optimize resource usage during off-peak hours.

Test Your Knowledge

Quick 3-question quiz — see how your ServiceNow skills stack up.

Question 1 of 3Performance

A list view on a table with millions of records is slow. Best fix?

Select an answer to continue