Process records in batches to avoid scheduled job timeout

ServiceNow scheduled jobs timeout after exactly 30 minutes, regardless of where they are in processing. The platform doesn't pause gracefully — it kills the job mid-execution, potentially leaving records in inconsistent states. Developers typically start with a simple query().next() loop that works fine on 10K records but fails silently at 100K. The solution isn't to process faster — it's to process in stateful chunks that can survive job restarts, using either offset-based pagination or cursor-style sys_id tracking stored in system properties.

When to use this pattern

• Processing more than 50,000 records in a single scheduled job execution
• Each record requires multiple API calls, web service requests, or complex calculations
• Historical data cleanup jobs that run monthly or quarterly on large tables
• Data migration scripts where you need to guarantee every record gets processed exactly once

When NOT to use this pattern

• Simple record counts or aggregate calculations — use GlideAggregate instead
• Real-time processing triggered by user actions — use business rules or Flow Designer
• Processing records that other jobs might be modifying — you'll get inconsistent state
• When you need transaction rollback across the entire dataset — each batch commits independently

Key behaviors and gotchas

• Always orderBy('sys_id') for cursor-based pagination — sys_id is guaranteed unique and indexed on every table
• Store progress in system properties, not scratch tables — properties survive instance restarts and clones
• Use processedCount == BATCH_SIZE to detect if more records exist — never assume you're done
• Batch size of 5000-10000 is optimal for most tables — smaller wastes overhead, larger risks timeout
• Include date range filters even on large tables — prevents runaway jobs if your completion logic fails
• Property keys should include your app scope — avoid collisions with other batch jobs