Transform Map

What It Is

A Transform Map is the configuration that governs how data moves from an Import Set table to a target ServiceNow table, defining field-by-field mapping rules and data transformation logic. It sits in the System Import Sets application and acts as the bridge between raw imported data and your production tables. Transform Maps contain Field Maps that specify which Import Set columns map to which target table fields, coalesce fields that prevent duplicate record creation, and Transform Scripts that manipulate data during the transfer process.

Architecturally, Transform Maps live in the sys_transform_map table within the System Import Sets application scope. They operate at the data layer, executing after Import Set records are created but before target table records are written. The Transform Map references both the source Import Set table (source_table) and the destination ServiceNow table (target_table), creating a many-to-one relationship where multiple Import Set tables can target the same ServiceNow table through different Transform Maps.

Transform Maps integrate with ServiceNow's broader data import execution model, which includes Data Sources, Import Sets, and Load Tables. When you run a scheduled import or manually load data, the platform first populates the Import Set table with raw data, then executes the associated Transform Map to process that data into target tables. This execution happens through the Transform Engine, which processes Import Set records in batches and applies business rules, ACLs, and data policies to the resulting target table records.

You cannot function without Transform Maps in any scenario involving external data integration, whether from CSV files, web services, database imports, or third-party system feeds. Without a Transform Map, Import Set data remains isolated in staging tables and never reaches your production ServiceNow tables. This includes LDAP user imports, CMDB data feeds, incident creation from external monitoring tools, asset imports from discovery tools, and any custom integration where external systems need to create or update ServiceNow records. The Transform Map is also essential for data migration projects where you're moving from legacy systems into ServiceNow.

Platform owners typically create and configure Transform Maps during initial integration setup, while ServiceNow administrators maintain field mappings and troubleshoot transformation issues. Developers get involved when complex Transform Scripts are required for data manipulation, format conversion, or conditional logic that goes beyond simple field mapping. The relationship between these roles becomes critical during integration projects—platform owners design the data flow architecture, admins configure the practical field mappings based on business requirements, and developers implement custom transformation logic when standard mapping isn't sufficient.

Recent ServiceNow releases have enhanced Transform Map performance and error handling, particularly in Vancouver and later versions where batch processing improvements reduce memory consumption during large data imports. The introduction of Integration Hub and Flow Designer has created alternative integration paths, but Transform Maps remain the primary mechanism for bulk data imports and complex field transformation scenarios. Vancouver also improved the Transform Map debugging experience with better error logging and the ability to see transformation results in real-time during testing.

Where to Find and Configure It

Navigate to System Import Sets > Administration > Transform Maps to create and configure Transform Maps. This is where you define the source Import Set table, target ServiceNow table, set coalesce fields, and access the Field Maps related list. You can also reach Transform Maps through System Definition > Tables by opening any Import Set table and clicking the Transform Maps related list.

In ServiceNow Studio, access Transform Maps through the Application Explorer under Data Model > Transform Maps when working within a scoped application. App Engine Studio users can find them under Data > Import > Transform Maps. For troubleshooting active transformations, check System Import Sets > Administration > Import Log to see transformation results and error details.

How It Works Step by Step

Transform Maps operate within ServiceNow's Import Set framework as the processing engine that converts staging data into production records. When you trigger a data import—whether through scheduled jobs, manual import, or web service calls—the platform first loads raw data into the specified Import Set table without any validation or transformation. Each Import Set record receives a unique sys_id and an initial sys_import_state of inserted.

The Transform Map then processes these Import Set records through its configured Field Maps and Transform Scripts. Field Maps handle direct column-to-field transfers, applying any specified default values or choice mappings. Transform Scripts execute custom JavaScript logic for complex data manipulation, format conversion, or conditional field population. The platform maintains referential integrity by processing parent records before child records when dealing with reference fields, and it applies coalesce logic to determine whether to create new records or update existing ones based on the specified coalesce fields.

The Execution Order

1. ServiceNow loads raw data into the Import Set table with sys_import_state = 'inserted'
2. Transform Engine identifies Import Set records ready for processing and locks them to prevent concurrent processing
3. If an onBefore Transform Script exists, it executes first with access to the Import Set record and target table GlideRecord
4. Field Maps execute in order, transferring data from Import Set columns to target table fields
5. Coalesce logic evaluates specified fields to determine if this should create a new record or update an existing one
6. If an onAfter Transform Script exists, it executes after field mapping with access to both records
7. Target table record gets inserted or updated, triggering standard Business Rules, Workflows, and other server-side logic
8. Import Set record sys_import_state updates to processed, error, or ignored

// onBefore Transform Script example
// 'source' = Import Set record, 'target' = Target table record

// Skip processing if required field is empty
if (!source.u_employee_id) {
    ignore = true;
    return;
}

// Format phone number before field mapping
if (source.u_phone) {
    var phone = source.u_phone.toString();
    phone = phone.replace(/\D/g, ''); // Remove non-digits
    if (phone.length == 10) {
        source.u_phone = phone.substring(0,3) + '-' + phone.substring(3,6) + '-' + phone.substring(6,10);
    }
}

// Set default values based on source data
target.assignment_group = source.u_department == 'IT' ? '287ebd7da9fe198100f92cc8d1d2154e' : '';
target.priority = source.u_severity == 'High' ? '1' : '4';

Real-World Scenarios

Importing Employee Data with User Account Creation

HR provides daily employee data feeds that need to create user accounts and update employee records without creating duplicates. The Transform Map must coalesce on employee ID, format data consistently, and handle department references properly.

Configure the Transform Map with employee_number as the coalesce field on the sys_user table. Create Field Maps for standard fields like first_name, last_name, and email. Use an onBefore Transform Script to generate the user_name from first and last name, format phone numbers, and lookup department sys_ids from department names. Set active = true for new users and handle manager relationships by looking up existing user records.

Watch for duplicate user_name generation when employees have identical first and last names—implement a counter mechanism in your Transform Script. Department lookups frequently fail when HR uses different naming conventions than ServiceNow—create a reference mapping table or standardize names in the Transform Script. Manager relationships require two-pass processing since manager records might not exist when processing employee records, so consider running the Transform Map twice or handling manager updates separately.

CMDB Asset Import with Relationship Building

Discovery tools export server data including hostname, IP address, and installed software that must populate the CMDB with proper CI relationships. The Transform Map needs to handle CI identification, relationship creation, and software instance tracking.

Set up Transform Maps for both cmdb_ci_server and cmdb_rel_ci tables. Configure coalesce on name and ip_address for servers. Use Transform Scripts to lookup existing CI sys_ids, create CI relationships for installed software, and populate operational status based on discovery data. Handle CI classification by mapping discovery data to appropriate cmdb_ci subclasses based on server type or OS. Map hardware specifications to CI attributes and create Runs on::Runs relationships for software installations.

CI identification becomes complex when discovery tools provide inconsistent naming conventions—establish clear coalesce field priorities and validate IP address formats. Software relationships require careful handling of version differences and multiple installations of the same software on different paths. The cmdb_rel_ci Transform Map depends on both parent and child CIs existing first, so process CI creation before relationship creation in separate Transform Map runs.

Incident Creation from Monitoring Tool Alerts

External monitoring systems generate alerts that must become ServiceNow incidents with proper assignment, categorization, and CI relationships. The Transform Map must prevent duplicate incident creation, map external severity levels to ServiceNow priority, and assign to appropriate support groups.

Configure coalesce fields on correlation_id and caller_id to prevent duplicate incidents for the same alert. Create Field Maps for short_description, description, and cmdb_ci. Use Transform Scripts to map monitoring tool severity values (Critical, Warning, Info) to ServiceNow priority numbers (1-5), lookup CI sys_ids from hostname or IP address, and assign incidents to support groups based on CI category or alert type. Set default values for caller_id to a monitoring service account and contact_type to monitoring.

Monitoring systems often send resolution alerts that should close existing incidents rather than create new ones—implement logic to check incident state and update accordingly. CI lookup failures result in incidents without proper categorization and assignment, so establish fallback assignment groups for unknown CIs. Consider implementing time-based coalesce windows to handle alert flapping where the same issue generates multiple alerts within minutes—use Transform Scripts to check for recent incidents with similar characteristics.

The Classic Mistake

// Transform script trying to update existing incidents
// Coalesce field: sys_id only

(function runTransformScript(source, map, log, target) {
    // Trying to match existing records by sys_id from import
    target.sys_id = source.u_external_sys_id; // External system's ID
    target.number = source.u_incident_number;
    target.short_description = source.u_description;
    target.priority = source.u_priority;
    target.state = source.u_state;
    
    // This will fail - ServiceNow generates new sys_id
    // when it can't find the coalesce match
    log.info('Attempting to update incident: ' + source.u_incident_number);
})(source, map, log, target);

This creates duplicate records every time the transform runs because ServiceNow can't find existing records to update. The user sees multiple incidents with the same short_description and number values. Internally, ServiceNow generates a new sys_id for each import because it can't match the external system's ID format to existing ServiceNow records. The mistake is non-obvious because the import appears successful and creates records, but they're always new records instead of updates.

// Coalesce fields: number, correlation_id (business keys)
// Transform script for updating existing incidents

(function runTransformScript(source, map, log, target) {
    // Use business identifiers for coalescing
    target.number = source.u_incident_number;
    target.correlation_id = source.u_external_sys_id;
    target.short_description = source.u_description;
    target.priority = source.u_priority;
    target.state = source.u_state;
    
    // ServiceNow will find existing records by number or correlation_id
    // and update them instead of creating duplicates
    log.info('Processing incident: ' + source.u_incident_number);
})(source, map, log, target);

When to Use This vs Alternatives

Transform Maps are the correct choice for scheduled, batch data imports from external systems where you need field-level mapping control and duplicate prevention. They excel when importing structured data files (CSV, Excel, XML) or staging data from integration tables where the source schema doesn't match ServiceNow's target table structure.

Use Transform Maps When

You need complex field transformations, data cleansing, or conditional logic during import. REST APIs and direct database connections lack the granular field mapping and coalescing capabilities that Transform Maps provide. The scheduled execution model works perfectly for nightly imports from HR systems, asset management tools, or any batch data synchronization scenario.

Use REST/SOAP APIs Instead When

You need real-time, bidirectional integration or when the external system can format data to match ServiceNow's schema exactly. APIs handle authentication, error handling, and immediate responses better than Transform Maps. Choose APIs for integrations that require immediate feedback to the calling system or when building webhook-based event-driven integrations.

Use Both Together When

Building hybrid integrations where APIs receive real-time data into staging tables, then Transform Maps process the staged data during off-peak hours. This pattern works well for high-volume integrations where you need immediate data capture but complex processing can happen asynchronously. The API handles the immediate response while Transform Maps ensure data quality and proper field mapping.

Platform Interactions & Side Effects

• Business Rules fire normally during transform execution, including before, after, and async rules, which can trigger unexpected workflows or notifications during bulk imports
• ACLs evaluate against the transform user's roles, not the original data source, potentially blocking imports if the admin user lacks proper permissions on target tables
• Audit records in sys_audit show field changes with user set to the transform execution user, obscuring the actual data source in audit trails
• Transform execution logs write to sys_transform_log with detailed success/failure status and record counts, but logs auto-purge after 90 days by default
• Update Sets capture Transform Map configurations but not the Import Set data, causing incomplete deployments when moving integrations between instances
• Email notifications trigger normally from Business Rules during transforms, potentially sending hundreds of emails during bulk imports unless specifically controlled
• Database performance degrades during large transforms due to missing indexes on Import Set tables and excessive audit logging on target tables
• Transform scripts execute in global scope without access to current user session data, breaking any logic that relies on gs.getUser() or session properties
• Coalesce field queries bypass the normal GlideRecord query cache, causing direct database hits that can overwhelm the system during concurrent transforms
• Dictionary overrides and field-level security rules apply during transforms, potentially causing data truncation or field skipping without obvious error messages

Debugging and Troubleshooting

Transform failures typically manifest as "successful" imports that create no records or wrong record counts. Users report missing data while the import shows Complete status in System Import Sets > Import Set Tables. The most frustrating symptom is silent failures where transforms complete but skip records due to coalesce mismatches or ACL violations. Check System Logs > All for JavaScript errors in transform scripts, but many issues don't generate obvious error messages.

The primary debugging location is System Import Sets > Transform History which shows record-level processing results and error details. Transform script debugging requires adding log.info() statements since the Script Debugger doesn't work with transforms. Look for error messages like "Coalesce field not found" or "Access denied" in the transform history. Performance issues appear in System Diagnostics > Stats as elevated database query counts during transform execution.

Common error patterns include "Field map failed" messages when source data doesn't match target field types, "Coalesce returned multiple records" warnings when coalesce fields aren't unique, and "Transform script error" with JavaScript exceptions. Enable the glide.import.log_transform_scripts system property to capture detailed script execution logs. Memory errors during large imports appear as "OutOfMemoryError" in the application logs, typically caused by processing too many records in a single batch.

Diagnostic Checklist:

• Verify Import Set table contains expected data and row count matches source file
• Check Transform History for processing status and error messages on sample records
• Validate coalesce fields exist in target table and have appropriate uniqueness constraints
• Test transform user's ACL permissions on target table by manually creating a test record
• Run single record transform test using Transform action on Import Set record
• Review field map data types and ensure source values match target field constraints
• Check System Logs during transform execution for Business Rule conflicts or script errors

Quick Reference

• Transform Maps process maximum 10,000 records per batch by default, controlled by glide.import.max_records_per_batch system property
• Coalesce queries use "OR" logic between multiple fields, not "AND" - any single field match triggers an update instead of insert
• Import Set tables in u_* namespace automatically delete data after 120 days unless glide.import_set.cleanup.age is modified
• Transform scripts execute before Field Maps, allowing script logic to override automatic field mappings
• Choice field mappings fail silently when source values don't match target choice list options - records import with empty field values
• Reference field mappings require either sys_id values or Display Value setting enabled with matching display field data
• Transform Maps inherit target table ACLs, but Import Set tables use separate import_set_loader role permissions
• Scheduled imports via Data Sources create separate Import Set tables for each execution, not appending to existing tables
• Transform execution order follows Field Map sequence when multiple transforms target the same table, not creation order
• Domain separation applies to Transform Maps - transforms only process Import Set records in the same domain unless global domain is used