Post-Migration Validation with AI-Generated Test Scripts

The migration is done. The databases are online. The application connects. Everyone exhales — and then spends the next three weeks discovering things that are subtly wrong. A permission that didn’t carry over. A linked server that points to the old instance. An Agent job that references a drive letter that doesn’t exist on the new server. Row counts that don’t match because a trigger fired during the cutover window.

Post-migration validation is the phase everyone agrees is important and nobody allocates enough time for. An AI agent can generate a comprehensive validation runbook from a single prompt — but the real trick is capturing the “before” baseline while you still can.

The Problem

You’ve migrated 15 databases from a SQL Server 2017 instance to SQL Server 2025. You need to verify that everything made it across intact: data, permissions, objects, configurations, jobs, linked servers, and logins. Doing this manually means writing dozens of comparison queries and running them against both old and new instances. It’s exactly the kind of systematic, repetitive work where humans make mistakes and agents don’t.

The Prompt

What the Agent Produced

The agent generated a well-structured two-phase toolkit. Phase 1 created a [DBA_MigrationValidation] database on the source instance with snapshot tables for each validation area. Phase 2 ran comparison queries that joined baseline snapshots against the current state of the target instance.

The overall structure was solid. The row count comparison used sys.dm_db_partition_stats for speed rather than COUNT(*) on every table — a good choice for large databases where counting every row would take hours. The permission snapshot captured both database-level permissions (sys.database_permissions) and server-level permissions (sys.server_permissions). The Agent job comparison checked job name, enabled status, schedule details, and job step definitions.

What I Validated and Changed

Several areas needed refinement:

  • Checksum approach was wrong. The agent used CHECKSUM_AGG(CHECKSUM(*)) across entire tables. This is fast but unreliable — CHECKSUM is not collision-resistant and CHECKSUM_AGG ignores row order, meaning different data can produce the same checksum. For critical tables, I switched to HASHBYTES on concatenated key columns, sampled rather than exhaustive. For the general case, I kept CHECKSUM_AGG as a quick smoke test but added a warning that matching checksums don’t guarantee identical data.

  • The baseline capture missed sys.sql_modules definitions. Comparing object counts tells you if a procedure exists. Comparing definitions tells you if someone modified it between baseline and migration. I added a module definition hash comparison.

  • Login SID matching was incomplete. The agent compared login names but not SIDs. If you recreate logins on the target instead of transferring them, the SIDs won’t match — and database users mapped by SID become orphaned. I added SID comparison to the login validation.

  • Missing server-level objects. The baseline didn’t capture server triggers, endpoints, credentials, or proxies. These are easy to forget and painful to discover missing post-migration.

The Final Scripts

Phase 1: Baseline Capture (Run on Source Before Migration)

Populate the baseline tables using dynamic SQL to iterate across databases:

Phase 2: Post-Migration Comparison (Run on Target)

After restoring the DBA_MigrationValidation database on the target (or connecting back to the source via linked server), run these comparisons:

The Validation Runbook

When you run these scripts, work through the results in priority order:

  1. Missing databases — something failed during migration. Stop and investigate.
  2. Row count mismatches — could indicate incomplete data transfer or transactions that committed during cutover. Small differences (under 0.01%) on active tables are expected if the cutover window had any activity.
  3. Missing logins or SID mismatches — these cause immediate application failures. For SID mismatches, remap database users with ALTER USER ... WITH LOGIN = ... or recreate the login with the original SID using CREATE LOGIN ... WITH SID = 0x.... The classic sp_help_revlogin approach handles both login transfer and SID preservation. Note that ALTER LOGIN cannot change a login’s SID — the SID is immutable once the login is created.
  4. Object count or definition mismatches — a module was modified between baseline capture and migration, or the migration tool missed an object.
  5. Agent job discrepancies — jobs missing or with different step counts. Review each flagged job individually.
  6. Linked server changes — test connectivity with EXEC sp_testlinkedserver @servername; for each one.
  7. Configuration differences — compatibility level changes are expected if you’re intentionally upgrading. Recovery model and collation changes are not.

Capturing the Baseline: Timing Matters

The entire validation framework depends on capturing Phase 1 at the right time. Too early and changes between baseline and migration create false positives. Too late and you’re scrambling to capture a baseline while the migration is already in progress.

My approach: capture the baseline 24 hours before the maintenance window, then capture it again immediately before cutover starts. The first capture is your safety net in case something goes wrong with the second. The second capture is your actual comparison point.

For the pre-migration health check scripts that feed into this validation workflow, see Post 5: Health Checks and Inventory.

Try This Yourself

Don’t wait for a real migration. Pick one database and run Phase 1 to capture a baseline. Wait a day, then run Phase 2 against the same instance. You’ll likely see zero discrepancies — which confirms the scripts work correctly. Then intentionally create a discrepancy: add a column, modify a procedure, or change a permission. Run Phase 2 again and verify the scripts catch the change. That dry run gives you confidence the validation toolkit works before you need it under pressure at 2 AM during a migration window.

For the full migration planning workflow — from deprecated feature scanning to compatibility assessment to runbook generation — see Post 11: Migration Planning.


Part of the ALTER DBA ADD AGENT series.