THREADPOOL Waits: Reproducing Worker Thread Exhaustion in SQL Server
Picture an instance that is running, where CPU might be low, but new connections time out and SSMS might not be able to open a query window. Restarting the service makes the problem go away, and it also resets the wait statistics and other DMVs that would have shown the cause. One cause that fits this pattern is worker thread exhaustion, which shows up in the wait statistics as THREADPOOL.
I reproduced it on a test instance to see what each symptom looks like from inside SQL Server, and what a new connection sees from outside.
Workers and the THREADPOOL Wait
SQL Server runs each unit of work, called a task, on a worker thread. A worker is SQL Server’s representation of an operating system thread. A serial query has one task and uses one worker. A parallel query has a parent task and several child tasks, and each task uses its own worker.[1] A task can also be a login or a logout. When a task is ready to run and no worker is available, the task waits with the wait type THREADPOOL.[2]
Microsoft’s description of THREADPOOL says the worker setting can be misconfigured. It says the more common cause is batches that run for an unusually long time and keep their workers, and it lists blocking and parallelism among the things to reduce.[2]
How Many Workers You Get
The max worker threads option defaults to 0, which lets SQL Server size the pool at startup. On 64-bit SQL Server 2016 SP2 and later, with more than 4 and up to 64 logical CPUs, the formula is 512 + ((logical CPUs - 4) * 16). Above 64 CPUs, the multiplier is 32 instead of 16.[3] Microsoft’s table gives these defaults for 64-bit SQL Server 2016 SP2, SQL Server 2017 and later:
| Logical CPUs | Default max workers |
|---|---|
| 4 or fewer | 512 |
| 8 | 576 |
| 16 | 704 |
| 32 | 960 |
| 64 | 1,472 |
| 128 | 4,480 |
My two test instances are SQL Server 2019 (15.0.4500.2, Developer) and SQL Server 2025 (17.0.5015.4, Enterprise Developer), on one workstation with 14 logical CPUs. The formula gives 512 + (10 * 16) = 672, and both instances report 672:
|
1 2 3 4 5 6 7 8 |
USE [master]; SELECT [si].[cpu_count] , [si].[scheduler_count] , [si].[max_workers_count] FROM [sys].[dm_os_sys_info] AS [si]; |
The limit doesn’t cover every thread. System threads such as the lazy writer, checkpoint and log writer are created outside it.[3]
Reproducing It
I ran the test on SQL Server 2019 only. On both versions, sys.configurations shows is_dynamic = 1 for max worker threads, so a new value takes effect at RECONFIGURE without a restart. I lowered it to 128, the lowest value allowed, so the test needed fewer connections.[3]
The scratch database has one table with 1,000,000 rows. A blocker session opened a transaction, updated every 1,000th row, and left the transaction open:
|
1 2 3 4 5 6 7 8 9 10 |
USE [sat_threadpool]; BEGIN TRANSACTION; UPDATE [dbo].[orders] SET [customer_id] = [customer_id] WHERE [order_id] % 1000 = 0; |
Then a PowerShell script lowered the setting and opened new connections one at a time, with connection pooling off. Each connection started this query asynchronously, and the open transaction blocked it:
|
1 2 3 4 5 6 7 8 9 |
USE [sat_threadpool]; SELECT COUNT_BIG(1) FROM [dbo].[orders] WHERE [pad] <> 'y' OPTION (MAXDOP 8); |
After each new connection, the script counted THREADPOOL waiters through the dedicated administrator connection (DAC), described below. I ran the test once with MAXDOP 1 and twice with MAXDOP 8. Each time, the setting stayed at 128 for less than 50 seconds before I set it back to 0.
The RECONFIGURE also flushed the Object Plans, SQL Plans and Bound Trees stores of the plan cache, according to the error log. The error log reports these flushes in batches. The next set of flush messages, at 13:15:26, reported 6 occurrences for each store. That count matches my five later RECONFIGURE statements for max worker threads, including the ones that set it back to 0, plus one change to another setting at 13:15:26. So the RECONFIGURE that sets the value back flushes the plan cache too.
What Happened
With MAXDOP 1, each blocked query held one worker. The first THREADPOOL waiter appeared after the 125th blocked session, while the 126th was connecting. Ten sessions later, a new connection failed. It had a 5-second connect timeout, and it gave up after about 10 seconds with this error:
|
1 |
Connection Timeout Expired. The timeout period elapsed while attempting to consume the pre-login handshake acknowledgement. This could be because the pre-login handshake failed or the server was unable to respond back in time. The duration spent while attempting to connect to this server was - [Pre-Login] initialization=10013; handshake=0; |
The error doesn’t mention workers, and it reads like a network problem. A sqlcmd login with a 15-second login timeout hadn’t connected after 15 seconds.
With MAXDOP 8, the first 12 queries ran at DOP 8 and each took 9 tasks, which is 8 child tasks plus the parent. That’s 108 workers for 12 queries. The 13th query ran at DOP 6, and every query after it ran at DOP 1. Microsoft says a lack of workers can reduce the DOP a query gets at runtime.[1]
| Run | MAXDOP | Session opening when THREADPOOL first seen | Seconds | New connections |
|---|---|---|---|---|
| 1 | 1 | 126 | 19.0 | Session 136 timed out after 10 s. sqlcmd hadn’t connected after 15 s. |
| 2 | 8 | 52 | 6.6 | Session 53 failed after 13.5 s. sqlcmd connected after 5.1 s. |
| 3 | 8 | 56 | 9.3 | All 60 sessions opened. sqlcmd connected in 143 ms. |
The seconds column counts from the first blocked session. The script opened six to eight sessions per second, so that column mostly reflects the script’s speed. In run 2 the failed connection reported A network-related or instance-specific error occurred while establishing a connection to SQL Server, with The wait operation timed out. Both failed connections took longer than the 5-second connect timeout I set. I observed that, and I can’t explain it. In run 3, five or six tasks were waiting on THREADPOOL and a new login still got in. In my runs, with only a few tasks in the queue, a new connection may get through or may time out.
The worker counts in sys.dm_os_schedulers didn’t stop at 128. When the first THREADPOOL waiter showed up in run 1, active workers were at 165 and current workers were at 178. Current workers were already 139 before I opened a single session, because that count includes idle workers.[4] I don’t know how SQL Server counts workers against the limit, so I wouldn’t expect current_workers_count to line up with max_workers_count.
Looking in Through the DAC
The DAC is a connection reserved for members of sysadmin, for when normal connections can’t get in.[5] It runs on its own hidden scheduler, which has workers set aside at startup.[4] Connect with sqlcmd -S SQLPROD01 -A -E -N -C, or put admin: in front of the server name. -N asks for an encrypted connection, and -C trusts the server certificate without validating it.[6] That skipped validation allows a man-in-the-middle attack,[7] so drop -C where the server has a certificate your clients trust. By default the DAC only accepts connections from the server itself. Setting remote admin connections to 1 allows it from another computer.[8] Only one DAC can be open at a time, and a second attempt gets error 17810. The DAC can’t run parallel queries. Microsoft recommends snapshot-based isolation if possible, and otherwise READ UNCOMMITTED with a short LOCK_TIMEOUT, such as 2000 ms, so the DAC session doesn’t get blocked itself.[5]
During run 3, I ran a diagnostic script through the DAC, and it returned in 163 ms. This is a trimmed version of the script I ran:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
USE [master]; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED; SET LOCK_TIMEOUT 2000; /* Tasks waiting for a worker. */ SELECT COUNT_BIG(1) AS [threadpool_waiters] , MAX([wt].[wait_duration_ms]) AS [longest_wait_ms] FROM [sys].[dm_os_waiting_tasks] AS [wt] WHERE [wt].[wait_type] = N'THREADPOOL'; /* Worker totals across the visible schedulers. */ SELECT SUM([s].[current_workers_count]) AS [current_workers] , SUM([s].[active_workers_count]) AS [active_workers] , SUM([s].[work_queue_count]) AS [work_queue] FROM [sys].[dm_os_schedulers] AS [s] WHERE [s].[status] = N'VISIBLE ONLINE'; /* Blocking per task, which includes the child tasks of parallel queries. */ SELECT TOP (5) [wt].[blocking_session_id] , COUNT_BIG(1) AS [blocked_tasks] , COUNT_BIG(DISTINCT [wt].[session_id]) AS [blocked_sessions] FROM [sys].[dm_os_waiting_tasks] AS [wt] WHERE [wt].[blocking_session_id] IS NOT NULL AND [wt].[blocking_session_id] <> [wt].[session_id] GROUP BY [wt].[blocking_session_id] ORDER BY COUNT_BIG(1) DESC; /* Head blockers: they block another session's task, and none of their own tasks is blocked. */ SELECT [s].[session_id] , [s].[status] , [s].[program_name] , [s].[open_transaction_count] , DATEDIFF(SECOND, [s].[last_request_end_time], GETDATE()) AS [idle_seconds] FROM [sys].[dm_exec_sessions] AS [s] WHERE EXISTS ( SELECT 1 FROM [sys].[dm_os_waiting_tasks] AS [wt] WHERE [wt].[blocking_session_id] = [s].[session_id] AND [wt].[session_id] <> [s].[session_id] ) AND NOT EXISTS ( SELECT 1 FROM [sys].[dm_os_waiting_tasks] AS [wt2] WHERE [wt2].[session_id] = [s].[session_id] AND [wt2].[blocking_session_id] IS NOT NULL AND [wt2].[blocking_session_id] <> [s].[session_id] ); |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
threadpool_waiters longest_wait_ms ------------------ --------------- 6 3313 current_workers active_workers work_queue --------------- -------------- ---------- 209 201 6 blocking_session_id blocked_tasks blocked_sessions ------------------- ------------- ---------------- 67 141 59 54 8 1 session_id status program_name open_transaction_count idle_seconds ---------- -------- -------------- ---------------------- ------------ 54 sleeping sat_tp_blocker 1 12 |
work_queue_count is the number of tasks waiting for a worker to pick them up, so it matched the THREADPOOL count here.[4]
I first wrote the blocking part against sys.dm_exec_requests, and it pointed at the wrong session. In run 3 it reported session 67 blocking 47 requests. Session 67 was the first parallel query, itself blocked. Parallelism didn’t cause that. In serial run 1, the same DMV reported session 71, the first blocked query, blocking 134 requests, and the real blocker, session 68, blocking 1. The later lock waiters appear to report an earlier queued waiter as their blocker, instead of the session that holds the lock. That’s my inference, and I haven’t tested it.
Parallelism does hide the real blocker from that DMV. The 13 parallel requests all showed CXPACKET with a blocking_session_id of 0. For a parallel request, sys.dm_exec_requests shows the wait of the coordinating thread only, and Microsoft points to sys.dm_os_waiting_tasks for the waits of each task.[9] The only tasks session 54 blocked were the child tasks of session 67, so session 54 never showed up as a blocker there. It wasn’t in sys.dm_exec_requests as a request either, because it was sleeping with no request. At the task level, session 54 blocks 8 tasks, and the head-blocker query finds it: sleeping, one open transaction, idle for 12 seconds.
Blocked process reports can capture the same chain if the blocked process threshold is set. I covered them in Blocked Process Reports: From Capture to Root Cause.
Raising max worker threads Cleared THREADPOOL, Not the Blocking
The max worker threads page says a performance problem is probably not about how many workers there are. It says the more likely cause is activity that occupies workers and doesn’t release them, such as long-running queries or blocking, and recommends finding the root cause before changing the setting.[3] The same page also suggests connecting with the DAC and killing the process when the instance stops responding, and raising the setting to prevent it.[3]
Run 3 shows the limit of that second suggestion. I set max worker threads back to 0, which made the limit 672, while the blocker still held its transaction. Over the next 9 seconds the current worker count went from 209 to 274, and the THREADPOOL waiters dropped to 0. Those waiters weren’t my test queries. Just before the restore, the 60 queries had 162 tasks: 149 waiting on LCK_M_S and 13 waiting on CXPACKET. Every query task already had a worker. Run 1 looked the same: 135 sessions, 135 tasks waiting on LCK_M_S, and 12 more tasks waiting on THREADPOOL. I sampled LCK_M_S once after the restore, 3 seconds later, and all 149 tasks were still waiting. Whatever was waiting on THREADPOOL got a worker within about 9 seconds, but none of my 60 queries could finish until the blocker rolled back. Rolling back the blocker’s transaction released all 60 queries, and they finished within 3 seconds.
In my test, a higher limit just let more sessions queue behind the same blocker, and each one keeps its worker while it waits. A parallel query can keep one worker per task. The fix in my test was to end the head blocker’s transaction. On a production system, I’d also find out why that session left a transaction open.
What to Watch
sys.dm_os_wait_stats is cumulative since the last restart or manual reset.[2] Before any test, the 2019 instance had 145 THREADPOOL waits totalling 63 ms, and the 2025 instance had 2,370 totalling 1,007 ms. Neither instance was showing symptoms at the time, so those counts didn’t indicate a problem. On the 2019 instance, max_wait_time_ms for THREADPOOL went from 5 ms before run 1 to 10,007 ms partway through it, and to 33,150 ms by the end of it. I’d look at wait time over an interval and the longest wait, not the count. Wait Stats, Deadlocks, and Blocking Chains: AI-Assisted Diagnosis covers capturing wait stats over an interval instead of reading the totals since startup. The tool in Performance Monitor: Free SQL Server Monitoring That Doesn’t Phone Home monitors THREADPOOL directly. If you plan to use the DAC from another computer, Vulnerability Assessment recommends disabling Remote Admin Connections covers the security side.
Limits of This Test
- I tested on one workstation with 14 logical CPUs, and I reproduced the problem on SQL Server 2019 only. On SQL Server 2025 I only read the worker settings.
- I lowered
max worker threadsto 128. At the default of 672, it would take more blocked sessions. I didn’t test that. - The blocking was synthetic: one open transaction holding 1,000 row locks.
- I didn’t test a DAC connection from another computer.
Related: Finding the Queries That Use the Most CPU, and What Each Source Misses (available 2026-11-22), Spinlock Contention for DBAs: Measure a Delta, Not a Total (available 2026-11-23) and Connection Pool Exhaustion: Reproducing the Timeout and What SQL Server Sees (available 2026-11-17).
If you’ve found THREADPOOL waits behind something other than blocking, I’d like to hear what it was. Find me on Bluesky or LinkedIn.
References
- Thread and task architecture guide - Microsoft Learn. Defines tasks and worker threads, describes the parent and child tasks of a parallel request, and states that a lack of workers can reduce the DOP available at runtime. ↩
- sys.dm_os_wait_stats (Transact-SQL) - Microsoft Learn. Describes THREADPOOL as a task (query or login/logout) waiting for a worker thread, names long-running batches as the most common cause, lists blocking and parallelism among the bottlenecks to reduce, and states that the counters are cumulative since the last reset or restart. ↩
- Server Configuration: max worker threads - Microsoft Learn. The default of 0, the sizing formula and table, the 128 to 65,535 range, system threads outside the limit, and the recommendation to find the root cause before changing the setting. ↩
- sys.dm_os_schedulers (Transact-SQL) - Microsoft Learn. Defines current_workers_count (which includes workers not assigned a task), active_workers_count and work_queue_count, and describes the hidden DAC scheduler and the workers allocated to it at startup. ↩
- Diagnostic Connection for Database Administrators - Microsoft Learn. How to connect with sqlcmd -A or admin:, local-only by default, one DAC per instance (error 17810), no parallel queries, and the advice to use snapshot-based isolation if possible, otherwise READ UNCOMMITTED with a short LOCK_TIMEOUT. ↩
- sqlcmd utility - Microsoft Learn. -N requests an encrypted connection (o for optional, m for mandatory, s for strict), and -C trusts the server certificate without validation, equivalent to the ADO.NET option TRUSTSERVERCERTIFICATE = true. ↩
- TDS 8.0 - Microsoft Learn. Describes TrustServerCertificate=true as not recommended because the driver then doesn’t validate the server TLS/SSL certificate, and states that strict encryption doesn’t allow TrustServerCertificate=true, in order to prevent a man-in-the-middle attack. ↩
- Server Configuration: remote admin connections - Microsoft Learn. The DAC accepts only local connections by default, and setting the option to 1 allows remote DAC connections. ↩
- sys.dm_exec_requests (Transact-SQL) - Microsoft Learn. For parallel requests, only the coordinator thread is visible and its wait columns are the ones updated, and sys.dm_os_waiting_tasks shows the wait of each task. ↩