Manual mapping and thread_local freeze
I havent seen anything posted about this so i wanted to share some of the things ive learned by experimenting with manual mapping.
internal dll for cs2, manually mapped, not loadlibrary. ive got a hook on scenesystem's DrawObject for chams. added a small cache to a helper that runs inside that hook:
I injected and entire game froze. With no explanation. No logs or exceptions. the whole process just locked as soon as the dll went in.
why i thought tls was fine
my mapper handles tls: (or i thought so)
The problem is: running tls callbacks is not the same thing as tls working. theyre different parts of the loader and i assumed one implied the other.
when the real loader maps a dll with a tls directory it does two things:
a manual mapper does 1 and skips 2. _tls_index stays whatever it was before, which is 0.
accessing a thread_local compiles to roughly this:
so youre hitting slot 0, which belongs to whatever module actually got index 0. either that pointer is null or garbage and you AV, or its valid and youre corrupting another module's thread state. second one is worse.
why it froze instead of crashing (figuring this out took a sleepless night)
the hook wraps its work in seh:
so every access violation gets caught and ignored. no crash. but that function runs for every mesh of every draw call often thousands of times a frame and now every one of them faults and goes through exception dispatch.
exception dispatch is slow as gosh, we dont want that. thousands per frame means the process makes basically no forward progress. it looks exactly like a deadlock, which is what i wasted the first hour hunting for.
if you have seh anywhere in a hot path, a fault storm reads as a freeze, not a crash. worth remembering generally, not just here.
what fixed it was not using thread_local in a manually mapped dll. i changed it to a plain static and had the function copy into a caller provided stack buffer instead:
if you genuinely need per-thread storage, use TlsAlloc/TlsGetValue at runtime that goes through the api and gets you a real index instead of trusting the loader to have set one up. or implement proper tls index allocation in your mapper, but thats a lot more work for something you can usually design around.
its not only variables you declare yourself.
msvc implements thread-safe function-local statics using _Init_thread_epoch, which is itself a tls variable. so this:
also touches tls in a manually mapped dll. plain zero-initialized POD statics are fine those land in .bss with no runtime init and no guard. anything with a dynamic initializer is a landmine. parts of the crt/stl use thread_local internally too.
feel free to jump in anywhere and correct me. We're all here to learn.
also sorry for the awkward wording at places i havent slept.
tldr:
I havent seen anything posted about this so i wanted to share some of the things ive learned by experimenting with manual mapping.
internal dll for cs2, manually mapped, not loadlibrary. ive got a hook on scenesystem's DrawObject for chams. added a small cache to a helper that runs inside that hook:
C++:
static thread_local NameSlot slots[512] = {};I injected and entire game froze. With no explanation. No logs or exceptions. the whole process just locked as soon as the dll went in.
why i thought tls was fine
my mapper handles tls: (or i thought so)
C++:
// Execute TLS Callbacks
if (pOpt->DataDirectory[IMAGE_DIRECTORY_ENTRY_TLS].Size) {
auto* pTLS = reinterpret_cast<IMAGE_TLS_DIRECTORY*>(pBase + pOpt->DataDirectory[IMAGE_DIRECTORY_ENTRY_TLS].VirtualAddress);
auto* pCallback = reinterpret_cast<PIMAGE_TLS_CALLBACK*>(pTLS->AddressOfCallBacks);
for (; pCallback && *pCallback; ++pCallback) {
(*pCallback)(pBase, DLL_PROCESS_ATTACH, nullptr);
}
}The problem is: running tls callbacks is not the same thing as tls working. theyre different parts of the loader and i assumed one implied the other.
when the real loader maps a dll with a tls directory it does two things:
- runs the callbacks the part everyone implements
- allocates a tls index for the module, writes it to _tls_index, and grows every existing thread's ThreadLocalStoragePointer array so theres a slot for your module's tls block
a manual mapper does 1 and skips 2. _tls_index stays whatever it was before, which is 0.
accessing a thread_local compiles to roughly this:
Code:
mov rax, gs:[58h] ; TEB->ThreadLocalStoragePointer
mov ecx, _tls_index ; 0, because nobody ever set it
mov rax, [rax+rcx*8] ; slot 0 -> not your module's block
mov dword ptr [rax+..] ; read/write into someone else's tlsso youre hitting slot 0, which belongs to whatever module actually got index 0. either that pointer is null or garbage and you AV, or its valid and youre corrupting another module's thread state. second one is worse.
why it froze instead of crashing (figuring this out took a sleepless night)
the hook wraps its work in seh:
C++:
void __fastcall hkDrawObject(...) {
__try {
// identify mesh, pick chams settings, draw
} __except(EXCEPTION_EXECUTE_HANDLER) {
if (oDrawObject) oDrawObject(...);
}
}so every access violation gets caught and ignored. no crash. but that function runs for every mesh of every draw call often thousands of times a frame and now every one of them faults and goes through exception dispatch.
exception dispatch is slow as gosh, we dont want that. thousands per frame means the process makes basically no forward progress. it looks exactly like a deadlock, which is what i wasted the first hour hunting for.
if you have seh anywhere in a hot path, a fault storm reads as a freeze, not a crash. worth remembering generally, not just here.
what fixed it was not using thread_local in a manually mapped dll. i changed it to a plain static and had the function copy into a caller provided stack buffer instead:
C++:
static const char* GetNameStable(Material* mat, char* outBuf, size_t outSize);if you genuinely need per-thread storage, use TlsAlloc/TlsGetValue at runtime that goes through the api and gets you a real index instead of trusting the loader to have set one up. or implement proper tls index allocation in your mapper, but thats a lot more work for something you can usually design around.
its not only variables you declare yourself.
msvc implements thread-safe function-local statics using _Init_thread_epoch, which is itself a tls variable. so this:
C++:
static SomeThing thing = MakeThing(); // dynamic initializeralso touches tls in a manually mapped dll. plain zero-initialized POD statics are fine those land in .bss with no runtime init and no guard. anything with a dynamic initializer is a landmine. parts of the crt/stl use thread_local internally too.
feel free to jump in anywhere and correct me. We're all here to learn.
also sorry for the awkward wording at places i havent slept.
tldr:
- manual mappers run tls callbacks, they dont allocate a tls index
- _tls_index stays 0, so every thread_local access hits slot 0 which isnt yours
- AV inside an seh-wrapped hot path = freeze, not crash
- function-local statics with dynamic initializers hit this too, via _Init_thread_epoch
- use TlsAlloc, or just design around it
Last edited by a moderator:
