Investigate issues with thread locals

[seritools]

• TLS destructors don't run on 9x
• Alignment issues on Win7 and lower: Add initial MinGW support #60 (comment), #[thread_local] alignment broken on Windows, GNU ABI rust-lang/rust#135719 (comment)

[seritools]

Update: While working on Rust9x 1.84 I've been testing this once again - still no dice, even though we now use the native TLS setup 🤔

[Jookia]

Could you provide some code for how to check this? I've been poking around the TLS area so maybe I can check it out.

[seritools]

@Jookia https://github.com/rust9x/rust9x-sample/blob/0492688fed6f5468bbd38eb8617d5da892911279/src/main.rs#L205-L231 in the sample should show the issue (drop code not running)

[Jookia]

Alright, I tested the destructor issue mentioned earlier on NT4 and 98 with MinGW and has_thread_local set to true and false. On both NT4 reports the drop and 98 doesn't.

I also tested the alignment code shown in the bug on NT4 and 98 with has_thread_local set to true and false. For an alignment of 256, NT4 aligns to 128 bytes and 98 aligns to 12 bytes (???).

Regarding alignment, Windows 7 and Wine are ignoring IMAGE_SCN_ALIGN_16BYTES. There's a fix for Wine soon. But the GNU toolchain also doesn't tell Windows to align things using that flag, and that doesn't seem to be working on my mingw-lite toolchain as of today.

Bugs about the alignment:

rust-lang#138903
rust-lang#135719

GCC's emulated TLS has a bad reputation:

rust-lang#91659
rust-lang#102243
msys2/MINGW-packages#2519

In 2025 the Cygwin port using emulated TLS was merged and works:

rust-lang#141846
rust-lang#141719

The latest attempt to enable thread_local on Windows is from 2025:

rust-lang#141794

The author's opinion is that "enabling this is just the matter of waiting an unspecified amount of time before minimal versions for GCC+Binutils+mingw-w64 can be set high enough." This was written by the same developer that commented in the open bug about alignment being broken, so it seems it's going to be a matter of time. Maybe 2027 is the year of use_thread_local on -pc-windows-gnu!

But evidently not today.

[seritools]

Thanks for the analysis!

and 98 aligns to 12 bytes (???).

IIRC it's just DWORD-aligned on 9x (hence the issue with the hashmap random state).

I wonder if some ungodly hacks are possible to manually align them :)

[Jookia]

Oh oops, I was reading the hex value rather than the binary bits. Yes it's aligned to 4 bytes.

[Jookia]

Some more research: Windows PEs can have a TLS callback directory: https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#the-tls-directory The kernel calls these when a DLL or process is attached or detached: https://github.com/wine-mirror/wine/blob/master/dlls/ntdll/loader.c#L1647 , https://github.com/mirror/reactos/blob/master/reactos/dll/ntdll/ldr/ldrinit.c#L1111. The NTDLL symbol is LdrpCallTlsInitializers.

You can dump information in this table by putting things in linker sections like ".CRT$XLA" to ".CRT$XLB".

MinGW's CRT handles the C destructor code for us here: https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-crt/crt/tlssup.c .

Rust puts a destructor callback in ".CRT$XLB". This is explained in https://github.com/rust9x/rust/blob/rust9x/library/std/src/sys/thread_local/guard/windows.rs .

So maybe Windows 9x doesn't implement this?

---

Nynaeve has a good write-up on how Windows TLS stuff works: https://web.archive.org/web/20251109192547/http://www.nynaeve.net/?p=190 (go backwards from here). I highly recommend reading them.

This page has some more information on the state of things: http://www.icodeguru.com/cpp/ImperfectCppPracticalSolutions/ch10lev1sec5.html It has this note:

The second problem is that the Win32 API does not provide any ability to clean up the slot when a thread exits. This means that one has to somehow intercept the thread's exit and clean up the resources associated with the value in that thread's slot. Naturally, for C++ folks, this is a painful loss of the automatic destruction that the language provides for us, and can be next to impossible to work around in some scenarios.

---

There's a few solutions I see:

1. Link in a DLL that has DllMain and have it somehow call the TLS functions
2. Do garbage collection whenever registor_dtor runs
3. Have threads launched by Rust clean up when they return or call an exit function

It looks like option 1 is what old timey thread libraries (such as very old Boost) used, but isn't that great as you have to ship around an extra DLL.

Maybe 2 and 3 in combination could work? I'm not sure.

[seritools]

From nynaeve:

Update: Phil mentions that this support for TLS callbacks does not work before the Visual Studio 2005 release. Be warned if you are still using an old compiler package.

This KB article in the MSDN 2005 library seems to concur:

Apparently in the pre-2005 C++ reference in the MSDN Library there was this restriction:

To paraphrase page 397 of the "C++ Language Reference," class objects may be instantiated with the thread attribute if the objects do not need to be constructed or destructed.

In the one that I can see in the 2005 library, I cannot find such a restriction anymore. In fact, the [current description] seems to be basically identical to the 2005 one. Note how:

Win32 and the Microsoft C++ compiler now support statically bound (load-time) per-thread data in addition to the existing API implementation.

"now" has been in there since at least 2005, haha

---

Before Windows Vista, statically declared TLS data objects can be used only in statically loaded image files. This fact makes it unreliable to use static TLS data in a DLL unless you know that the DLL, or anything statically linked with it, will never be loaded dynamically with the LoadLibrary API function. However, starting with Windows Vista, improvements were made to the Windows loader to better support the dynamic loading of DLLs with static TLS. This change means that DLLs with statically declared TLS data objects can now be used more reliably, even if they are loaded dynamically using LoadLibrary. The loader is capable of allocating TLS slots for such DLLs at load time, mitigating the limitations present in earlier versions of Windows.

(ref)

This means rust9x shouldn't use native/static TLS at all, anyway, given that we want to support pre-Vista Windows.

I do want to know if a C++ program, made with VC2005, can successfully call the destructor of a declspec(thread) variable on Win98. If it does work, it's "only" a matter of figuring out how it works, hopefully?