Cheat Engine

[CodeReaver]

I'm trying to use the fxch command to swap floats to the top of the FPU stack, but whenever I do, st(0) is set to NaN. The register that I'm swapping it with starts out with a legitimate floating point number, but seems to be getting corrupted by the fxch function.

Is it supposed to do that? Is there a way I can stop it?

[ParkourPenguin]

Post your code. Whichever of the floating point registers you're swapping st(0) with obviously contains a NaN value. Set a breakpoint on the fxch instruction and look at the FPU registers to see.

[CodeReaver]

[ParkourPenguin]

This isn't a common problem to see, so I wouldn't put it past some people to accidentally do something stupid like use a register they didn't mean to.

The only reason I can find after taking a brief look through the documentation is that the tag of the register st(2) is marked as empty, causing the fxch instruction to substitute it for a QNaN prior to the exchange.

To confirm this, pop st(0) and st(1), execute an FXAM instruction, and check the FPU status word (FNSTSW) to see if st(2) is empty or not. Another more explicit way would be to use FNSTENV to get the FPU tag word. The order of the FPU tag word corresponds to the physical registers, so get which register is the top of the stack from the FPU status word (should already have it via FNSTENV) in order to figure out which physical register is st(2).

If st(2) is marked as empty (which means that register technically isn't a number ), store the FPU environment, clear that tag's bits, load the FPU environment (FLDENV), and check if it works then.

See Intel's Software Developer's Manual Volume 1 Chapter 8 for more detailed information.

An easier solution might be to simply choose a different injection point and/or backtrace to where the program uses the value of st(2) and hook there.