View previous topic :: View next topic |
Author |
Message |
predprey Master Cheater Reputation: 24
Joined: 08 Oct 2015 Posts: 486
|
Posted: Sat Aug 31, 2019 9:51 pm Post subject: Calling convention for KERNEL32.SetFilePointerEx |
|
|
I understand that most x64 executables compiled for Windows uses MS's x64 calling convention. So far, that has yielded no problems and I interpret the arguments passed into a function as rcx,rdx,r8,r9 etc. However, while reversing the file structure for Zone of Enders, I encountered this behaviour for Window's SetFilePointerEx function which goes against the argument passing rule of the above convention. As shown in the screenshot below, the offset to move the file pointer to seems to be stored in RDI instead of RDX. The syntax for SetFilePointerEx is
Code: | BOOL SetFilePointerEx(
HANDLE hFile,
LARGE_INTEGER liDistanceToMove,
PLARGE_INTEGER lpNewFilePointer,
DWORD dwMoveMethod
); |
So going by MS's calling convention, the distance to move should be stored in RDX. However, the screenshot seems to suggest the it is pass through RDI instead. I couldn't get the process to break on SetFilePointer so it seems only SetFilePointerEx is used to move the pointer. Disassembling the ASM in the screenshot suggests the routine is used to get the current file pointer instead of moving it which explains how RDX is XOR-ed and dwMoveMethod is 1 for FILE_CURRENT. However, I'm not sure how the game is moving the file pointer backwards as RDI moves back and forths when breaking on the screenshotted location. Can someone explain what is going on here exactly? Thanks.
Description: |
|
Filesize: |
254.03 KB |
Viewed: |
1584 Time(s) |
|
|
|
Back to top |
|
|
Dark Byte Site Admin Reputation: 458
Joined: 09 May 2003 Posts: 25296 Location: The netherlands
|
Posted: Sat Aug 31, 2019 11:17 pm Post subject: |
|
|
This is just a way to get the current filepointer location
It moves the current filepointer by 0 and stores that in the address pointed at by r8 (local var in this case)
There is likely another setfileposition call somewhere else that sets it, or it's done by closing and reading useless data to get back
_________________
Do not ask me about online cheats. I don't know any and wont help finding them.
Like my help? Join me on Patreon so i can keep helping |
|
Back to top |
|
|
predprey Master Cheater Reputation: 24
Joined: 08 Oct 2015 Posts: 486
|
Posted: Sun Sep 01, 2019 12:48 am Post subject: |
|
|
Thanks, I had doubts on whether it was really just getting the current file pointer even though the ASM code and calling convention rules says it is. The file pointer most likely got updated before this routine got executed because double checking with the file itself, RDI is always the position to be read in next. Now I'm just puzzled how the game is moving its pointer.
Backtracing from a breakpoint within the kernel function itself only gave me this location that calls SetFilePointerEx for this exact file handle though. All other calls for SetFilePointerEx handles other files. Game doesn't break on SetFilePointer either.
Will closing an existing file handle and opening another for the same file return the same file handle value? The file handle for the same file remains the same for the same session though it may change between different runs.
|
|
Back to top |
|
|
Dark Byte Site Admin Reputation: 458
Joined: 09 May 2003 Posts: 25296 Location: The netherlands
|
Posted: Sun Sep 01, 2019 12:54 am Post subject: |
|
|
It's impossible to predict if the filehandle value will be different when it gets closed and reopened. Best assume different
Perhaps instead of normal read and writes it uses filemapping
_________________
Do not ask me about online cheats. I don't know any and wont help finding them.
Like my help? Join me on Patreon so i can keep helping |
|
Back to top |
|
|
predprey Master Cheater Reputation: 24
Joined: 08 Oct 2015 Posts: 486
|
Posted: Sun Sep 01, 2019 2:18 am Post subject: |
|
|
Dark Byte wrote: | It's impossible to predict if the filehandle value will be different when it gets closed and reopened. Best assume different
Perhaps instead of normal read and writes it uses filemapping |
Thanks, I will go read up on filemapping. And maybe backtrace from the screenshotted location to see how the handle gets created.
|
|
Back to top |
|
|
|