Cheat Engine Forum Index Cheat Engine
The Official Site of Cheat Engine
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 


Calling convention for KERNEL32.SetFilePointerEx

 
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine
View previous topic :: View next topic  
Author Message
predprey
Master Cheater
Reputation: 18

Joined: 08 Oct 2015
Posts: 451

PostPosted: Sat Aug 31, 2019 9:51 pm    Post subject: Calling convention for KERNEL32.SetFilePointerEx Reply with quote

I understand that most x64 executables compiled for Windows uses MS's x64 calling convention. So far, that has yielded no problems and I interpret the arguments passed into a function as rcx,rdx,r8,r9 etc. However, while reversing the file structure for Zone of Enders, I encountered this behaviour for Window's SetFilePointerEx function which goes against the argument passing rule of the above convention. As shown in the screenshot below, the offset to move the file pointer to seems to be stored in RDI instead of RDX. The syntax for SetFilePointerEx is
Code:
BOOL SetFilePointerEx(
  HANDLE         hFile,
  LARGE_INTEGER  liDistanceToMove,
  PLARGE_INTEGER lpNewFilePointer,
  DWORD          dwMoveMethod
);

So going by MS's calling convention, the distance to move should be stored in RDX. However, the screenshot seems to suggest the it is pass through RDI instead. I couldn't get the process to break on SetFilePointer so it seems only SetFilePointerEx is used to move the pointer. Disassembling the ASM in the screenshot suggests the routine is used to get the current file pointer instead of moving it which explains how RDX is XOR-ed and dwMoveMethod is 1 for FILE_CURRENT. However, I'm not sure how the game is moving the file pointer backwards as RDI moves back and forths when breaking on the screenshotted location. Can someone explain what is going on here exactly? Thanks.



SS.jpg
 Description:
 Filesize:  254.03 KB
 Viewed:  219 Time(s)

SS.jpg


Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 370

Joined: 09 May 2003
Posts: 21837
Location: The netherlands

PostPosted: Sat Aug 31, 2019 11:17 pm    Post subject: Reply with quote

This is just a way to get the current filepointer location

It moves the current filepointer by 0 and stores that in the address pointed at by r8 (local var in this case)

There is likely another setfileposition call somewhere else that sets it, or it's done by closing and reading useless data to get back

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
predprey
Master Cheater
Reputation: 18

Joined: 08 Oct 2015
Posts: 451

PostPosted: Sun Sep 01, 2019 12:48 am    Post subject: Reply with quote

Thanks, I had doubts on whether it was really just getting the current file pointer even though the ASM code and calling convention rules says it is. The file pointer most likely got updated before this routine got executed because double checking with the file itself, RDI is always the position to be read in next. Now I'm just puzzled how the game is moving its pointer.

Backtracing from a breakpoint within the kernel function itself only gave me this location that calls SetFilePointerEx for this exact file handle though. All other calls for SetFilePointerEx handles other files. Game doesn't break on SetFilePointer either.

Will closing an existing file handle and opening another for the same file return the same file handle value? The file handle for the same file remains the same for the same session though it may change between different runs.
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 370

Joined: 09 May 2003
Posts: 21837
Location: The netherlands

PostPosted: Sun Sep 01, 2019 12:54 am    Post subject: Reply with quote

It's impossible to predict if the filehandle value will be different when it gets closed and reopened. Best assume different

Perhaps instead of normal read and writes it uses filemapping

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
predprey
Master Cheater
Reputation: 18

Joined: 08 Oct 2015
Posts: 451

PostPosted: Sun Sep 01, 2019 2:18 am    Post subject: Reply with quote

Dark Byte wrote:
It's impossible to predict if the filehandle value will be different when it gets closed and reopened. Best assume different

Perhaps instead of normal read and writes it uses filemapping


Thanks, I will go read up on filemapping. And maybe backtrace from the screenshotted location to see how the handle gets created.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group

CE Wiki   IRC (#CEF)   Twitter
Third party websites