Cheat Engine Forum Index Cheat Engine
The Official Site of Cheat Engine
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 


C++ - How to scan memory so fast?

 
Post new topic   Reply to topic    Cheat Engine Forum Index -> General programming
View previous topic :: View next topic  
Author Message
DaviFN
Cheater
Reputation: 0

Joined: 23 Oct 2016
Posts: 32

PostPosted: Fri Nov 18, 2016 9:22 am    Post subject: C++ - How to scan memory so fast? Reply with quote

Hello everyone!

I need to write a function that searches the memory of a process (I'm injected to it, with a DLL) for a specific array of bytes. I wonder the best (i.e. fastest) way to do that, just like CE does it.

I've done searches in files before, and it was really fast, but my memory searches are extremely slow.I basically get the starting address using GetModuleHandle() and implement a loop with a lot of VirtualQuerys, ignoring memory pages I don't want (I only want read/write memory), and jumping directly to the next page if I ignore it (I think I made it wrong, but anyway...). So I tried to save the entire memory of the process into a file, but I failed to do so. I've to say that my C++ programming skills are bad, since it's just a hobby for me.

This code is some scratch I tested. How can I make something better?

Code:
DWORD base=(DWORD)GetModuleHandle("Minesweeper.exe");
DWORD scanning=base;
BYTE scanned;
MEMORY_BASIC_INFORMATION mbi;
while(1)
{
if(VirtualQuery((LPVOID)scanning,&mbi,sizeof(mbi))!=0)
{
if(mbi.Protect&PAGE_READWRITE)
{
memcpy(&scanned,(LPVOID)scanning,1);
if(scanned==1)
{         
//please don't laugh...
}
if(scanning==0xffffffff){MessageBox(NULL,"Nothing found, scanning again in 3...2...","!",MB_OK);scanning=base-1;}
}else{scanning=scanning+mbi.RegionSize-1;}
}
scanning++;
}




So my question is: How can I create such a fast memory scan? Is it best to save the whole memory into a file and then search this file, or search directly in memory? (the array of bytes I'm searching for doesn't change)


Examples would be greatly appreciated!
Back to top
View user's profile Send private message
Zanzer
I post too much
Reputation: 126

Joined: 09 Jun 2013
Posts: 3278

PostPosted: Fri Nov 18, 2016 9:39 am    Post subject: Reply with quote

Good example of how to do it just like CE does it can be found here:
https://github.com/cheat-engine/cheat-engine/

I'm sure there are a ton of threads in this very forum with C++ specific examples.

Always do your work in memory. No reason to bring a slow hard drive into the picture!
Back to top
View user's profile Send private message
STN
I post too much
Reputation: 43

Joined: 09 Nov 2005
Posts: 2676

PostPosted: Fri Nov 18, 2016 10:07 am    Post subject: Reply with quote

Multi-threading
_________________
Cheat Requests/Tables- Fearless Cheat Engine
https://fearlessrevolution.com
Back to top
View user's profile Send private message
atom0s
Moderator
Reputation: 205

Joined: 25 Jan 2006
Posts: 8587
Location: 127.0.0.1

PostPosted: Fri Nov 18, 2016 2:01 pm    Post subject: Reply with quote

Here is my personal method I made and use:
Code:
        /**
         * Locates a pattern of data within a large block of memory.
         *
         * @param {std::vector} data - The data to scan for the pattern within.
         * @param {uintptr_t} baseAddress - The base address to add to the offset when the pattern is found.
         * @param {const char*} pattern - The pattern to scan for.
         * @param {uintptr_t} offset - The offset to add after the pattern has been found.
         * @param {uintptr_t} count - The result count to use if the pattern is found multiple times.
         * @returns {uintptr_t} The address where the pattern was located, 0 otherwise.
         */
        static uintptr_t FindPattern(std::vector<uint8_t> data, uintptr_t baseAddress, const char* pattern, uintptr_t offset, uintptr_t count)
        {
            // Ensure the incoming pattern is properly aligned..
            if (strlen(pattern) % 2 > 0)
                return 0;

            // Convert the pattern to a vector of data..
            std::vector<std::pair<uint8_t, bool>> vpattern;
            for (size_t x = 0, y = strlen(pattern) / 2; x < y; x++)
            {
                // Obtain the current byte..
                std::stringstream stream(std::string(pattern + (x * 2), 2));

                // Check if this is a wildcard..
                if (stream.str() == "??")
                    vpattern.push_back(std::make_pair(00, false));
                else
                {
                    auto byte = strtol(stream.str().c_str(), nullptr, 16);
                    vpattern.push_back(std::make_pair((uint8_t)byte, true));
                }
            }

            auto scanStart = data.begin();
            auto result = (uintptr_t)0;

            while (true)
            {
                // Search for the pattern..
                auto ret = std::search(scanStart, data.end(), vpattern.begin(), vpattern.end(),
                    [&](uint8_t curr, std::pair<uint8_t, bool> currPattern)
                {
                    return (!currPattern.second) || curr == currPattern.first;
                });

                // Did we find a match..
                if (ret != data.end())
                {
                    // If we hit the usage count, return the result..
                    if (result == count || count == 0)
                        return (std::distance(data.begin(), ret) + baseAddress) + offset;

                    // Increment the found count and scan again..
                    ++result;
                    scanStart = ++ret;
                }
                else
                    break;
            }

            return 0;
        }


An example of using this:
Code:

    // Obtain the module information..
    MODULEINFO mod = { 0 };
    if (!::GetModuleInformation(::GetCurrentProcess(), ::GetModuleHandle("Game.dll"), &mod, sizeof(MODULEINFO)))
        return false;

    // Read the module memory into a vector..
    std::vector<uint8_t> raw((uint8_t*)mod.lpBaseOfDll, (uint8_t*)mod.lpBaseOfDll + mod.SizeOfImage);

    // Scan for a pattern..
    auto result = FindPattern(std::ref(data), (uintptr_t)mod.lpBaseOfDll, "112233??AABBCC??99001122", 1, 0);


I recommend you loop the pages of the memory and scan within those rather than dumping the whole module, but it depends on your target and its protection levels etc.

Allows you to scan within a vector of data (if you are injected you can cast the memory region to a vector easily). Coupled with std::async and std::shared_future objects, you can make it async and extremely fast. I search for over 40-50 patterns in an MMO for one of my tool and using this approach I can find all of them in under half a 10th of a second.

If you are getting into searching / pattern finding, I really recommend that you look into multi-threading aspects of things, especially with C++11/14's new features of std::async and std::shared_future. They are extremely powerful when used properly.

_________________
- Retired.
Back to top
View user's profile Send private message Visit poster's website
DaviFN
Cheater
Reputation: 0

Joined: 23 Oct 2016
Posts: 32

PostPosted: Fri Nov 18, 2016 3:08 pm    Post subject: Reply with quote

Thanks for posting your method, moderator atom0s, but I couldn't compile it... (maybe I'm too dumb)

I'm using Dev-C++ and got the following errors:

    [quote]line 16: 'vector' was not declared in this scope
    line 16: expected primary-expression before '>' token
    line 16: 'data' was not declared in this scope
    line 16: expected primary-expression before "baseAddress"
    line 16: expected primary-expression before "const"
    line 16: expected primary-expression before "offset"
    line 16: expected primary-expression before "count"
    line 17: initializer expression list treated as compound expression
    line 17: expected ',' or ';' before '{' token[
Back to top
View user's profile Send private message
atom0s
Moderator
Reputation: 205

Joined: 25 Jan 2006
Posts: 8587
Location: 127.0.0.1

PostPosted: Fri Nov 18, 2016 9:13 pm    Post subject: Reply with quote

If you are doing Windows development, I suggest that you switch to Visual Studio. Dev-C++ is extremely outdated and really not the best of things to use.
Visual Studio is free if you use the Community edition, which has very little stripped from it.
https://www.visualstudio.com/free-developer-offers/

As for the compiler errors, you need to add some headers.

std::vector requires:
#include <vector>

The standard integer types use:
#include <stdint.h>
or
#include <cinttypes>

std::stringstream uses:
#include <sstream>

I'd suggest you learn the basics before jumping into things like this as these warnings/errors are pretty easy to fix and figure out if you know what you are doing.

_________________
- Retired.
Back to top
View user's profile Send private message Visit poster's website
DaviFN
Cheater
Reputation: 0

Joined: 23 Oct 2016
Posts: 32

PostPosted: Sat Nov 19, 2016 9:36 am    Post subject: Reply with quote

I'm going to download Visual Studio. Thank you for your recommendation!

One more thing: Would it be faster than my current approach to get an entire page with ReadProcessMemory() and then search for a pattern in it? What is faster, RPM or memcpy?

Is reading 1000 bytes (example) into an array and then analysing that array faster than reading and analysing byte per byte?

And, besides RPM and memcpy, what other options I have to search the memory?
Back to top
View user's profile Send private message
atom0s
Moderator
Reputation: 205

Joined: 25 Jan 2006
Posts: 8587
Location: 127.0.0.1

PostPosted: Sat Nov 19, 2016 11:47 am    Post subject: Reply with quote

If you can, avoid ReadProcessMemory always, as API like itself have a lot of overhead for usage.

You should be reading the entire page at a time for speed/optimization. As long as the page has proper access you should have no problem handling the whole thing at a time.

You can directly cast the memory into a readable vector if you are injected directly into the process without needing to memcpy or ReadProcessMemory which will be the fastest option.

If you are reading externally, then you are stuck using ReadProcessMemory or its direct Nt counterpart.

_________________
- Retired.
Back to top
View user's profile Send private message Visit poster's website
DaviFN
Cheater
Reputation: 0

Joined: 23 Oct 2016
Posts: 32

PostPosted: Sat Nov 19, 2016 1:07 pm    Post subject: Reply with quote

Can I verify if a certain byte at a memory location using inline assembly? And would it be faster?
Back to top
View user's profile Send private message
atom0s
Moderator
Reputation: 205

Joined: 25 Jan 2006
Posts: 8587
Location: 127.0.0.1

PostPosted: Sat Nov 19, 2016 3:15 pm    Post subject: Reply with quote

Yes, you can. As for it being faster, really depends on the code you are making in the parent language. Everything is compiled down to some form of assembly then to machine code regardless of the language you use. The performance of what is generated depends on your coding style as well as the compilers ability to optimize things.
_________________
- Retired.
Back to top
View user's profile Send private message Visit poster's website
H4x0rBattie
Advanced Cheater
Reputation: 0

Joined: 10 Nov 2016
Posts: 58

PostPosted: Wed Nov 23, 2016 3:14 am    Post subject: Reply with quote

atom0s wrote:
If you can, avoid ReadProcessMemory always, as API like itself have a lot of overhead for usage.

You should be reading the entire page at a time for speed/optimization. As long as the page has proper access you should have no problem handling the whole thing at a time.

You can directly cast the memory into a readable vector if you are injected directly into the process without needing to memcpy or ReadProcessMemory which will be the fastest option.

If you are reading externally, then you are stuck using ReadProcessMemory or its direct Nt counterpart.


What exactly is it's NT counterpart?

_________________
Back to top
View user's profile Send private message
atom0s
Moderator
Reputation: 205

Joined: 25 Jan 2006
Posts: 8587
Location: 127.0.0.1

PostPosted: Wed Nov 23, 2016 3:26 am    Post subject: Reply with quote

NtReadVirtualMemory and ZwReadVirtualMemory.
_________________
- Retired.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Cheat Engine Forum Index -> General programming All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group

CE Wiki   IRC (#CEF)   Twitter
Third party websites