Cheat Engine

[mgr.inz.Player]

For now, first scan, scan type 'unknown initial value' works like this:
- it dumps all memory (depends on memory scan options) to temporary files.

My idea for improvement is: using FAST hash algorithm. Not CRC32.

Something like MD5 or SHA1 or even better, one of those (those are noncryptographic):
xxHash
xxHash256
MurmurHash
CityHash
BLAKE2S (cryptographic)


This scan type could be called: "unknown initial value (hash method)"
I think, 256 bytes chunk/block should be enough. After first scan, "unknown initial value (hash method)", we will see:
"Found: 947812 chunks"

instead of 242639872 addresses.


Of course next scan type will be only:
- changed
- unchanged

We took "changed" and after scan we will see "Found: 166562 chunks"

Of course, after 1st (2nd, 3rd, 4th, .....) NEXT SCAN, if found counter will be less than 39060 chunks ( about 10'000'000 addresses),
then, from now on, "NEXT SCAN" will work normally (it will save values instead of hashes), "increased, smaller, ..." will be available again.

One address (address of chunk) will convert to 256 addresses.



Of course I wanted to make something like this by using "custom type". (with implemented simple fast hash mentioned before, or cryptographic hash). When first scan or next scan have more than 39060 chunks, AA script will return 32bit hash and we can use changed/unchanged scan type. When less than 39060 chunks, AA script will return normal 32bit int value or 32bit float.


But there are some problems:

* we can not change custom type behavior, depending on current found chunks
Well, I could use something like this, inside "custom type AA":

[Dark Byte]

Actually, unknown initial value doesn't dump it to a tempfile, but it does generate an "first scan" file if you wish to compare against it later on with compare to first scan (it's a feature but you can remove it for speed gains)

(With an unknown initial value scan, the whole memory of the game is saved in ce's memory for the first next scan)

Also, the first scan unknown initial value scans are not stored as addresses but address ranges that are used to lookup the old value in the memory, which is just the process memory in a contiguous form

The number of addresses found is an calculation based on the total memory read divided by the alignment

When you do a next scan, it will compare the memory stored in ce with the memory of the game, and if any byte doesn't match check if the new value adheres to the scan parameters.

For the first next scan hashes probably wouldn't make a difference, since to generate a hash every byte has to be read of the chunk you make a hash for, and you might as well use that time to compare it against the memory that's already stored

-
Of course, the changed/unchanged method for next next scan's might be something to look into if the value type is too big

[mgr.inz.Player]

[Dark Byte]

first next scan doesn't use ADDRESSES.FIRST at all. At most it's still waiting for the first scan to finish
When a scan is done, it's not really done. It's still saving some data away for later usage (like the first scan results) but you can already use it. If you do a next scan, it has to wait till the scan is really done (memscan.waittillreallydone) and only then start a new scan

But yes, an unchanged value will generate a huge result set and there a faster harddisk can be useful

The memory ranges thing isn't something I can export. It's too entangled

I'm not sure, but if I get time I may look into making a low memory/disk usage scan (I need to anyhow for this other ce project I work on)
--
Just wondering, is this firefox? Because what you describe is the paging file of windows constantly reading/writing as well. And due to firefox's gigantic memory leaks it sends off unused memory to the swap, but when ce scans it it gets loaded back in

[mgr.inz.Player]

Nope, it's Metro LL. I want to find flags, and those can be everywhere, this is why I use *X*W protection flags.


"unused memory to the swap, but when ce scans it it gets loaded back in"
Metro LL too. At least under WinXP.



I think I can try make my own "unknown scans", but I can't start without returnRangesAsTable I mentioned in "second edit".

[Dark Byte]

addresses.first contains those ranges in this format (TMemoryregion):
http://code.google.com/p/cheat-engine/source/browse/trunk/Cheat%20Engine/CEFuncProc.pas#52

[mgr.inz.Player]

"Also, if you just want the memory regions, you might have more luck spawning the memory regions windows and read the output from the listview (get memory browser, get main menu, find view menu item, find memory regions submenu, and execute it's onClick)"

Those regions visible inside that window are used by memscan objects. Not quite?

I have to gather all regions which are:
- State ~= 'Free'
- Type == 'Image'

And just get Address and Size. Right ?

[Dark Byte]

State=='Commit'
Protect should have at least "Write" (else it'd be useless to do an changed/unchanged memory) and you can skip 'Guard' and "Write Copy"
Type should be 'Private' and optionally 'Image' (Image means it's inside a module, static)

And then just Address and Size

--
I do understand what you mean :
Make the "Hashed region scan" (needs a better name) a completely separated scan from the normal scan routines (Do not modify the existing scan routines and probably a different gui as well)
Then when you're left with a reasonable bunch of chunks either press a button to convert these results into scanfiles that the normal memory scanner can use. (address/value groups of the wanted type), or do it automatically
And from that point do the normal scans

It 'might' be possible with lua to do that last step if you make use of the undocumented property : ScanresultFolder
Disable the foundlist, swap out the scanfiles with your own an reinitialize

It needs a memscan that has already scanned once though, but that can be a range of 00400000 to 00400001

[mgr.inz.Player]

Interesting, any example? Thanks.

Now I make function, function to grab those "regions".

WIP:

[Dark Byte]

( Address is item[x].Caption )

registerFormAddNotification example:

[mgr.inz.Player]

Delay is necessary....

Hash function (injected ASM) and abnormal scan routines (unknown initial hash, first, next changed, next unchanged) shouldn't be a problem.


The problem is: how to convert these results (chunk addresses) into scanfiles. And of course how exactly ScanresultFolder property works.

[Dark Byte]

Abnormal scan routines means you'll have to do your own ReadProcessMemory calls as memscan will not be useful in this case. (I really do recommend plugin dlls)

Chunk address to scanfile isn't too difficult. The addressfile is just an array of addresses (using 4 or 8 bytes/entry depending on which ce is used) and memory is an array of chunks with the varlength of the current type
So just split up the chunks into separate addresses based on if fastscan alignment is on or not and fill in the memory file with the current memory of those addresses

The addresslist starts with the text "NORMAL" followed by a 0 terminator

The ScanresultFolder property of a memscan object returns the path where it stores the temp files during it's scan, so you don't have to guess where it is