Cheat Engine Forum Index Cheat Engine
The Official Site of Cheat Engine
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 


Reading process total physical memory.
Goto page Previous  1, 2
 
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine Lua Scripting
View previous topic :: View next topic  
Author Message
mgr.inz.Player
I post too much
Reputation: 218

Joined: 07 Nov 2008
Posts: 4438
Location: W kraju nad Wisla. UTC+01:00

PostPosted: Tue Nov 26, 2013 10:50 pm    Post subject: Reply with quote

For now, I changed your script to this:
Code:
SmartAttach =
 {
  wanted_processes = { 'chrome.exe','plugin%-container.exe',
                       'flashplayerplugin','opera.exe','iexplore.exe' }
                       ,
  currentprocesslist = {}
 }

function SmartAttach:Start(  )
  self.form  =  createForm( false )
  setProperty( self.form ,  "Position",  "poScreenCenter" )
  setProperty( self.form ,  "BiDiMode",  "bdLeftToRight" )
  self.form.Caption  =  'Smart attach test'
  self.form.height  =  70
  self.form.width  =  400
  self.combobox  =  createComboBox( self.form )
  self.combobox.top  =  10
  self.combobox.left  =  10
  self.combobox.width  =  220
  self.combobox.ReadOnly  =  true
  self.button  =  createButton( self.form )
  self.button.Caption  =  'Attach'
  self.button.top  =  10
  self.button.left  =  240
  self.button.onClick  =  function ( sender ) self:Attach( sender ) end
  self.button.width  =  45
  self.pickhighestmem  =  createButton( self.form )
  self.pickhighestmem.Caption  =  'Refresh list'
  self.pickhighestmem.top  =  10
  self.pickhighestmem.left  =  300
  self.pickhighestmem.width  =  95
  self.pickhighestmem.onClick  =  function ( sender ) self:RefreshList( sender ) end
  self.form.show()
  SmartAttach:RefreshList()
end


function SmartAttach.SortByKey(T,K,reverse)
  if reverse==true then table.sort(T, function (a,b) return a[K] > b[K] end)
                   else table.sort(T, function (a,b) return a[K] < b[K] end) end
  return T
end

function SmartAttach.ProcessMatch(s)
  for _,v in pairs(SmartAttach.wanted_processes) do
    if s:match(v:lower()) then return true end
  end
  return false
end

function SmartAttach.gatherData()
  local handle  =  io.popen( "tasklist.exe /fo csv" )
  local processTable = {}
  local skip = true
  local tmp = nil
  for line in handle:lines() do
    if skip then skip = false
    else
      tmp = {}
      for t in line:gmatch('\"(.-)\"') do
       table.insert(tmp,t)
      end
      local processName = tmp[1]
      if SmartAttach.ProcessMatch(processName:lower()) then
        local PID         = tmp[2]
      --local Session     = tmp[3]
      --local SessionNum  = tmp[4]
        local MemoryUsage = tmp[5]:gsub("[^%d]",'')+0  -- +0 converts to number

        local entry = { processName=processName, PID=PID, MemoryUsage=MemoryUsage }
        processTable[#processTable+1] = entry
      end
    end
  end
  handle:close()
  return SmartAttach.SortByKey(processTable,'MemoryUsage',true)
end

function SmartAttach.RefreshList()
  local items = SmartAttach.combobox.items
  items.clear()
  SmartAttach.currentprocesslist = SmartAttach.gatherData()
  if ( #SmartAttach.currentprocesslist > 0 ) then
    for _,v in ipairs( SmartAttach.currentprocesslist ) do
      strings_add(items, string.format('%.1f',v.MemoryUsage/1024)..'MB'..' - '..v.processName)
    end
  SmartAttach.combobox.ItemIndex  =  0
  else
    return showMessage( "Couldn't find any useful process" )
  end
end

function SmartAttach:Attach(sender)
  local index = SmartAttach.combobox.ItemIndex
  if index < 0 then return end
  openProcess( tonumber(SmartAttach.currentprocesslist[index + 1].PID) )
end

SmartAttach:Start()




"Also, I won't change yet the title, maybe let's try to create the memory scan based method?"
OK, when I find more free time.

_________________


Last edited by mgr.inz.Player on Thu Nov 28, 2013 1:48 pm; edited 1 time in total
Back to top
View user's profile Send private message MSN Messenger
DaSpamer
Grandmaster Cheater Supreme
Reputation: 52

Joined: 13 Sep 2011
Posts: 1578

PostPosted: Thu Nov 28, 2013 6:51 am    Post subject: Reply with quote

Nice job.
Although you should divide the MemoryUsage by 100 to get the right memory amount.

And I'm facing now another issue.
I'm trying to getAddress of the .dll inside the browser.
But facing HUGE memory usage by cheat engine (150MB - 450MB per cycle).

This for example, does not consume much memory.
Code:
getAddress("kernel32.dll")~=0

But this does
Code:
getAddress("pepflashplayer.dll")~=0


So my question, how to use as less as possible memory?
Nulling vars and clearing up stuff after usage and using collectgarbage() does not effect.

I'm assuming Dark Byte calls this function
Code:
reinitializeSymbolhandler();

When trying to get external .dll?
(I've tested few stuff, and when calling this function, the memory jumps by 150MB if not more (11 processes) ).


I mean this is not a problem for me (8GB ram are enough).
But there's people who has 2GB ram, and if they attach/refresh list and I verify the process based .dll addresses, they'll run out of RAM (without restarting the trainer).

_________________
HEY Hitler
Do you get lazy when making trainers?
Well no more!
My CETrainer will generate it for you in seconds, so you won't get lazy! Very Happy

http://forum.cheatengine.org/viewtopic.php?t=564919
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 458

Joined: 09 May 2003
Posts: 25296
Location: The netherlands

PostPosted: Thu Nov 28, 2013 7:02 am    Post subject: Reply with quote

Where do you check memory usage ?
Does the amount of committed memory to the CE process increase ? The working set size?
Or do you just look at the total physical memory ? (That's a bad display)

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
DaSpamer
Grandmaster Cheater Supreme
Reputation: 52

Joined: 13 Sep 2011
Posts: 1578

PostPosted: Thu Nov 28, 2013 7:41 am    Post subject: Reply with quote

Dark Byte wrote:
Where do you check memory usage ?
Does the amount of committed memory to the CE process increase ? The working set size?
Or do you just look at the total physical memory ? (That's a bad display)

Here's image, of using mgr.inz.Player script (without using getAddress or any of the 'reading' address functions').

Here's image of my script (same as mgr.inz.Player, just filtering the 14 processes to relevant ones for me, by using the getAddress('wanted.dll'))



The more processes are being opened, the more RAM, it'll use.

_________________
HEY Hitler
Do you get lazy when making trainers?
Well no more!
My CETrainer will generate it for you in seconds, so you won't get lazy! Very Happy

http://forum.cheatengine.org/viewtopic.php?t=564919
Back to top
View user's profile Send private message
mgr.inz.Player
I post too much
Reputation: 218

Joined: 07 Nov 2008
Posts: 4438
Location: W kraju nad Wisla. UTC+01:00

PostPosted: Thu Nov 28, 2013 12:39 pm    Post subject: Reply with quote

DaSpamer wrote:
Nice job.
Although you should divide the MemoryUsage by 100 to get the right memory amount.

Looks like tasklist output is different and depends on operating system language. Could you post output of "tasklist.exe /fo csv" from your machine ?
(first three lines is enough)

I will change "pattern" so it should work for everyone.

_________________
Back to top
View user's profile Send private message MSN Messenger
DaSpamer
Grandmaster Cheater Supreme
Reputation: 52

Joined: 13 Sep 2011
Posts: 1578

PostPosted: Thu Nov 28, 2013 1:33 pm    Post subject: Reply with quote

mgr.inz.Player wrote:
DaSpamer wrote:
Nice job.
Although you should divide the MemoryUsage by 100 to get the right memory amount.

Looks like tasklist output is different and depends on operating system language. Could you post output of "tasklist.exe /fo csv" from your machine ?
(first three lines is enough)

I will change "pattern" so it should work for everyone.

here you go.
Code:
"Image Name","PID","Session Name","Session#","Mem Usage"
"System Idle Process","0","Services","0","24 K"
"System","4","Services","0","3,100 K"
"smss.exe","460","Services","0","2,396 K"
"csrss.exe","656","Services","0","13,256 K"
"csrss.exe","728","Console","1","124,872 K"
"wininit.exe","736","Services","0","4,716 K"
"winlogon.exe","772","Console","1","16,296 K"
"services.exe","832","Services","0","12,704 K"
"lsass.exe","840","Services","0","13,312 K"
"lsm.exe","848","Services","0","4,728 K"
"svchost.exe","936","Services","0","18,180 K"
"nvvsvc.exe","1000","Services","0","7,592 K"
"svchost.exe","248","Services","0","18,588 K"
"svchost.exe","712","Services","0","47,312 K"
"svchost.exe","532","Services","0","197,436 K"
"svchost.exe","1048","Services","0","37,696 K"
"svchost.exe","1072","Services","0","48,180 K"
"svchost.exe","1176","Services","0","14,108 K"
"SbieSvc.exe","1264","Services","0","4,176 K"
"svchost.exe","1360","Services","0","20,564 K"
"nvxdsync.exe","1508","Console","1","37,728 K"
"nvvsvc.exe","1516","Console","1","23,628 K"
"spoolsv.exe","1776","Services","0","20,544 K"
"explorer.exe","1812","Console","1","170,384 K"
"dwm.exe","1832","Console","1","597,880 K"
"taskhost.exe","1904","Console","1","24,068 K"
"svchost.exe","1916","Services","0","26,596 K"
"avp.exe","1732","Services","0","38,320 K"
"ProtectedObjectsSrv.exe","2100","Services","0","4,908 K"
"HeciServer.exe","2152","Services","0","13,312 K"
"Jhi_service.exe","2212","Services","0","4,580 K"
"LMIGuardianSvc.exe","2248","Services","0","15,440 K"
"sqlservr.exe","2308","Services","0","8,656 K"
"nvtray.exe","2380","Console","1","25,360 K"
"RtkNGUI64.exe","2508","Console","1","31,564 K"
"NvTmru.exe","2688","Console","1","9,128 K"
"System 32 - GPU Booster.exe","2724","Console","1","47,416 K"
"nvstreamsvc.exe","2732","Services","0","9,032 K"
"Origin.exe","2820","Console","1","112,036 K"
"RDVGHelper.exe","1148","Console","1","12,960 K"
"nvstreamsvc.exe","2532","Console","1","23,380 K"
"conhost.exe","116","Console","1","4,732 K"
"daemonu.exe","2568","Services","0","14,000 K"
"PnkBstrA.exe","3328","Services","0","7,672 K"
"avp.exe","3336","Console","1","8,856 K"
"jusched.exe","3364","Console","1","4,744 K"
"RtlService.exe","3380","Services","0","4,528 K"
"LWS.exe","3400","Console","1","12,228 K"
"SRService.exe","3648","Services","0","4,872 K"
"sqlwriter.exe","3720","Services","0","6,668 K"
"SRServer.exe","3728","Console","1","14,908 K"
"SSUService.exe","3772","Services","0","5,456 K"
"svchost.exe","3820","Services","0","16,324 K"
"CameraHelperShell.exe","3940","Console","1","17,112 K"
"SRFeature.exe","3460","Console","1","15,020 K"
"firefox.exe","4584","Console","1","956,436 K"
"SearchIndexer.exe","4368","Services","0","34,388 K"
"svchost.exe","5076","Services","0","5,844 K"
"alg.exe","5444","Services","0","12,708 K"
"wmpnetwk.exe","5568","Services","0","11,968 K"
"svchost.exe","5832","Services","0","26,984 K"
"LMS.exe","724","Services","0","5,280 K"
"svchost.exe","6116","Services","0","102,400 K"
"Steam.exe","3508","Console","1","134,900 K"
"Skype.exe","5144","Console","1","191,288 K"
"UNS.exe","6080","Services","0","11,432 K"
"plugin-container.exe","4208","Console","1","46,052 K"
"FlashPlayerPlugin_11_9_900_117.exe","5512","Console","1","12,792 K"
"FlashPlayerPlugin_11_9_900_117.exe","3576","Console","1","200,352 K"
"audiodg.exe","2340","Services","0","22,952 K"
"notepad++.exe","4724","Console","1","33,592 K"
"chrome.exe","5904","Console","1","110,160 K"
"chrome.exe","5484","Console","1","76,332 K"
"chrome.exe","2304","Console","1","69,372 K"
"chrome.exe","6328","Console","1","54,556 K"
"chrome.exe","6500","Console","1","53,820 K"
"chrome.exe","6640","Console","1","95,668 K"
"chrome.exe","6832","Console","1","45,680 K"
"chrome.exe","6956","Console","1","52,448 K"
"opera.exe","7024","Console","1","154,692 K"
"chrome.exe","6196","Console","1","27,012 K"
"plugin-container.exe","6740","Console","1","15,344 K"
"plugin-container.exe","8048","Console","1","12,808 K"
"Winject.exe","1040","Console","1","9,348 K"
"wmplayer.exe","4092","Console","1","85,524 K"
"WUDFHost.exe","4388","Services","0","16,432 K"
"chrome.exe","9224","Console","1","44,684 K"
"cheatengine-x86_64.exe","9308","Console","1","35,312 K"
"WmiPrvSE.exe","10116","Services","0","7,720 K"
"cmd.exe","8052","Console","1","2,824 K"
"conhost.exe","5308","Console","1","5,320 K"
"tasklist.exe","9516","Console","1","5,848 K"

_________________
HEY Hitler
Do you get lazy when making trainers?
Well no more!
My CETrainer will generate it for you in seconds, so you won't get lazy! Very Happy

http://forum.cheatengine.org/viewtopic.php?t=564919
Back to top
View user's profile Send private message
mgr.inz.Player
I post too much
Reputation: 218

Joined: 07 Nov 2008
Posts: 4438
Location: W kraju nad Wisla. UTC+01:00

PostPosted: Thu Nov 28, 2013 1:46 pm    Post subject: Reply with quote

lol, first three lines.

Mine looks like this:
Code:
"Nazwa obrazu","PID","Nazwa sesji","Nr sesji","Użycie pam."
"System Idle Process","0","Services","0","24 KB"
"System","4","Services","0","324 KB"
"smss.exe","292","Services","0","568 KB"
"csrss.exe","440","Services","0","2 400 KB"


In my script there is
:gsub("[^%w]",'') - it removes all non "word" characters (spaces tabs dots semicolons)
:sub(1,-3)+0 - it removes last two chars and converts to number

You don't have last "B".

Anyway, better "pattern":
Code:
local MemoryUsage = tmp[5]:gsub("[^%d]",'')+0


gsub("[^%d]",'')+0 - it removes everything except digits and converts to number

_________________
Back to top
View user's profile Send private message MSN Messenger
DaSpamer
Grandmaster Cheater Supreme
Reputation: 52

Joined: 13 Sep 2011
Posts: 1578

PostPosted: Thu Nov 28, 2013 2:32 pm    Post subject: Reply with quote

Cool thanks mate.
Can you test this script (you, should adjust it, it might take up 1-2GB ram).
Code:
local handle = io.popen( "tasklist.exe /fo csv");
for line in handle:lines() do
   local data_table = {}
   for t in line:gmatch('\"(.-)\"') do
      table.insert(data_table,t)
   end
   reinitializeSymbolhandler();
   openProcess(tonumber(data_table[2]));
end
handle:close();

To see how much memory it uses of your PC ram.

_________________
HEY Hitler
Do you get lazy when making trainers?
Well no more!
My CETrainer will generate it for you in seconds, so you won't get lazy! Very Happy

http://forum.cheatengine.org/viewtopic.php?t=564919
Back to top
View user's profile Send private message
mgr.inz.Player
I post too much
Reputation: 218

Joined: 07 Nov 2008
Posts: 4438
Location: W kraju nad Wisla. UTC+01:00

PostPosted: Thu Nov 28, 2013 2:58 pm    Post subject: Reply with quote

Indeed using

reinitializeSymbolhandler
openProcess

in the loop will increase memory usage.

But, openProcess already is doing "initializeSymbolhandler" so we don't need reinitializeSymbolhandler.

Better this
Code:
local handle = io.popen( "tasklist.exe /fo csv");
local iterator = handle:lines() -- get iterator and assign it to "iterator"
local firstline = iterator()        -- read first line (skipping header)

for line in iterator do
   local data_table = {}
   for t in line:gmatch('\"(.-)\"') do
      table.insert(data_table,t)
   end
   openProcess(tonumber(data_table[2]))
end
handle:close()


without reinitializeSymbolhandler in the loop, CE is still at 20MB usage.

If you really need reinitializeSymbolhandler , use it in loop only when you have filtered processlist:
- only opera, flash...., plugin...., etc )
- only bigger than 100MB (or what is "average" for most games)

_________________
Back to top
View user's profile Send private message MSN Messenger
DaSpamer
Grandmaster Cheater Supreme
Reputation: 52

Joined: 13 Sep 2011
Posts: 1578

PostPosted: Thu Nov 28, 2013 3:32 pm    Post subject: Reply with quote

when doing openProcess();, without using getAddress, C.E memory for me is around the 40-50MB usage.
But the moment I use getAddress('external.dll'), it jumps instantly, therefore I assume getAddress calls reinitializeSymbolhandler.
So I don't have any other method to verify .dll?

And yes I filter base flashplayerplugin,opera,chrome.
And then filter based .dll to reduce the processes that user may attach to.
Using this in C.E 6.3
Code:
local handle = io.popen( "tasklist.exe /fo csv");
local iterator = handle:lines()
iterator()

for line in iterator do
   local data_table = {}
   for t in line:gmatch('\"(.-)\"') do
      table.insert(data_table,t)
   end
   reinitializeSymbolhandler();
   openProcess(tonumber(data_table[2]))
end
handle:close()

Ended up with 940MB being used (after applying it for different 94 processes*).

While using the same script in C.E 6.2+
I ended up with 350MB being used (same amount of processes).

_________________
HEY Hitler
Do you get lazy when making trainers?
Well no more!
My CETrainer will generate it for you in seconds, so you won't get lazy! Very Happy

http://forum.cheatengine.org/viewtopic.php?t=564919
Back to top
View user's profile Send private message
mgr.inz.Player
I post too much
Reputation: 218

Joined: 07 Nov 2008
Posts: 4438
Location: W kraju nad Wisla. UTC+01:00

PostPosted: Thu Nov 28, 2013 3:57 pm    Post subject: Reply with quote

Anyway,
Order you are using:
1. reinitializeSymbolhandler - it will reinitialize symbol handler, it will load all symbols from previously opened process
2. openProcess - it will open process, CE will gather symbols


Order should be:
Code:
openProcess(tonumber(data_table[2]))
reinitializeSymbolhandler()


1. openProcess - it will open process, CE will gather symbols
2. reinitializeSymbolhandler - it will reinitialize symbol handler





EDIT:

Hmm, indeed, about 500MB after launching this script

Code:
errorOnLookupFailure(false)
local handle = io.popen( "tasklist.exe /fo csv");
local iterator = handle:lines()
iterator()

for line in iterator do
   local data_table = {}
   for t in line:gmatch('\"(.-)\"') do
      table.insert(data_table,t)
   end
   openProcess(tonumber(data_table[2]))
   getAddress('external.dll')
end
handle:close()


Don't know. Memory leak? Or, CE is just designed to be used only with one process.

_________________
Back to top
View user's profile Send private message MSN Messenger
Display posts from previous:   
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine Lua Scripting All times are GMT - 6 Hours
Goto page Previous  1, 2
Page 2 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group

CE Wiki   IRC (#CEF)   Twitter
Third party websites