Cheat Engine Forum Index Cheat Engine
The Official Site of Cheat Engine
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 


How to avoid large memory consumption in the trainer?

 
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine Lua Scripting
View previous topic :: View next topic  
Author Message
Razi
Expert Cheater
Reputation: 1

Joined: 17 Jan 2018
Posts: 202

PostPosted: Tue Mar 20, 2018 4:47 am    Post subject: How to avoid large memory consumption in the trainer? Reply with quote

Form created with 135 editboxes. Code reads values from 135 addresses and writes them in the 135 editboxes. But after 1 hour of execution, trainer consumes 1.2 Gigabytes of memory and loads CPU up to 50 percents. How to avoid large memory consumption in the trainer? The following code is used to read values from addresses in the trainer:
Code:
timer = createTimer(true)
timer:setInterval(1000)
timer.OnTimer = function()
--
local addr = 0x009E8624
  for x = 1, 135 do
   if x < 16 then
    UDF1["CEEdit"..x].Text = readBytes(addr+x)
    elseif x < 15*2+1 then
    local addr = addr + 0x84
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15))
    elseif x < 15*3+1 then
    local addr = addr + 0x84*2
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*2))
    elseif x < 15*4+1 then
    local addr = addr + 0x84*3
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*3))
    elseif x < 15*5+1 then
    local addr = addr + 0x84*4
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*4))
    elseif x < 15*6+1 then
    local addr = addr + 0x84*5
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*5))
    elseif x < 15*7+1 then
    local addr = addr + 0x84*6
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*6))
    elseif x < 15*8+1 then
    local addr = addr + 0x84*7
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*7))
    else
    local addr = addr + 0x84*8
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*8))
  end
end
end  -- end of timer function
Back to top
View user's profile Send private message
Corroder
Grandmaster Cheater Supreme
Reputation: 75

Joined: 10 Apr 2015
Posts: 1667

PostPosted: Tue Mar 20, 2018 6:06 am    Post subject: Reply with quote

seem timer not stop or destroy after reach several conditions.
to get know where is memory leak :

Code:
collectgarbage("collect")


force a garbage collection at the end of every OnTimer cycle.

_________________
Stealing Code From Stolen Code...
And Admit It.. Hmmm....Typically LOL


Last edited by Corroder on Tue Mar 20, 2018 6:16 am; edited 1 time in total
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 457

Joined: 09 May 2003
Posts: 25262
Location: The netherlands

PostPosted: Tue Mar 20, 2018 6:07 am    Post subject: Reply with quote

assuming you're not creating new timers over and over, in your timer add
Code:

collectgarbage()

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
Razi
Expert Cheater
Reputation: 1

Joined: 17 Jan 2018
Posts: 202

PostPosted: Tue Mar 20, 2018 7:23 am    Post subject: Reply with quote

Can you tell me if the code in the trainer is written correctly? Or how to avoid memory leak without collectgarbage() code?
Code:
--getAutoAttachList().add("ePSXe ENG.exe")
timer = createTimer(true)
timer:setInterval(1000)
timer.OnTimer = function()
--
local addr = 0x009E8624
  for x = 1, 135 do
   if x < 16 then
    UDF1["CEEdit"..x].Text = readBytes(addr+x)
    elseif x < 15*2+1 then
    local addr = addr + 0x84
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15))
    elseif x < 15*3+1 then
    local addr = addr + 0x84*2
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*2))
    elseif x < 15*4+1 then
    local addr = addr + 0x84*3
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*3))
    elseif x < 15*5+1 then
    local addr = addr + 0x84*4
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*4))
    elseif x < 15*6+1 then
    local addr = addr + 0x84*5
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*5))
    elseif x < 15*7+1 then
    local addr = addr + 0x84*6
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*6))
    elseif x < 15*8+1 then
    local addr = addr + 0x84*7
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*7))
    else --if x < 15*9+1 then
    local addr = addr + 0x84*8
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*8))
  end
end



for x = 1, 135 do
UDF1["CEEdit"..x].OnKeyPress = function(sender, key)
timer.Enabled = false
    local a = tonumber(sender.Text)
    if a ~= nil then
      if isKeyPressed(VK_RETURN) == true then
      if x < 16 then
        writeBytes(addr+x,UDF1["CEEdit"..x].Text)
      elseif x < 15*2+1 then
        local addr = addr + 0x84
        writeBytes(addr+(x-15),UDF1["CEEdit"..x].Text)
      elseif x < 15*3+1 then
        local addr = addr + 0x84*2
        writeBytes(addr+(x-15*2),UDF1["CEEdit"..x].Text)
      elseif x < 15*4+1 then
        local addr = addr + 0x84*3
        writeBytes(addr+(x-15*3),UDF1["CEEdit"..x].Text)
      elseif x < 15*5+1 then
        local addr = addr + 0x84*4
        writeBytes(addr+(x-15*4),UDF1["CEEdit"..x].Text)
      elseif x < 15*6+1 then
        local addr = addr + 0x84*5
        writeBytes(addr+(x-15*5),UDF1["CEEdit"..x].Text)
      elseif x < 15*7+1 then
        local addr = addr + 0x84*6
        writeBytes(addr+(x-15*6),UDF1["CEEdit"..x].Text)
      elseif x < 15*8+1 then
        local addr = addr + 0x84*7
        writeBytes(addr+(x-15*7),UDF1["CEEdit"..x].Text)
      else
        local addr = addr + 0x84*8
        writeBytes(addr+(x-15*8),UDF1["CEEdit"..x].Text)
      end
      timer.Enabled = true
    end
    return key
    end
end
end


end --timer function end

--[[
function startCollectgarbageProcess(interval, showStatus)
  local timer1 = createTimer(true)
  timer1.Interval = interval
  timer1.onTimer = function ()
    if (showStatus) then
      --print(string.format('Lua memory usage %.1f MiB',collectgarbage('count')/1024))
    end
    collectgarbage("restart")
    collectgarbage("collect")
    if (showStatus) then
      --print(string.format('Lua memory usage %.1f MiB',collectgarbage('count')/1024))
    end
  end
end

startCollectgarbageProcess(60000, true)
]]


Dark Byte wrote:
assuming you're not creating new timers over and over, in your timer
I'm not sure about this, I need confirmation (or hint) from experienced users.
Corroder wrote:
seem timer not stop or destroy after reach several conditions.
And how to fix this without "collectgarbage" code?
Back to top
View user's profile Send private message
Corroder
Grandmaster Cheater Supreme
Reputation: 75

Joined: 10 Apr 2015
Posts: 1667

PostPosted: Tue Mar 20, 2018 7:39 am    Post subject: Reply with quote

Code:
local addr = 0x009E8624
  for x = 1, 135 do
   if x < 16 then
    UDF1["CEEdit"..x].Text = readBytes(addr+x)
    elseif x < 15*2+1 then
    local addr = addr + 0x84
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15))
    elseif x < 15*3+1 then
    local addr = addr + 0x84*2
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*2))
    elseif x < 15*4+1 then
    local addr = addr + 0x84*3
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*3))
    elseif x < 15*5+1 then
    local addr = addr + 0x84*4
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*4))
    elseif x < 15*6+1 then
    local addr = addr + 0x84*5
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*5))
    elseif x < 15*7+1 then
    local addr = addr + 0x84*6
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*6))
    elseif x < 15*8+1 then
    local addr = addr + 0x84*7
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*7))
    else --if x < 15*9+1 then
    local addr = addr + 0x84*8
    UDF1["CEEdit"..x].Text = readBytes(addr+(x-15*8))
  end


Is this code work without timer ?. Did you try ?. Because with 'for...do ' 1 to 135 it's already get value from the address with several conditions.

_________________
Stealing Code From Stolen Code...
And Admit It.. Hmmm....Typically LOL
Back to top
View user's profile Send private message
Razi
Expert Cheater
Reputation: 1

Joined: 17 Jan 2018
Posts: 202

PostPosted: Tue Mar 20, 2018 9:23 am    Post subject: Reply with quote

Corroder wrote:
Is this code work without timer ?. Did you try ?. Because with 'for...do ' 1 to 135 it's already get value from the address with several conditions.

This code used to work with timer. Every second this code reads values from addresses. Tried few minuts ago to execute code without timer, and code reads values one time in to corresponding CEEdit. So, code works without timer.
How do I fix the code, to do not get value from the address with several conditions?
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 457

Joined: 09 May 2003
Posts: 25262
Location: The netherlands

PostPosted: Tue Mar 20, 2018 12:03 pm    Post subject: Reply with quote

you need to use collectgarbage() in your timer.

Each time your timer runs it will allocate some temporary objects for referencing the edit boxes, and when it finishes it will clean up some of them (64 or so). But I think you're beyond the limit that it will autoclean for you, so you will need to clean up the extra objects manually (there is nothing wrong with "collectgarbage()" it's a very valid thing to do in scripts that eat more than the average amount of memory )

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
Razi
Expert Cheater
Reputation: 1

Joined: 17 Jan 2018
Posts: 202

PostPosted: Tue Mar 20, 2018 2:25 pm    Post subject: Reply with quote

Dark Byte wrote:
you need to use collectgarbage() in your timer.

Each time your timer runs it will allocate some temporary objects for referencing the edit boxes, and when it finishes it will clean up some of them (64 or so).

Thank you for answer. I want to create trainer with 550 edit boxes and 150 comboboxes at least (total 700-800 control elements on the 3 tabs). Currently done only for 300 controls. Is it possible in the CE, to make a trainer with 700-900 control elements?
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 457

Joined: 09 May 2003
Posts: 25262
Location: The netherlands

PostPosted: Tue Mar 20, 2018 3:39 pm    Post subject: Reply with quote

no problem

just occasionally call the garbage collector (e.g when not much happens using a timer or something)

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
Corroder
Grandmaster Cheater Supreme
Reputation: 75

Joined: 10 Apr 2015
Posts: 1667

PostPosted: Tue Mar 20, 2018 6:50 pm    Post subject: Reply with quote

By the way, here example use to compare when use 'collectgarbage()' and not use.

Code:
-- First, test memory without use collectgarbage()
local function printMemUsage()
    local memUsed = (collectgarbage("count")) / 1000
    print("\n---------MEMORY USAGE INFORMATION---------")
    print("System Memory Used:", string.format("%.03f", memUsed), "Mb")
    print("------------------------------------------\n")
    return true
end

timer = createTimer(true)
timer.Interval = 1000
timer.OnTimer = function()
for i=1,135 do
 print("Hello CEF "..i)
 if i == 135 then
-- collectgarbage()
 printMemUsage() end
end
end

printMemUsage()

-- result :
---------MEMORY USAGE INFORMATION---------
System Memory Used: 5492 Mb              -- next increase to : 6122 Mb / 1st cycle...  6457 Mb...and so on
------------------------------------------------------



Code:
-- Second, test memory use collectgarbage()
local function printMemUsage()
    local memUsed = (collectgarbage("count")) / 1000
    print("\n---------MEMORY USAGE INFORMATION---------")
    print("System Memory Used:", string.format("%.03f", memUsed), "Mb")
    print("------------------------------------------\n")
    return true
end

timer = createTimer(true)
timer.Interval = 1000
timer.OnTimer = function()
for i=1,135 do
 print("Hello CEF "..i)
 if i == 135 then
collectgarbage()
 printMemUsage() end
end
end

printMemUsage()

-- result :
---------MEMORY USAGE INFORMATION---------
System Memory Used: 5492 Mb              -- next decrease to : 1728 Mb / 1st cycle...  1715 Mb...and  stay at 1715 - 1790 Mb
------------------------------------------------------


So,it's nothing wrong with use collectgarbage() to manage RAM....

_________________
Stealing Code From Stolen Code...
And Admit It.. Hmmm....Typically LOL
Back to top
View user's profile Send private message
mgr.inz.Player
I post too much
Reputation: 218

Joined: 07 Nov 2008
Posts: 4438
Location: W kraju nad Wisla. UTC+01:00

PostPosted: Wed Mar 21, 2018 9:25 am    Post subject: This post has 1 review(s) Reply with quote

@Razi, your code from Tue Mar 20, 2018


You're changing (re-assigning) OnKeyPress function over and over, inside OnTimer function, for every CEEdit object. Do it once.

You're using those if then elseif then elseif then elseif...
and you're using the same variable name (addr) in different "scopes" (can lead to mistakes in future).

Use math:
Code:
x = <1,15>  , for reading we use "addr + 0x84*0 + (x - 15*0)"
x = <16,30> , for reading we use "addr + 0x84*1 + (x - 15*1)"
x = <31,45> , for reading we use "addr + 0x84*2 + (x - 15*2)"

and so on...

so, formula for address is:
Code:
offset = 0x84*((x-1) // 15) + (((x-1) % 15) + 1)
reading (addr + offset)



Also you can use Lua tables for caching...



EDIT:
your code optimized:
Code:
--getAutoAttachList().add("ePSXe ENG.exe")

addr = 0x009E8624
timer = createTimer(nil,false)
timer.Interval = 1000



-- prepare cache
cacheTable_CEEdits = {}
for x = 1, 135 do
  cacheTable_CEEdits[x] = UDF1["CEEdit"..x]
end



-- set OnKeyPress once for each CEEdit
for x = 1, 135 do
  local offset = 0x84*((x-1) // 15) + (((x-1) % 15) + 1)
  local currentCEEdit = cacheTable_CEEdits[x]

  cacheTable_CEEdits[x].OnKeyPress =
    function(sender, key)
      timer.Enabled = false
      local a = tonumber(sender.Text)
      if (a~=nil) and isKeyPressed(VK_RETURN) then
        writeBytes(addr + offset, currentCEEdit.Text)
        timer.Enabled = true
      end
      return key
    end

end



-- OnTimer
timer.OnTimer = function()
  for x = 1, 135 do
    local offset = 0x84*((x-1) // 15) + (((x-1) % 15) + 1)
    cacheTable_CEEdits[x].Text = readBytes(addr+offset)
  end
end



-- enable timer
timer.Enabled = true


Of course you can add collectgarbage("collect") inside ontimer function.


Code:
-- OnTimer
counterUsedForGC = 0
timer.OnTimer = function()
  for x = 1, 135 do
    local offset = 0x84*((x-1) // 15) + (((x-1) % 15) + 1)
    cacheTable_CEEdits[x].Text = readBytes(addr+offset)
  end

  counterUsedForGC = counterUsedForGC + 1
  if counterUsedForGC>59 then collectgarbage("collect"); counterUsedForGC = 0 end  -- every 60 ticks
end

_________________
Back to top
View user's profile Send private message MSN Messenger
Razi
Expert Cheater
Reputation: 1

Joined: 17 Jan 2018
Posts: 202

PostPosted: Fri Mar 23, 2018 7:31 am    Post subject: Reply with quote

mgr.inz.Player wrote:

your code optimized:

Of course you can add collectgarbage("collect") inside ontimer function

I checked optimized code. After 1 hour of execution, the code consumes 300-400 megabytes of memory and loads CPU up to 0-10 percent. (And that's without collectgarbage("collect") code) . Your code is 10 times better, than the one I used.

It would be nice if CE had a Table Control (screenshots below). And instead of many edit boxes, use one Table Control, in the Trainer. But, I do not know, if users will often use such control element. And Edit boxes looks more beautiful than Table Control.
Back to top
View user's profile Send private message
ByTransient
Expert Cheater
Reputation: 5

Joined: 05 Sep 2020
Posts: 240

PostPosted: Thu May 06, 2021 6:54 am    Post subject: Reply with quote

I have to repeat the subject.
Although there is "collectgarbage ()", memory continues to increase.
In the code in the picture, the memory started with 50 and continues with 60+.
In the Trainer I created, memory goes above 150 and 220+ continues to increase even though I put "collectgarbage ()" into a Timer.
There is 1 Timer running and having an interval of 4000.
Why does it keep rising?
Back to top
View user's profile Send private message
atom0s
Moderator
Reputation: 198

Joined: 25 Jan 2006
Posts: 8516
Location: 127.0.0.1

PostPosted: Thu May 06, 2021 1:01 pm    Post subject: This post has 1 review(s) Reply with quote

Lua's garbage collection is only accountable for things it is allocating itself. Any extensions added onto the language (ie. all of CE's API being exposed via Lua) are not the responsibility of Lua to manage/maintain the memory of. Calling 'collectcarbage()' is not a guarantee that objects/memory are being cleaned up that you created outside of Lua's own scope of functions. If your memory is constantly growing even after calling 'collectgarbage', then you are probably leaking memory somewhere or there is a memory leak in CE in general with one of the free/destroy/cleanup calls not working properly/as expected.

It'd be easier to tell what may be the cause of your memory increase if you can show a full example of code rather than just your function showing memory info. (Keep in mind, Lua's memory reporter is also not going to tell you anything about allocations done by CE itself when creating CE's own objects/data. At most, Lua will only see those as a single pointer (4 or 8 bytes) and nothing more regardless if say, the object takes 100mb in memory.)

_________________
- Retired.
Back to top
View user's profile Send private message Visit poster's website
ByTransient
Expert Cheater
Reputation: 5

Joined: 05 Sep 2020
Posts: 240

PostPosted: Thu May 06, 2021 2:17 pm    Post subject: Reply with quote

In other words, the add-ons we added in CE may also cause this.
This makes sense.
I guess the memory rising in me will not be the same in other users.
However, I added a "collectgarbage ()" to the timer I used in Trainer.
Thanks admin for the detailed answer.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine Lua Scripting All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group

CE Wiki   IRC (#CEF)   Twitter
Third party websites