Cheat Engine Forum Index Cheat Engine
The Official Site of Cheat Engine
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 


Export values to a database

 
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine Lua Scripting
View previous topic :: View next topic  
Author Message
jonaskp
Newbie cheater
Reputation: 0

Joined: 29 May 2020
Posts: 14

PostPosted: Mon Apr 05, 2021 2:24 am    Post subject: Export values to a database Reply with quote

Hi,

I have made a Cheat table where I record values, that I then need to use in an Excel sheet, to Update values there. The purpurs is that I run a tournament, so I need the resultat ind the Excel.

Currently I have it working with a LUA script that copies the entitet table to clipboard every Half second. Then in Excel I run a script where I paste the clipboard (with the values from the cheat table) and then I use it from there.

This is not the best Way, as it Blocks the clipboard from any other use, and it also has some other issues, as the copy/paste sometimes switches focus.


Ideally I would like to have cheat engine put the values into a database, that Excel then Can pull data from. I run a local mariaDB database (SQL), so if that Can be used, it would be ideal.

I also thought about exporting cheat table to a .txt file (I have seen examples of this in the Forum), but as I need to write to the file every 0,5-1 second, and read it at the same rate, it doesnt Seem feasible.


Is the database connection something that Can be Done in LUA? Or is there some other approach that could work, that I have not thought of?
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 458

Joined: 09 May 2003
Posts: 25287
Location: The netherlands

PostPosted: Mon Apr 05, 2021 8:13 am    Post subject: Reply with quote

ce lua can connect to sqlite3 database and to databases configired with an odbc connection

check out celua.txt for more info on this.

i can't currently given an example as i can't use a pc right now, but ask specific questions on hiw to use the database functiond and i'll try to help out.

Do you understand sql queries ? If not, perhaps lua tables might be easier to work with and a custom save/load functions

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
jonaskp
Newbie cheater
Reputation: 0

Joined: 29 May 2020
Posts: 14

PostPosted: Mon Apr 05, 2021 1:14 pm    Post subject: Reply with quote

Thanks for the answer.

I have been using SQL queries for a while now, so I have an ok understanding of it.
I am quite good with VBA, and have dabbled with HTML, php, javascript.

LUA is completely new to me, and the only thing I have done with it, is the script I am using now, which is completely copy-paste from this forum, with a few modifications that I asked for help for.


The database I currently have running is a MariaDB (which is similar to MySQL as far as I know), and I am using SQL Queries to interface it with Excel.
In my VBA code in Excel, I am using "driver={MySQL ODBC 8.0 UNICODE Driver}" in the connection string, so I guess it should be possible using ODBC from CE LUA as well.
I am sure I can figure out that part, from the database side.


The actual problem:

I have a Cheat Table with around 900 "rows"/values in it.
For each of these "rows", I need to export the Description-field and the value-field.

My idea is that the database will have 2 fields/columns: "description" (primary key) and "value".
The LUA script will then upload the values from the Cheat Table, to the database, so that the database will contain these 900 rows.
This will repeat every 0,5 - 1 second, overwriting the existing values (I just need the most current value in the database, not a historic record)


As I said, I can write the SQL queries and I can code stuff in VBA, but I don't know LUA, so this is the part I need help with.


I don't know if the above is specific enough or if it is even possible to give an example of it (or some guidance on how to research it myself, without necessarily having to completely learn LUA) - without spending enormous amount of time on it.

Any help would be appriciated Smile
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 458

Joined: 09 May 2003
Posts: 25287
Location: The netherlands

PostPosted: Mon Apr 05, 2021 2:36 pm    Post subject: Reply with quote

the addresslist can be read out using the AddressList object

e.g
Code:

for i=0,AddressList.Count -1 do
print(string.format("%s = %s", AddressList[i].Description,  AddressList [i].Value))
end


check out the AddressList in celua.txt as well


tip:
play around in lua engine (ctrl+alt+shift+ L)

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
jonaskp
Newbie cheater
Reputation: 0

Joined: 29 May 2020
Posts: 14

PostPosted: Tue Apr 06, 2021 3:57 am    Post subject: Reply with quote

That LUA engine tip was very helpfull, thanks.

The addreslist is something I already used in the current script, just a bit different I think.


After playing around a bit, I now have a script that can create the SQL Query that I need. The query works fine when I manually run it on the database

This is what I have so far:

Code:
 local al=getAddressList()
  local strSQLCommand=""
  local strDescription=""
  local strValue=""

  --Sets beginning of SQL query
  strSQLCommand="INSERT INTO `cheat_engine_output` (`description`, `value`, `order`) VALUES "

  --Loop Cheat table, and add each set of description/value to the SQL Query
  for i=0, al.Count-1 do

      --get descritption and value
      strDescription=al[i].Description
      strValue=al[i].Value

      -- Replace ' in value field, as it messes with SQL
      strValue = string.gsub(strValue, "'", "´")

      --create single SQL query
      -- the "i" at the end, is to also update the column "order", which makes it so I can keep the table in same order as cheat engine
      strSQLCommand= strSQLCommand.."\r\n('"..strDescription.."', '"..strValue.."', '"..i.."') "

      --Adds comma to the end of each sets of value, as long as the current set is not the last one (no comma should be at the end)
      if i<al.Count-1 then
         strSQLCommand = strSQLCommand .. ","
      end

  end

  --Sets end of SQL query
  strSQLCommand=strSQLCommand.."ON DUPLICATE KEY UPDATE `description`=VALUES(`description`), `value`=VALUES(`value`), `order`=VALUES(`order`)"

  print(strSQLCommand)
  writeToClipboard(strSQLCommand)



The SQL Query it creates looks like this (the real one is 860 rows, but I cut it down):
Code:
INSERT INTO `cheat_engine_output` (`description`, `value`, `order`) VALUES
('--------HOME TEAM----------', 'Argentina', '2') ,
('HomeTeamPasses', '8', '3') ,
('HomeTeamPassesComplete1', '8', '4') ,
('HomeTeamPassesComplete2', '8', '5') ,
('HomeTeamShots', '0', '6') ,
('HomeTeamShotsOnGoal', '0', '7') ,
('homePenaltyShootoutGoals', '0', '8') ,
('homePenaltyShootoutShots', '0', '9') ,
('homeTeamGoals', '0', '10') ON DUPLICATE KEY UPDATE `description`=VALUES(`description`), `value`=VALUES(`value`), `order`=VALUES(`order`)


The next bit - and the one I think I will have a hard time figuring out - is making the connection from LUA to the database. I tried looking in the celua.txt, but that didn't really help me a lot.
I have also tried searching, but all I seem to find, is people trying to connect to a games internal SQLite3 database to extract and change values.

Do you know of a guide or example of connecting via ODBC and passing a query to the database?


Thanks a lot for the help so far, it has steered me in the right direction (I think).
When I get this working, I think it will be a great improvement over the current CopyPaste solution.
Back to top
View user's profile Send private message
jonaskp
Newbie cheater
Reputation: 0

Joined: 29 May 2020
Posts: 14

PostPosted: Tue Apr 20, 2021 6:25 am    Post subject: Reply with quote

So, other stuff happened, but I am finally able to get back to this.

I have looked around (on the forum and in celua.txt), but am completely stuck.
I have found mention of "CreateODBCConnection(owner)", but have no idea how to use it in code.


My status so far:

What I have:
- An SQL Query (see post above)
- Connection the my SQL Database setup in ODBC Data sources (windows)

What I need:
- An example of how to make a connection to a mySQL database (ie. using ODBC) where I can then run the above SQL Query.


I hope someone is able to help with this. I don't necessarily need a finished code example (though i would be nice Smile), but all mentions I have found so far is just somethin along the line of "use createODBCConnection(owner) and set it up correctly" - and I don't know how to use it or set it up...
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 458

Joined: 09 May 2003
Posts: 25287
Location: The netherlands

PostPosted: Tue Apr 20, 2021 8:07 am    Post subject: Reply with quote

ai never tried the odbc connection myself and only exposed whatbis available in lazarus so not sure

right now I still can not get behind a computer to test (perhaps next week)


but how about working with an sqlite3 database?
With lua you can creat3 an sqlite3 database and also do sql querries to fill it.
Once you get that working it's a lot easier to switch out the connection component

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
jonaskp
Newbie cheater
Reputation: 0

Joined: 29 May 2020
Posts: 14

PostPosted: Tue Apr 20, 2021 9:22 am    Post subject: Reply with quote

Dark Byte wrote:

but how about working with an sqlite3 database?
With lua you can creat3 an sqlite3 database and also do sql querries to fill it.
Once you get that working it's a lot easier to switch out the connection component

From what I Can read, sqlite3 is not ideal for simultaneously read and write, which is what I need.
I Will have 2-3 computers writing to it, and 1 computer Reading from it, almost constantly.
So I don’t Think it is viable as a permanent solution. Unless I have misunderstood something about sqlite3?

Unless your suggestion was to only use sqlite3 as a temporary solution, to get practice with LUA and database connection? In that case it might be an idea.


It would be fantastic if you could look at Odbc connection once you have time and access to computer. There is no Big rush right now, so I Can wait Smile.
I will trymessing around some more with it myself as Well.
Back to top
View user's profile Send private message
jonaskp
Newbie cheater
Reputation: 0

Joined: 29 May 2020
Posts: 14

PostPosted: Fri Jun 04, 2021 1:36 pm    Post subject: Reply with quote

So, I am getting closer to when I need this working, and still I have not gotten any further. Looking into this, I have come up completely empty:

- I tried looking at the "celua.txt", but I don't really think that explains how "createODBCconnection" should be implemented.

- I tried searching the forums, but there is barely any mentioning of "createODBCconnection" and no working solutions.

- I looked in the Wiki, but all the pages regarding database connection is empty.


Dark Byte, if you have had time to look into this, or have more suggestions, I would be grateful.
The database-stuff is implemented in CheatEngine, so it must be possible to get it working.

I just lack the knowledge on how to do it, without more guidance than what is currently in the "celua.txt" file.

[EDIT]
Apparently I can't double post, so I'll add this to this post instead:

To recap what I need:

- I have gotten the script to write a SQLQuery to the variable "strSQLCommand"

- What I need help with, is getting Cheat Engine to run this SQLQuery on my database.
Preferably through an ODBC-connection, but for now I can live with hardcoding credentials directly into the LUA script, as this will only run locally under my supervision (and is not "mission critical" for anything important).


Any help would be greatly appriciated.
Back to top
View user's profile Send private message
ParkourPenguin
I post too much
Reputation: 140

Joined: 06 Jul 2014
Posts: 4289

PostPosted: Fri Jun 04, 2021 2:30 pm    Post subject: Reply with quote

There's a lot of stuff in CE that you'd only find by reading the source.
https://github.com/cheat-engine/cheat-engine/blob/master/Cheat%20Engine/LuaSQL.pas

_________________
I don't know where I'm going, but I'll figure it out when I get there.
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 458

Joined: 09 May 2003
Posts: 25287
Location: The netherlands

PostPosted: Sun Jun 06, 2021 5:13 am    Post subject: Reply with quote

Oops, looks like I forgot to document some parameters

lets say you have an ODBC datasource named "mariadbtest"
then you can create an ODBC connection using:
Code:

c=createODBCConnection(nil)
c.DatabaseName='mariadbtest'
c.Connected=true
if c.Connected then print("success") end

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
jonaskp
Newbie cheater
Reputation: 0

Joined: 29 May 2020
Posts: 14

PostPosted: Mon Jun 07, 2021 10:34 am    Post subject: Reply with quote

[EDIT]I am having some issues. See bottom of post...
You guys are AWESOME!!

Finally managed to get it all working today.

@ParkourPenguin: Thanks for guiding me to the source code.
I didn't understand what you meant at first. It wasn't until after Dark Byte posted his replied, that I somehow realised, that you were referring to the example at the very beginning of the Source Code.
I had been searching for "ODBC" and looking there... Rolling Eyes


I managed to reverse-engineer the Code example down to this:


Code:
c=createODBCConnection(nil)
c . DatabaseName='ODBC_cheat_engine'
c . Connected=true
if c . Connected then print("DB Connection: Succes") end

--strSQLCommand = "INSERT INTO `fifavm`.`cheat_engine_output` (`pc_description`, `value`) VALUES ('PC1_test description', 'TEST_value')"

t=createSQLTransaction()
t . SQLConnection=c
t . Active=true

    c . ExecuteDirect(string.format(strSQLCommand))

t . Commit()
t . Active=false
c . Connected=false
t . destroy()
c . destroy()

Code looks a bit weird, because forum wouldn't let me post it. Said I can't post URLs.
So I left spaces around the "." and it seems to have allowed me to post.

My initial code is the same as in the first post, where the code gathers values from the addresslist and turns them into an SQL Query saves it in "strSQLCommand" (I left my test Query in the aboved code - commented out - for reference).

I still need to do a bit more testing, write some comments for myself for later, but it looks like the actual database connection works beautifully.

Thanks for all your help.



[EDIT]
So, I spoke a bit too soon.
I have just noticed that I have issues with character encoding.

I am using the character set "utf8mb4", as I have some special characters.

If I take the SQL Query and manually run it on my database, there is no issues - special characters are saved correctly.
Example: "L. Martínez Quarta"

However, when I run the Query through Cheat Engine, it doesn't format correctly.
Example: "L. MartÃ-­nez Quarta"


I am expecting it to be an issue either with Cheat Engine (where I am missing some code to correct this) or with the ODBC connection.

I am using "MySQL ODBC 8.0 Unicode Driver". I have tried setting the Character set to "utf8mb4" in the ODBC data source setting, but that didn't help.

I tried switching to using "MySQL ODBC 8.0 ANSI Driver".
That fixed most of the Special Characther issues (it seems), but instead I am now getting an error "Incorrect string value: '\x99'" - and nothing is saved to the database.
Back to top
View user's profile Send private message
jonaskp
Newbie cheater
Reputation: 0

Joined: 29 May 2020
Posts: 14

PostPosted: Tue Jun 08, 2021 5:07 am    Post subject: Reply with quote

I have done more testing, but haven't found a solution yet.

I think the issue lies somewhere with my code in LUA.
If I use the same ODBC connection to pull data into Excel, the characters show up fine (if they were correct in the Database). So atleast when using "SELECT", the ODBC driver is not doing some weird conversion.

If I copy the SQL Query string made by Cheat Engine, and run it manually on the Database (through SQLyog), the characters show up fine.

But when I run it through Cheat Engine and LUA (using ODBC), the special characters are getting messed up.


Am I missing a parameter somewhere in my LUA script, that forces the Query to use utf8mb4?
Back to top
View user's profile Send private message
Dark Byte
Site Admin
Reputation: 458

Joined: 09 May 2003
Posts: 25287
Location: The netherlands

PostPosted: Tue Jun 08, 2021 6:14 am    Post subject: Reply with quote

the functions ansiToUtf8 or utf8ToAnsi might help somewhat?

CE gui components are in UTF8, so first convert the UTF8 to ansi, and then use the ansi odbc connection


Also, where does the formatting go wrong? In during insert, during select , or when rendering to a GUI control in CE?

_________________
Do not ask me about online cheats. I don't know any and wont help finding them.

Like my help? Join me on Patreon so i can keep helping
Back to top
View user's profile Send private message MSN Messenger
jonaskp
Newbie cheater
Reputation: 0

Joined: 29 May 2020
Posts: 14

PostPosted: Tue Jun 08, 2021 6:48 am    Post subject: Reply with quote

Dark Byte wrote:
Also, where does the formatting go wrong? In during insert, during select , or when rendering to a GUI control in CE?

I Think it happens during INSERT INTO (from the SQL Query) when run from CE.

I am using SQLyog as a GUI for the database, and it is when I look at the table in SQLyog, that the wrong formatting shows up.


Currently the LUA code creates a SQL QUERY (saved to a variable strSQLCommand) and then runs that Query.
I also have it set to copy the SQL Query to the clipboard, so I Can paste it into SQLyog and run it manually.
When I run it manually, the formatting is correct.

So CE creates the Query correctly, with the right formatting - but then at some point during running the Query in CE or the ODBC connection, the formatting gers messed up.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine Lua Scripting All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group

CE Wiki   IRC (#CEF)   Twitter
Third party websites