Cheat Engine

[Corroder]

Hi everyone,

I am currently developing a Proof of Concept (PoC) for an integrated Screen Recorder inside a Cheat Table utilizing a combination of CE Lua GUI, Auto Assemble (AA) scripts, and a custom C++ DLL.

The Goal:
The user clicks a "Start Recording" button on a Lua Form, which triggers a custom C++ DLL via executeCodeLocal or executeCode. The C++ DLL uses Windows Media Foundation (IMFSinkWriter) to record the screen frame-by-frame and outputs an MP4 file to the %TEMP% directory. To prevent STATUS_ACCESS_VIOLATION caused by DEP/UAC when running background loops/threads inside the DLL, we redesigned the architecture into a thread-less, single-frame execution pipeline. The 40ms capture loop is driven entirely by a Cheat Engine Lua timer (createTimer), which repeatedly fires the frame capture function inside the DLL.

The exported C++ functions are compiled using MinGW/w64devkit with the __cdecl calling convention to match Cheat Engine's execution profile.

The Problem:
Despite the thread-less architecture and running Cheat Engine as Administrator, the operating system strictly blocks the custom DLL from being mapped or initialized. I have tried multiple approaches to load the binary, but every single one of them fails:

1. loadlibrary() / Local AA Injection fails: Attempting to force the Cheat Engine process to call LoadLibraryA via a temporary AA thread results in an immediate failure or 0 address return

2. loadModule() / injectDLL() locally fails: Cheat Engine refuses to map the absolute path of the custom DLL into its own local memory space.

3. loadPlugin() fails: Trying to register the DLL as an official CE plugin via loadPlugin("D:\\recorder64.dll") returns a failure, even on an elevated (Run as Administrator) CE session.injectDLL() into Target Context fails: Shifting the entire architecture to target-process execution (injecting the DLL straight into the game memory space to bypass CE's hardened space) also fails. Even on simple Win32 desktop targets like notepad.exe (Win11 modern app container) or basic games, the module gets rejected by the Windows kernel security layer immediately after injection.

It seems that modern Windows Security mitigations (Exploit Protection, CFG, AppContainer Sandboxing, or strict DLL Side-Loading blockades) are intercepting these memory mapping attempts.

Has anyone successfully integrated a custom C++ DLL using a frame-by-frame callback method from Lua without getting blocked by OS-level protections? Are there specific compiler flags or hidden CE API functions that I should use to legally register a custom binary inside the CE ecosystem?

Any code snippets, architectural advice, or pointers would be highly appreciated.

Thank you!

CPP CODE

[Dark Byte]

I don't know about your dll injection issues, it injects just fine (try on the tutorial)

(make sure that if the target process is 64-bit that you compile it for 64-bit)

try adding reinitializeSymbolhandler() after injectdll

if you need to inject in a process that doesn't have read access then use auto assembler {c} blocks instead

also executeCode("ce_scrStart", pW, pH) is a wrong call , that's not how that function works

[Corroder]

Hi DarkByte,

Thank you for your valuable insight and recommendations regarding the reinitializeSymbolhandler() trick and the usage of {c} compiler blocks to handle local process context execution securely

.Following your suggestion, I refactored the trainer to run entirely standalone without external assembly overhead or dynamic link dependencies, relying purely on the internal TCC compilation block. I also made sure that the execution memory allocated global structures beforehand to prevent standard script lifecycle timing issues.

However, upon executing the function via executeCodeLocal, it consistently throws an unhandled Error: Access violation exception right on the Lua Engine console layout. There are no detailed traceback pointers captured by Lua because the process threads encounter a hard exception immediately at the address jump.

Based on our local environment debugging on older hardware architectures, it appears the internal TCC compiler might be failing to construct a structurally safe stack initialization frame/epilogue when handling this specific type of localized custom C parameter syntax (char* outputPath). The CPU hits an illegal memory access violation before the code inside the block can properly initialize or return

.For your reference and further analytical oversight to help improve the TCC feature set in upcoming Cheat Engine updates, here is the exact, complete Cheat Table Lua script that triggers this behavior. I hope this helps you spot any underlying compiler bugs or alignment mismatches that we might have missed!

The Cheat Engine Table Lua Script Used:

[Dark Byte]

You don't check the return value of initCBlockRecorder so because it fails to assemble that code you crash because you execute nil

also it's {$c}/{$asm} and there is no __try

and including windows.h is generally not the recommended way to use tcc as that ends up compiling way more than needed,reply in morsecode

[Corroder]

Hi DB,

Thank you for pointing that out! You are absolutely right. Checking the return value of autoAssemble prevented the nil execution crash entirely.

However, even with the updated {$c} / {$asm} block tags, the auto-assembler still consistently returns false on our environment. It appears that Cheat Engine's internal TCC compiler lacks the implicit linkage definitions required for external Windows OS APIs.

For instance, when I declare int __stdcall GetSystemMetrics(int nIndex);, the compilation fails during the linking stage because it cannot resolve the import symbols natively without an explicit link library directive (like user32.lib).

Since I want to keep the tool fully functional, lightweight, and independent of external compiler setups, I took an alternative approach by embedding and piping an asynchronous ffmpeg.exe instance directly through Table Files, which works flawlessly.

For reference and in case anyone else tries to achieve the same standalone result, here is the verified script that works beautifully without blocking the target process read space:

[Dark Byte]

static linking is not supported no. That's why you use loadlibrary and getprocaddress as those are present in all running processes, or use ce's injectdll beforehand

Anyhow, please reply in klingon and glad you fixed it by using an external tool. which is what my 2nd suggestion for your dll was as well. You could have just compiled it as an .exe

[Corroder]

Thanks for the insight, Dark Byte!

You're absolutely right about the static linking issue. I worked around it by injecting the DLL directly into the target process context using injectDLL() and then resolved the exports via getProcAddress() inside Cheat Engine's Lua environment. Everything works perfectly now, capturing frames smoothly through the CE assembly proxy wrappers.

Thanks again for the support. Case closed! 👍

[AylinCE]

After reviewing your project, I realized you followed very complicated steps.
Although you've closed the topic, please allow me to participate, even if it's late.

Since you're using VS, I have a suggestion for a shorter, clearer, and easier-to-use solution.

The architecture I designed for this system allows you to obtain fully asynchronous, stable screen recording at 30/60 frames per second (FPS) while bypassing Cheat Engine's security restrictions.

Below you will find the detailed Design Description of the project, the C# code to create your project and combine it into a single .exe file, and the necessary installation steps.

The system is based on the principle of two independent processes shaking hands through temporary flag files created on disk.


Invisibility:

Because the C# application compiles as a WinExe (Windows Application), it does not open any console or CMD windows on the system.
It runs entirely in the background, as a silent Windows API process.

We will use the ScreenRecorderLib library in the background.
This library reads screen pixels directly from the GPU using Windows' high-performance (DirectX) API and encodes them in H.264 MP4 format with zero CPU load.

C# Project Setup Steps (Visual Studio), (Also explained separately for those unfamiliar.)
Open Visual Studio and create a new Console App (.NET 8.0 or .NET 9.0) project. Name it "CeScreenRecorder".

Double-click the project's .csproj file and update its contents as follows
(This setting allows the application to run completely windowless and invisibly in the background, and to consolidate all dependencies into a single .exe):