LLVM-YX-Callobfuscator : Enhancing Windows x64 Native Call Security At Compile Time

A powerful LLVM plugin designed to bolster the security of Windows x64 native calls during the compilation process.

This innovative tool transparently applies stack spoofing and indirect syscalls, providing enhanced protection against potential threats.

LLVM plugin to transparently apply stack spoofing and indirect syscalls to Windows x64 native calls at compile time.

“I’ve 5 Mins, What Is This?”

This project is a plugin meant to be used with opt, the LLVM optimizer.

Opt will use the pass included in this plugin to hook calls to Windows functions based on a config file and point those calls to a single function, which, given an ID identifying the function to be called, will apply dynamic stack obfuscation, and if the function is a syscall stub, will invoke it using “indirect syscalling”.

Usage brief: Set up a config file indicating the functions to be hooked, write your C code without caring about Windows function calls, compile with clang to generate .ir files, give them to opt along with this plugin, opt hooks functions, llc compiles the .ir to a.obj, and ld links it to an executable that automatically obfuscates function calls.

Table Of Contents

• Setup
• Usage and example
• Developer guide
  • File distribution
  • How the pass works
  • How the dispatching system works
• Thanks
• TODO

Setup

This instructions are written for Windows, but it should be possible to setup this environment in Linux easily (still output executables can only be built for Windows x64). Tested with LLVM 16.x and 17.x.

All the commands described in the following steps are supossed to be used in an MSYS2 terminal ie they have Linux format.

• Dependencies : To be able to compile this project, we mainly need 2 things: LLVM (libraries and headers, along with some tools) and CMAKE.LLVM libraries and tools can be either compiled from source, downloaded from the LLVM releases, or installed through MSYS2 (using pacman).
  • To make it easier, here we will use MSYS2.
    • We also need to set up CMAKE and our build tools. Clang is required to compile the helpers library. For the linker, we don’t care too much.
      • Lastly, as a generator, I prefer Ninja, but make, nmake or msbuild will work.Everything listed above can be installed with pacman by using MSYS2.
        • Download and install MSYS2.Launch an MSYS2 Mingw64 terminal and install the following packages:Install LLVM: pacman -S mingw-w64-x86_64-llvmInstall Clang: pacman -S mingw-w64-x86_64-clangInstall Nasm: pacman -S mingw-w64-x86_64-nasmInstall Cmake: pacman -S mingw-w64-x86_64-cmakeInstall Ninja: pacman -S mingw-w64-x86_64-ninjaInstall Git: pacman -S git
        Restart the MSYS2 terminal, It may help with env variables, and gives luck for the following building ritual.
• Building : First, clone this project, pretty obvious:

  git clone https://github.com/janoglezcampos/llvm-yx-callobfuscator

To build this project, we will be using CMAKE. Because I find it convenient, I use VScode with the CMake extension. You can find my VSCODE config at .vscode_conf/setting.json

In any other case, launch an MSYS2 Mingw64 terminal and do exactly what I say (without checking what any of the commands Im giving you will do to your beloved machine):

Go to the root directory of this repo:

  cd <whatever>/llvm-yx-obfuscator

Create a build directory and change the directory to it:

  mkdir build; cd build

Choose an install location. I recommend doing this so it will be easier to either get the files in the right place or just be able to pick them up easily.

Also, the usage section will use this folder as the relative folder for accessing these files when needed, so if you add it to the PATH, the commands will work right away.

I use  C:/Users/<my-user>/llvm-plugins, but creating a folder in the project directory called install, side by side with build will do.

This folder will not be edited if you don’t run the install command, but you will have to go get the files in the build folder.

Configure the project; here you specify the installation folder. DCMAKE_BUILD_TYPE will set the default mode: Debug, Release or MinSizeRel.

Depending on which generator you are using, you are going to be able to change this later or not (means you will have to reconfigure or not). I use Nija as generator, but you can use any other.

  cmake -G Ninja -DCMAKE_INSTALL_PREFIX="<path_to_install_folder>" -DCMAKE_BUILD_TYPE=Release ./..

Build the project; optionally choose mode with —-config Release (if your generator lets you):

  cmake --build .

Move the generated files to the install directory you chose before.

If you don’t want to use the install “feature” of CMake, just get the files (libCallObfuscatorHelpers.a and CallObfuscatorPlugin.dll) from the build directory and put them in a place you remember; you will need them.

For more information click here