Vulnerability Hunting for Sport
If you've been following our blog, you might notice some favoritism when it comes to embedded targets... We've been exploring the NETGEAR R7000 for several blog posts. This pattern stems from a number of product characteristics, one of which is that the device is easy to emulate in QEMU, which provides an alternative to testing the actual device and is much more researcher-friendly. While it's not strictly required, nor is it the only factor that we consider, it does make for a more enjoyable vulnerability research experience! In this blog post, we'll walk through emulating the R7000's UPnP daemon in QEMU to aid in the discovery and exploitation of vulnerabilities. Towards that end, this blog will demonstrate an exploit for a post-authenticated stack overflow vulnerability, and how to easily unpack the encrypted firmware updates for the R7000's Circle update daemon.
Unpacking the R7000 Firmware
git clone https://github.com/ReFirmLabs/binwalk/ cd binwalk vim Dockerfile # Remove the last WORKDIR/USER lines # And change ENTRYPOINT to /bin/bash docker build -t binwalk . cd /path/to/firmware/ docker run -it -v $PWD:/data binwalk cd /data unzip R7000-V220.127.116.11_10.2.112.zip binwalk -e R7000-V18.104.22.168_10.2.112.chk --run-as=root exit
binwalk will extract the firmware's root filesystem into the directory _R7000-V22.214.171.124_10.2.112.chk.extracted/squashfs-root/.
Setting up an Emulator for the R7000
qemu-system-aarch64 -m 2G -M virt -cpu max -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd -drive if=none,file=debian-10-openstack-arm64.qcow2,id=hd0 -device virtio-blk-device,drive=hd0 -device e1000,netdev=net0 -netdev user,id=net0,hostfwd=tcp:127.0.0.1:5555-:22 -nographic # qemu-system-aarch64 = the QEMU emulator to run, will vary based on the # emulated device's architecture # -m 2G = set the amount of memory (2 GB) # -cpu max = Select the specific CPU to emulate, this will vary based # on the emulated device # -drive if=none,file=debian-10-openstack-arm64.qcow2,id=hd0 # = select the specific hard drive file (built or downloaded # previously) # -device virtio-blk-device,drive=hd0 # = assign the hard drive file to the a device # -device e1000,netdev=net0 = assign a NIC # -netdev user,id=net0,hostfwd=tcp:127.0.0.1:5555-:22 # = Specify user networking for the NIC and forward the # host's port 5555 to the guest's port 22 (for ssh)
Next, we need to copy the extracted file system to the VM. Once binwalk has extracted the file system (see the previous section), you'll need to save it to a tarball. This step should be done as root to ensure the owner/groups and permissions are kept intact:
cd /path/to/extracted/rootfs/parent sudo tar cf rootfs.tar squashfs-root/ scp -P 5555 rootfs.tar [email protected]: ssh [email protected] -p 5555 # Username and password will vary tar xf rootfs.tar
After unpacking the file system, we'll make a few modifications to the root filesystem to allow the processes to run. First, we'll need to make a few generic device files and directories that the system will expect:
cd squashfs-root/ mknod dev/urandom c 1 9 mknod dev/null c 1 3 mknod dev/zero c 1 5 mkdir -p tmp/var/run mount --bind /proc/ proc/
On NETGEAR devices, libnvram reads configuration information from the device's NVRAM and is used by most processes to read their relevant settings, such as the configured IP address and device type.
Since we're emulating the target and don't have the NVRAM, we'll need to replace the libnvram library with a custom version that GRIMM developed. In order to get upnpd running, we'll need to return expected values for each setting, which we dumped from a live R7000.
To compile it and copy it to the guest, run the following commands (on the host):
# Install an arm compiler sudo apt install gcc-10-arm-linux-gnueabi # Clone libnvram-faker from the NQ0DF repo git clone https://github.com/grimm-co/NotQuite0DayFriday.git cd NotQuite0DayFriday/2022.04.15-netgear-emulate/libnvram-faker/ # Compile and upload it to the VM make CC=arm-linux-gnueabi-gcc-10 LD=arm-linux-gnueabi-ld all scp -P 5555 libnvram-faker.so [email protected]:squashfs-root/usr/lib/libnvram.so
Running the UPnP Daemon
The next step is to chroot inside of the root file system and execute the UPnP daemon:
chroot squashfs-root /bin/sh # /usr/sbin/upnpd
If you try to run the UPnP daemon without the custom libnvram library, the daemon will exit almost immediately after starting. As often happens when emulating a service from an embedded device, some misconfiguration or incorrect device setup has caused the process to end early.
Let's turn to our friendly neighborhood debugger, spider-GDB, to help us figure out what's going wrong. Using our custom libnvram library causes the process to sleep for 5 seconds at startup so we can attach to it with a debugger. After starting up upnpd, we switch to another terminal, start GDB, and then attach to upnpd as shown in the below screenshot. After attaching, we'll set a breakpoint on the exit() function to find where upnpd is exiting.
The above screenshot shows that upnpd called exit from the function at 0x1e9e0. Taking a look at this function in IDA (shown below), we can see that a failure in setsockopt causes the call to exit.
- Fix the underlying network to match what is expected by the process
- Patch the binary to remove the setsockopt call.
- Use GDB to redirect execution flow to return, rather than exit upon failure.
# upnpd daemonizes itself and the parent exits, so we follow the child set follow-fork-mode child # Break at the basic block with the exit call break *0x1E9D4 commands 1 # and jump to the function epilogue set $pc=0x1E988 # set follow fork mode back to parent, so we don't leave upnpd set follow-fork-mode parent cont end
Now we can attach to GDB again, and it'll automatically skip over the exit function call. Without that exit call, upnpd runs normally and we can start dynamically exploring its network interface.
An Old Friend
After a bit of static and dynamic analysis, we noticed a rather suspicious string in the upnpd binary, *#$^. If you've been following our blog, you'll notice that it's the same magic header used for parsing firmware updates used in the web server. As the previous firmware update check contained an exploitable stack overflow, this portion of code is worth examining. Upon further investigation, we can see that this code contains the exact same stack overflow that was previously fixed in the httpd daemon.
After a bit of dynamic testing with the upnpd web interface, we discover that this command is only accessible to authenticated users. Unlike the previous issue in the httpd web interface, the authentication is checked before parsing the firmware update. As such, only an authenticated attacker can exploit this issue. The below screenshots demonstrate how to login to the UPnP server and trigger a stack overflow using this bug. After triggering this bug, we can examine the registers and stack in GDB to help us understand the vulnerability and work on the exploit.
Intro to Anti-Debug
Wait, where was the anti-debug?
- Check /proc/PID/status and /proc/PID/cmdline for the parent process's pid. If these files contain ltrace, strace, or GDB, the check will fail.
- Check for the environment variables for $COLUMNS and $LINES, which are set by GDB.
- Check the current program path environment variable, i.e. $_, to see if it contains the string ltrace, strace, or gdb.
- Check to ensure that the $LD_PRELOAD environment variable is not defined. If defined, this environment variable loads an additional library into the circled's address space. This check detects arbitrary code being injected, rather than trying to detect a debugger.
- Rather than starting circled in GDB, attach to it with GDB's -p argument after it has started. In this case, circled's parent process will be init (PID 1), which pass the checks. The previous section bypasses these checks using this technique.
- Use GDB to catch the program before it runs the debug checks and skip them entirely, similar to how we fixed the UPnP daemon issue earlier in the blog.
- Modify the circled executable to skip the debug checks.
- Replace a library used by circled, rather than injecting an additional one via $LD_PRELOAD.
- Rename the GDB, strace, or ltrace binaries.
- Use a non-GDB debugger.
- Unset the $COLUMNS, $LINES, or $_ environment variables.
- Use an emulation framework, such as Unicorn, to emulate only the decryption function.