Analyzing a Linux Memory Dump

Earlier, we talked about different techniques that could be used to extract memory from a Linux system. At this point, we have a memory dump, though we are still missing something when it comes to being able to analyze it. Virtuality requires some awareness of how the memory is laid out. With Windows, this is well documented and Virtuality includes profiles for different types of Windows installs between processor architectures and version of Windows. The thing about any type of memory forensics is that you need to know where everything is. When it comes to Linux, different versions of the Linux kernel and different kernel parameters may lead to a different layout of the memory space. The only way to determine the memory layout is to take a look at the memory map. 

A map is a symbol table indicating where in memory to locate a particular function. The memory map includes the address, the type of entry it is and name of the entry. You can see an example of a memory map from a Linux Mint installation on a 64-bit system running a 3.11 kernel. 

c1a3d180 d cpu_worker_pools

c1a3d600 D runqueues

c1a3dc00 d sched_clock_data

c1a3dc40 d csd_data

c1a3dc80 d call_single_queue

c1a3dcc0 d cfd_data

c1a3dd00 D softnet_data

c1a3ddc0 D __per_cpu_end

c1a3e000 D __init_end

c1a3e000 R __smp_locks

c1a45000 B __bss_start

c1a45000 R __smp_locks_end

c1a45000 b initial_pg_pmd

c1a46000 b initial_pg_fixmap

c1a47000 B empty_zero_page

c1a48000 B swapper_pg_dir

c1a49000 b dummy_mapping

c1a4a000 B idt_table

c1a4b000 B trace_idt_table

c1a4c000 b bm_pte

c1a4d000 B initcall_debug

c1a4d004 B reset_devices

c1a4d008 B saved_command_line

c1a4d00c b panic_param

 

You can see three columns in this memory map. The first is the address in memory that the entry can be located. The second is the type of entry that it is. While there are a number of different types of memory segments, some of the types you will run into in a system.map file on a Linux system are as follows:

A is an absolute location

B or b is an uninitialized data segment, referred to as BSS and a stack segment

D or d for an initialized data segment

G or g for a global segment of initialized data which may be thought of as a heap

R or r for read only data segments

T or t for text segments, which is where executable code is stored

U for undefined

W or w for weak objects that have not been tagged as weak objects

You can see from the listing above that we have some stack segments and some initialized data segments from the short sample of the system.map file from this one system. This system.map file would be required to create a profile that we can use with Volatile to analyze the memory dump we have created. We also need additional data, though. We need some debugging information, which leads to another package that needs to be installed. The package is based on DWARF, a debugging format. On Ubuntu and systems that are based on it, the package is called dwarfdump. This will provide us with additional information Volatility needs to be able to extract the relevant pieces from the memory dump. Once dwarfdump is installed, we can build the module. In tools/linux inside the Volatility source tree, you run make and that will create a module.dwarf. You can see the process below as well as the dwarf.module that we will need. 

kilroy@quiche:~/volatility-2.3.1/tools/linux$ make 

make -C //lib/modules/3.11.0-12-generic/build CONFIG_DEBUG_INFO=y M=/home/kilroy/volatility-2.3.1/tools/linux modules

make[1]: Entering directory `/usr/src/linux-headers-3.11.0-12-generic'

  CC [M]  /home/kilroy/volatility-2.3.1/tools/linux/module.o

  Building modules, stage 2.

  MODPOST 1 modules

  CC      /home/kilroy/volatility-2.3.1/tools/linux/module.mod.o

  LD [M]  /home/kilroy/volatility-2.3.1/tools/linux/module.ko

make[1]: Leaving directory `/usr/src/linux-headers-3.11.0-12-generic'

dwarfdump -di module.ko > module.dwarf

make -C //lib/modules/3.11.0-12-generic/build M=/home/kilroy/volatility-2.3.1/tools/linux clean

make[1]: Entering directory `/usr/src/linux-headers-3.11.0-12-generic'

  CLEAN   /home/kilroy/volatility-2.3.1/tools/linux/.tmp_versions

  CLEAN   /home/kilroy/volatility-2.3.1/tools/linux/Module.symvers

make[1]: Leaving directory `/usr/src/linux-headers-3.11.0-12-generic'

kilroy@quiche:~/volatility-2.3.1/tools/linux$ head module.dwarf 

 

.debug_info

 

<0><0x0+0xb><DW_TAG_compile_unit> DW_AT_producer<"GNU C 4.8.1 -m32 -msoft-float -mregparm=3 -mpreferred-stack-boundary=2 -march=i686 -mtune=generic -maccumulate-outgoing-args -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -g -O2 -p -fno-strict-aliasing -fno-common -fno-delete-null-pointer-checks -freg-struct-return -fno-pic -ffreestanding -fstack-protector -fno-asynchronous-unwind-tables -fno-omit-frame-pointer -fno-optimize-sibling-calls -fno-strict-overflow -fconserve-stack"> DW_AT_language<DW_LANG_C89> DW_AT_name<"/home/kilroy/volatility-2.3.1/tools/linux/module.c"> DW_AT_comp_dir<"/usr/src/linux-headers-3.11.0-12-generic"> DW_AT_stmt_list<0x00000000>

<1><0x1d><DW_TAG_typedef> DW_AT_name<"__s8"> DW_AT_decl_file<0x00000001 include/uapi/asm-generic/int-ll64.h> DW_AT_decl_line<0x00000013> DW_AT_type<<0x00000028>>

 

We now need to create the profile we are going to use before we can actually spin Volatility up. We need to zip up the Dwarf module and the system.map for the system that we gathered the memory dump from. In my case, I am going to use the command zip LinuxMint.zip tools/linux/module.dwarf /boot/System.map-3.11.0-12-generic and that will result in the profile I need. In order to get Volatility to find it, though, we need to put it in the right place. If I were running volatility from the directory that was created when I extracted the tarball,I would put the resulting .zip file into volatility/plugins/overlays//linux. However, if I have gone through the process of installing Volatility after making it, I need to put the resulting zip file into /usr/local/lib/python2.7/dist-packages/volatility-2.3.1-py2.7.egg/volatility/plugins/overlays/linux. This is where Volatilty looks to find the profiles for Linux systems. Once we have it in place, we can verify that Volatility has found it by running vol.py —info and looking for Linux, as seen below. 

kilroy@quiche:~$ vol.py --info | grep Linux

Volatility Foundation Volatility Framework 2.3.1

LinuxLinuxMintx86 - A Profile for Linux LinuxMint x86

linux_banner            - Prints the Linux banner information

linux_yarascan          - A shell in the Linux memory image

 

Based on the name LinuxMint.zip that I gave to the file, Volatility has prepended linux onto it as a profile name because the profile is in the linux folder. The profile name turns into linuxLinuxMint. Now we have a profile and a memory dump file, so we can do a little digging into the memory. The first thing we need to do is figure out what commands we can use. Running vol.py yields a list of commands that are targeted at Windows profiles. In order to get the list of Linux-related commands, we have to run vol.py —info which shows up as an option if you run vol.py -h. Now that we have a list of things we can do, let’s dig into the memory dump we previously obtained. The capture below, showing a couple of commands, is taken from a 64-bit system running Kali Linux. 

 

root@quiche:~# vol.py linux_banner --profile=LinuxSystemx64 -f linux.dd 

Volatility Foundation Volatility Framework 2.3.1

Linux version 3.7-trunk-amd64 (debian-kernel@lists.debian.org) (gcc version 4.7.2 (Debian 4.7.2-5) ) #1 SMP Debian 3.7.2-0+kali8

root@quiche:~# vol.py linux_ifconfig --profile=LinuxSystemx64 -f linux.dd

Volatility Foundation Volatility Framework 2.3.1

Interface        IP Address           MAC Address        Promiscous Mode

---------------- -------------------- ------------------ ---------------

lo               127.0.0.1            00:00:00:00:00:00  False          

eth0             10.0.0.182           00:00:00:00:00:00  False          

 

While these commands are not all that impressive in terms of useful data, they do show that we can extract information from the system dump. There are a lot of other commands we can perform against the memory dump that could be used to extract data from running memory. Perhaps this next example will be more intriguing to you. This is a list of shell commands that have been run. They have been extracted from the memory dump. There is absolutely no question that this is a list of commands that I have run on the system where the memory was captured. The command time is a little more specious and it seems to reflect the date and time that the capture was created, more than the time and date the command was created. 

 

root@quiche:~# vol.py linux_bash --profile=LinuxSystemx64 -f linux.dd

Volatility Foundation Volatility Framework 2.3.1

Pid      Name                 Command Time                   Command

-------- -------------------- ------------------------------ -------

    3577 bash                 2014-04-01 00:11:43 UTC+0000   cd /media/cdrom/

    3577 bash                 2014-04-01 00:11:43 UTC+0000   ls /etc/init.d

    3577 bash                 2014-04-01 00:11:43 UTC+0000   ifconfig -a

    3577 bash                 2014-04-01 00:11:43 UTC+0000   cd

    3577 bash                 2014-04-01 00:11:43 UTC+0000   rm -Rf installer

    3577 bash                 2014-04-01 00:11:43 UTC+0000   ls

    3577 bash                 2014-04-01 00:11:43 UTC+0000   cd 

    3577 bash                 2014-04-01 00:11:43 UTC+0000   netdiscover

    3577 bash                 2014-04-01 00:11:43 UTC+0000   rm install

    3577 bash                 2014-04-01 00:11:43 UTC+0000   rm -Rf kmods

    3577 bash                 2014-04-01 00:11:43 UTC+0000   sh install

    3577 bash                 2014-04-01 00:11:43 UTC+0000   cp -Rf * ~

    3577 bash                 2014-04-01 00:11:43 UTC+0000   shutdown -r now

    3577 bash                 2014-04-01 00:11:43 UTC+0000   rm -Rf tools

    3577 bash                 2014-04-01 00:11:43 UTC+0000   ls

    3577 bash                 2014-04-01 00:11:43 UTC+0000   shutdown -h now

    3577 bash                 2014-04-01 00:11:43 UTC+0000   ifconfig -a

    3577 bash                 2014-04-01 00:11:43 UTC+0000   ./install

    3577 bash                 2014-04-01 00:11:43 UTC+0000   rm install-gui 

    3577 bash                 2014-04-01 00:11:43 UTC+0000   ./install

    3577 bash                 2014-04-01 00:11:43 UTC+0000   ls

    3577 bash                 2014-04-01 00:11:43 UTC+0000   shutdown -h now

    3577 bash                 2014-04-01 00:11:43 UTC+0000   ls

    3577 bash                 2014-04-01 00:11:43 UTC+0000   rm version

    3577 bash                 2014-04-01 00:11:43 UTC+0000   ls

    3577 bash                 2014-04-01 00:11:43 UTC+0000   clear

    3577 bash                 2014-04-01 00:11:43 UTC+0000   sudo nmap -sS -O -T 4 172.30.42.41

 

The interesting thing about this is the process id shown in this list appears to be the same across all of the commands in the list. Checking the system itself, there is a process that has that particular pid and the process is a bash process. The commands were not all run from that bash session, though, but clearly the shell loads all of the history into memory for use. I can also check the memory dump to see if there was a process with the pid 3577. I would use the linux_pslist command to look for that process. That command will show us the list of processes that were running at the time the capture was created. Searching through the list of processes can be a very time consuming task. Running linux_pstree is much faster. You can see the output below showing the process tree that includes the bash process with a pid of 3577 that had the history loaded up into it. 

 

.gdm3                2468            0              

..gdm-simple-slav    2474            0              

...Xorg              2482            0              

...gdm-session-wor   3219            0              

....x-session-manag  3256            0              

.....ssh-agent       3317            0              

.....gnome-settings- 3326            0              

.....metacity        3361            0              

.....gnome-panel     3375            0              

......gnome-terminal 3570            0              

.......gnome-pty-helpe 3576            0              

.......bash          3577            0              

 

When you run vol.py —info | grep linux, you can see the list of commands you can run, as noted above. You can see that the list is shorter than that of the list of commands available for Windows. One of the most interesting commands available for Windows that, sadly, isn’t available for Linux is the one that extracts files from memory. This difference in the commands available has to do with the way memory is laid out and managed by the operating system. On top of that, Windows is used far more often and there is more call for forensics of Windows systems so it could simply be that there was more development done on the Windows commands and plugins. Whatever the case, Linux has fewer commands that are available but the ones that are available are still very powerful and useful.