Svadev mailing list

[John Criswell <criswell AT illinois.edu>]

On 10/24/11 6:32 PM, Matthew Wala wrote:
> It looks like you've configured everything okay. As far as I know no
> one's been running the tests in the mem_safety directory recently so
> it's not surprising SAFECode isn't catching everything.
>
> There's at least two reasons I can think of that a lot of the double
> frees are going unnoticed. Firstly just going through the runtime
> library source code it seems that the function checkForBadFrees is
> currently disabled in the debug runtime (I'm not sure why and I don't
> know much about that part of the code, so something else might be
> going on).

That's correct.  Looking over the revision history, I found out why I 
disabled it.  Quoting from the commit log of Revision 136632:

"Do not report errors for bad frees; we need complete vs. incomplete 
versions of pool_unregister() to report errors accurately."

The short answer is that several enhancements need to be made to get it 
to work without generating false positives.

For the curious, there are two types of checks in SAFECode: incomplete 
and complete.  When SAFECode transforms a program one compilation unit 
at a time within Clang, it does not know everything about the program 
(because it can't do whole-program analysis), and so it inserts 
incomplete checks.  Incomplete checks try their best to detect memory 
safety errors, but sometimes they conservatively allow operations to 
proceed if they can't find the memory object in question in the lookup 
tables.  The assumption is that the memory object isn't registered 
because it was allocated by external code.

SAFECode's version of libLTO (which is now available but not yet 
documented in the Install Guide) will do whole-program analysis and 
convert incomplete checks to complete checks when appropriate.  It will 
use DSA to figure out which pointers always point to memory objects 
allocated within the program and which pointers can point to memory 
objects allocated by external library code.  Checks on the former 
pointers will be changed to complete checks; these checks will raise a 
SAFECode run-time error if they can't find the memory object to which 
the pointer points.

So, there's three issues here:

1) We need to have complete and incomplete versions of the checks for 
invalid frees.  Real programs were flagging false positives because 
checkForBadFrees() always acted like a complete check.

2) We need to get libLTO polished and ready.  Many of SAFECode's checks 
just aren't valuable without it.

3) We should modify SAFECode's libLTO to transform the program to use 
the automatic pool allocation memory allocator.  This allocator is 
tolerant of invalid frees (i.e., it will detect them and ignore them).


>   Also a number of those tests use indirect function calls to
> free(), which I think SAFECode doesn't handle....

This is a good point.  There was a transform pass called 
RaiseAllocationsPass that ensured that all calls to malloc() and free() 
were direct calls.  I don't recall if this was an LLVM pass or Poolalloc 
pass, but we should get it working again with LLVM 3.0.

Time to go file some bug reports...

-- John T.

> Matt
>
> On Mon, Oct 24, 2011 at 3:46 PM, geremy 
> condra<debatem1 AT gmail.com>
>   wrote:
> > Hey all, me again.
> >
> > After getting safecode to compile cleanly against 3.0 (thanks for all
> > the help there!) I started running through some of the test cases to
> > help me understand what the limitations of the technology were. While
> > I'm still not 100% sure I'm doing everything right, it looks like a
> > lot of the tests in the safecode/test/mem_safety/double_free directory
> > are failing. Can you help me understand why, for instance
> > double_free-001.c still runs without generating safecode warnings? Or
> > why double_free-031.c aborts with a glibc double free error? I assume
> > I've misconfigured something.
> >
> > Here's how I'm building:
> >
> >
> > #! /bin/sh
> >
> > set -e
> >
> > # We do all of this in /tmp to make sure we're building locally
> > cd /tmp
> >
> > # Get mainline llvm
> > echo "Getting llvm..."
> > svn co http://llvm.org/svn/llvm-project/llvm/tags/RELEASE_30 llvm
> >
> > # Get all the projects we need
> > echo "Getting safecode projects..."
> > cd llvm/rc1/projects
> > svn co http://llvm.org/svn/llvm-project/poolalloc/trunk poolalloc
> > svn co http://llvm.org/svn/llvm-project/safecode/trunk safecode
> > svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
> >
> > # Configure everything
> > echo "Configuring..."
> > cd ..
> > ./configure
> >
> > # Make everything
> > echo "Making..."
> > make -j16 tools-only>  /tmp/make_tools.out
> > cd projects/poolalloc
> > make -j16>  /tmp/make_poolalloc.out
> > cd ../safecode
> > make -j16>  /tmp/make_safecode.out
> > cd ..
> >
> > # Install
> > echo "Installing..."
> > cd ..
> > sudo make install>  /tmp/install_all.out
> > cd projects/poolalloc
> > sudo make install>  /tmp/install_poolalloc.out
> > cd ../safecode
> > sudo make install>  /tmp/install_safecode.out
> >
> > # Go home
> > echo "Done."
> >
> >
> >
> > Here's how I'm compiling and running the test:
> >
> > clang -g -fmemsafety -o test double_free-031.c -L/usr/local/lib
> > -lsc_dbg_rt -lpoolalloc_bitmap -lstdc++ -lgdtoa
> > ./test
> >
> >
> >
> > And here's the output:
> >
> > user@machine:/tmp/llvm/rc1/projects/safecode/test/mem_safety/double_free$
> >  ./test
> > 0x402910
> > 0x402910
> > *** glibc detected *** ./test: double free or corruption (!prev):
> > 0x000000000161d830 ***
> > ======= Backtrace: =========
> > /lib/libc.so.6(+0x775b6)[0x7f69286fb5b6]
> > /lib/libc.so.6(cfree+0x73)[0x7f6928701e83]
> > ./test[0x40334c]
> > ./test[0x403631]
> > /lib/libc.so.6(__libc_start_main+0xfd)[0x7f69286a2c4d]
> > ./test[0x402bb9]
> > ======= Memory map: ========
> > 00400000-00435000 r-xp 00000000 fc:00 1847636
> >   /tmp/llvm/rc1/projects/safecode/test/mem_safety/double_free/test
> > 00634000-00635000 r--p 00034000 fc:00 1847636
> >   /tmp/llvm/rc1/projects/safecode/test/mem_safety/double_free/test
> > 00635000-00636000 rw-p 00035000 fc:00 1847636
> >   /tmp/llvm/rc1/projects/safecode/test/mem_safety/double_free/test
> > 0161d000-0163e000 rw-p 00000000 00:00 0                                  
> > [heap]
> > 7f6924000000-7f6924021000 rw-p 00000000 00:00 0
> > 7f6924021000-7f6928000000 ---p 00000000 00:00 0
> > 7f69281ea000-7f6928200000 r-xp 00000000 fc:00 393235
> >   /lib/libgcc_s.so.1
> > 7f6928200000-7f69283ff000 ---p 00016000 fc:00 393235
> >   /lib/libgcc_s.so.1
> > 7f69283ff000-7f6928400000 r--p 00015000 fc:00 393235
> >   /lib/libgcc_s.so.1
> > 7f6928400000-7f6928401000 rw-p 00016000 fc:00 393235
> >   /lib/libgcc_s.so.1
> > 7f6928401000-7f6928483000 r-xp 00000000 fc:00 394146
> >   /lib/libm-2.11.1.so
> > 7f6928483000-7f6928682000 ---p 00082000 fc:00 394146
> >   /lib/libm-2.11.1.so
> > 7f6928682000-7f6928683000 r--p 00081000 fc:00 394146
> >   /lib/libm-2.11.1.so
> > 7f6928683000-7f6928684000 rw-p 00082000 fc:00 394146
> >   /lib/libm-2.11.1.so
> > 7f6928684000-7f69287fe000 r-xp 00000000 fc:00 393815
> >   /lib/libc-2.11.1.so
> > 7f69287fe000-7f69289fd000 ---p 0017a000 fc:00 393815
> >   /lib/libc-2.11.1.so
> > 7f69289fd000-7f6928a01000 r--p 00179000 fc:00 393815
> >   /lib/libc-2.11.1.so
> > 7f6928a01000-7f6928a02000 rw-p 0017d000 fc:00 393815
> >   /lib/libc-2.11.1.so
> > 7f6928a02000-7f6928a07000 rw-p 00000000 00:00 0
> > 7f6928a07000-7f6928afd000 r-xp 00000000 fc:00 131545
> >   /usr/lib/libstdc++.so.6.0.13
> > 7f6928afd000-7f6928cfd000 ---p 000f6000 fc:00 131545
> >   /usr/lib/libstdc++.so.6.0.13
> > 7f6928cfd000-7f6928d04000 r--p 000f6000 fc:00 131545
> >   /usr/lib/libstdc++.so.6.0.13
> > 7f6928d04000-7f6928d06000 rw-p 000fd000 fc:00 131545
> >   /usr/lib/libstdc++.so.6.0.13
> > 7f6928d06000-7f6928d1b000 rw-p 00000000 00:00 0
> > 7f6928d1b000-7f6928d3b000 r-xp 00000000 fc:00 394186
> >   /lib/ld-2.11.1.so
> > 7f6928f18000-7f6928f1c000 rw-p 00000000 00:00 0
> > 7f6928f37000-7f6928f3a000 rw-p 00000000 00:00 0
> > 7f6928f3a000-7f6928f3b000 r--p 0001f000 fc:00 394186
> >   /lib/ld-2.11.1.so
> > 7f6928f3b000-7f6928f3c000 rw-p 00020000 fc:00 394186
> >   /lib/ld-2.11.1.so
> > 7f6928f3c000-7f6928f3d000 rw-p 00000000 00:00 0
> > 7fff9e35b000-7fff9e37c000 rw-p 00000000 00:00 0                          
> > [stack]
> > 7fff9e3ff000-7fff9e400000 r-xp 00000000 00:00 0                          
> > [vdso]
> > ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
> >   [vsyscall]
> > Aborted
> >
> >
> >
> > The same procedure works on many other tests, generating the correct
> > safecode warnings, so I'm fairly confused. Apologies if this is a dumb
> > question, and thanks for your time-
> >
> > Geremy Condra
> > _______________________________________________
> > svadev mailing list
> > svadev AT cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/svadev
> _______________________________________________
> svadev mailing list
> svadev AT cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/svadev