xmalloc() considered harmful

I’ve seen many places where xmalloc() and xfree() are used as replacements for the standard malloc() and free() functions (most of the time, to call abort() or print debugging/log messages to some file.

The most recent example of these macros was the source of an HTTP proxy server.

My short comment to this sort of macros is:

If you feel inclined to add debugging and/or logging support to malloc() and free(), then BY ALL MEANS steer clearly and far away from such ugly “hacks”.

There are various reasons why using the C preprocessor to wrap malloc() is silly and will turn around and bite you in the end:

  • You do not really have any sort of control about what malloc() function is called by already compiled, “object” code.
  • You will probably have to resort to even more ugly preprocessor hacks to handle the portability aspects of this code. Some systems define their internal malloc() function as void *malloc(size_t); others use int as the size argument; some may typedef size_t using compile-mode specific hacks (i.e. to support “large files” or a segmented memory model).
  • You will probably miss some of the entry points to the system malloc() code, i.e. the posix_memalign() function; once you do, there are two possibilities: you will either print bogus traces/logs or you will misinterpret some of the traced data and consider it a bug—when, in reality, there is no bug in the allocator, but in the wrappers you wrote.

Before you embark on the “journey” of the xwrapper() functions, make sure that you spend a considerable amount of time pondering the following:

Do you really think you can outsmart the platform vendor, who has spent a lot of time testing and verifying the system malloc() implementation?

About these ads
This entry was posted in Uncategorized and tagged , , , , , , . Bookmark the permalink.

4 Responses to xmalloc() considered harmful

  1. Marty Leisner says:

    I really don’t understand the point:

    Do you really think you can outsmart the platform vendor, who has spent a lot of time testing and verifying the system malloc() implementation?

    Malloc can fail. You’re out of memory.

    Do you sprinkle 100s of ifs around your code or do you do it once.

    marty

  2. keramida says:

    Hi Marty,

    ‘Out of memory’ is an exceptional condition. Handling an exception deep inside the calling stack, where xmalloc() lives may not be possible without ‘more context’.

    I tend to prefer type-stable wrappers around malloc, who have exactly this sort of extra context to deal with malloc() failures. This way instead of forcing something like “oops, we failed in malloc(), let’s fall back to abort()”, you can do smarter things like call back into a zone allocator to flush its cached objects, hoping that you can restart malloc() and succeed.

    My main objections against xmalloc() are then: (a) that it is an attempt to force one particular way of failing during an allocation to every part of the program, (b) that is doesn’t necessarily have enough context to do a ‘smarter’ thing, and (c) that it almost certainly doesn’t work with thirdparty object code at all, or that it may even conflict with object code that exports the same symbol but does something amazingly stupid or at least sufficiently different to cause problems.

  3. keramida says:

    Oh, and “Yes, I sprinkle a lot of checks all over the place”. I sometimes overdo it, but fortunately profilers can often point precisely at the places where I went completely over the edge.

  4. Lelanthran says:

    I just did a quick xmalloc/xfree library. My reasons were not to “outsmart” the vendor. I am calling C code from ECL (which has it’s own garbage collector), and thus could not run tests using valgrind. My solution was to write xmalloc/xfree that kept a list of each allocation, it’s size and the filename, function and line number it was called from.

    When an error is detected (running out of memory is a good one) I don’t want the code to try recovery – I want it to die, immediately, leaving a corefile and the blocklist from xmalloc/xfree.. Running out of memory is a clear indication of a memory leak. The program is designed to use a certain amount, and that is deterministic. If a malloc returns NULL it means that I’ve got a leak somewhere, and at that point I want it to go down immediately so I can see what the stack looked like. Viewing the blocklist is a bonus.

    Since my xmalloc/xfree can be tuned during execution to switch over to being a thin wrapper around malloc/free, there is no actual downside to this.

Comments are closed.