[OT] Abusing the C preprocessor

[afoken]

Hi!

At work, I'm currently using x-macros to allow compile-time configuration of some filtering software. It boils down to a configuration file that roughly looks like this:

CFG_START()
CFG_ADD(int, FOO, /* some more stuff */)
CFG_ADD(short, BAR, /* some more stuff */)
CFG_ADD(float, BAZ, /* some more stuff */)
CFG_END()

This file generates some variables, and a meta-data table for the running software so it can know the types (int, short, float) for the variables associated with the identifiers (FOO, BAR, BAZ).

So far, no problems, just a little bit of creative usage of the C preprocessor.

The running software has to invoke qsort. qsort needs a compare function. The meta-data table contains pointers to the compare function matching the type, simply by appending the type name to a common prefix:

/* simplified */
MetaData_t metadata = {
    #define CFG_ADD(TYPE,NAME,STUFF) { .compFunc = compare_ ## TYPE; /
+* and some more stuff */ }
    #include "configfile.h"
    #undef CFG_ADD
}

My source contains a lot of compare functions for different types that could be used in the configuration file. But not all of the functions are actually used, because not all of the types are used in the configuration file. So, the compiler warns about some unused functions (e.g. compare_double() for the example shown above).

Yes, it is harmless, but ugly. Our code should compile without warnings. Disabling the warning is not an option. I would prefer to be able to hide unused functions from the C compiler.

Something like this (and yes, I know this won't work):

#define CFG_ADD(TYPE,NAME,STUFF) \
    #ifndef USING_ ## TYPE /* does not work */ \
    #define USING_ ## TYPE /* does not work */ \
    #endif
#include "configfile.h"
#undef CFG_ADD

/* and later: */

#ifdef USING_int
static int compare_int(const void * a, const void * b);
#endif
#ifdef USING_double
static int compare_double(const void * a, const void * b);
#endif

/* further down */

#ifdef USING_int
static int compare_int(const void * a, const void * b)
{
    /* ... */
}
#endif

#ifdef USING_double
static int compare_double(const void * a, const void * b)
{
    /* ... */
}
#endif

Any clever ideas that I missed?

Environment: GCC 4.x cross-compiling for an embedded system (so code and data size might become a problem). GCC might be updated in the future, but switching to a different compiler is very unlikely. So a GCC-only solution is acceptable.

Alexander

[roboticus]

afoken:

Update: I think I misunderstood what was wanted originally (removal of warnings and minimization of unused code), so while my original answer did what I wanted it to do, I doubt it did what afoken wanted done. So here's what I think you want.

First, you can create a library of the various methods you may want to include:

$ cat lib_int.c
int compare_int(const void *a, const void *b) {
    return *(int *)a - *(int *)b;
}

$ cat lib_double.c
int compare_double(const void *a, const void* b) {
    return *(double *)a - *(double *)b;
}

$ gcc -c -o lib_int.o lib_int.c

$ gcc -c -o lib_double.o lib_double.c

$ ar cr libtypes.a lib_int.o lib_double.o

Now your program can simply use the extern directive to tell the linker which items to pull into your executable:

$ cat foo.c
#include <stdio.h>
#include <stdlib.h>

#if defined(DO_INT)
int     a1[] = { 1, 100, 5, 50, 25 };
#endif

#if defined(DO_DOUBLE)
double  a2[] = { 1.0, 10.0, 5.0, 25.0 };
#endif

int (*cmp_fn)(const void *, const void *);

#define CFG_ADD(TYPE)       \
    extern int compare_ ## TYPE(const void*, const void*);   \
    cmp_fn = compare_ ## TYPE

int main(int argc, char **argv) {

#if defined(DO_INT)
    CFG_ADD(int);
    qsort(a1, sizeof(a1)/sizeof(a1[0]), sizeof(a1[0]), cmp_fn);
    for (int i=0; i<sizeof(a1)/sizeof(a1[0]); i++) {
        printf("%d, ", a1[i]);
    }
#endif

#if defined(DO_DOUBLE)
    printf("\n\n");
    CFG_ADD(double);
    qsort(a2, sizeof(a2)/sizeof(a2[0]), sizeof(a2[0]), cmp_fn);
    for (int i=0; i<sizeof(a2)/sizeof(a2[0]); i++) {
        printf("%f, ", a2[i]);
    }
#endif

}

$ gcc foo.c -DDO_INT -L. -ltypes -o foo_int

$ gcc foo.c -DDO_DOUBLE -L. -ltypes -o foo_double

$ gcc foo.c -DDO_INT -DDO_DOUBLE -L. -ltypes -o foo_both

And you can tell that the linker did what it wanted:

$ ./foo_int
1, 5, 25, 50, 100,

$ ./foo_double

1.000000, 5.000000, 10.000000, 25.000000,

$ ./foo_both
1, 5, 25, 50, 100,

1.000000, 5.000000, 10.000000, 25.000000,

$ nm foo_int.exe | grep compare
0000000100403040 r .rdata$.refptr.compare_int
0000000100403040 R .refptr.compare_int
0000000100401170 T compare_int

$ nm foo_double.exe | grep compare
0000000100403040 r .rdata$.refptr.compare_double
0000000100403040 R .refptr.compare_double
00000001004011a0 T compare_double

$ nm foo_both.exe | grep compare
0000000100403040 r .rdata$.refptr.compare_double
0000000100403050 r .rdata$.refptr.compare_int
0000000100403040 R .refptr.compare_double
0000000100403050 R .refptr.compare_int
0000000100401230 T compare_double
0000000100401210 T compare_int

I hope this gets you where you want to go...

My original response was:

Read more... (1457 Bytes)

...roboticus

When your only tool is a hammer, all problems look like your thumb.

[afoken]

Yes, a library would be the cleanest way to go. Except that it would require a good amount of changes. Currently, all we need to do to include the filtering software in a project is to copy a source file and a header file into the project, and write a simple configuration file. That works for all of our platforms and build environments. (Actually, this is what can be done in about a week. So far, we need to change the source file for each project.)

I'll rely on gcc optimizing away unused functions, and disable the warnings for the compare functions that may be unused. (Re^2: [OT] Abusing the C preprocessor).

Alexander

[Anonymous Monk]

Well, you cannot use #if conditions in a macro replacement. You could use indirection, though:

#define CFG_USED(type, name, stuff) \
        { .compFunc = compare_ ## type; /* */ },
#define CFG_UNUSED(type, name, stuff)   /* unused */

#define CFG_ADD(type, name, stuff)  CFG_USE_ ## type (type,name,stuff)

#ifdef USING_int
#define CFG_USE_int CFG_USED
#else
#define CFG_USE_int CFG_UNUSED
#endif
...

Regarding warnings: you can suppress unused warnings on individual basis with an __attribute__((__unused__)).

The library approach suggested by roboticus is a good one. Let the linker work it out where possible. However, some types may not be available on all platforms, etc., so this is not always an option.

But then, there are more options. You could define the compare functions via inline versions:

static inline int _inline_cmpfunc_int(void *a, void *b)
{
        int _a = *(int*)a, _b = *(int*)b;
        return (_a > _b) - (_a < _b);
}

There should be no warnings for unused inline functions. After that, you might use the same configure mechanism to conditionally generate function definitions:

static foo
compare_foo(void *a, void *b) { return _inline_cmpfunc_foo(a, b); }

Tricks like #include "definitions.h" multiple times, are sometimes used.

[afoken]

#define CFG_USED(type, name, stuff) \
        { .compFunc = compare_ ## type; /* */ },
#define CFG_UNUSED(type, name, stuff)   /* unused */

#define CFG_ADD(type, name, stuff)  CFG_USE_ ## type (type,name,stuff)

#ifdef USING_int
#define CFG_USE_int CFG_USED
#else
#define CFG_USE_int CFG_UNUSED
#endif

I don't get it. To me, this looks backwards. I don't wan't to define USING_int. I want the precompiler to "know" if the configuration file contains one or more CFG_ADD(int, ...) or not. Building on that "knowledge", the precompiler should either emit the function definition for compare_int() or not.

You could define the compare functions via inline versions. [...] There should be no warnings for unused inline functions.

Nice idea. I suspected that CC would not accept inline functions for function pointer, but my Linux gcc 5.x does:

/tmp>cat testing.c
#include <stdio.h>
#include <stdlib.h>

static inline int compare_short(const void * a, const void * b)
{
        if (*(short *)a < *(short *)b) return -1;
        if (*(short *)a > *(short *)b) return 1;
        return 0;
}

static inline int compare_int(const void * a, const void * b)
{
        if (*(int *)a < *(int *)b) return -1;
        if (*(int *)a > *(int *)b) return 1;
        return 0;
}

static int foo[5]={47,11,0,8,15};

int main(int argc, char ** argv)
{
        qsort(foo,sizeof(foo)/sizeof(foo[0]),sizeof(foo[0]),&compare_i
+nt);
}

/tmp>CFLAGS="-Wall -pedantic" make testing
cc -Wall -pedantic    testing.c   -o testing

/tmp>cc --version
cc (GCC) 5.3.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There i
+s NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PUR
+POSE.

/tmp>

And so does the gcc 4.x for my target processor. COOL!

After that, you might use the same configure mechanism to conditionally generate function definitions:

static foo compare_foo(void *a, void *b) { return _inline_cmpfunc_foo(
+a, b); }

That function does not make sense in this form. Compare functions are expected to return int, and I guess that you wanted to call _inine_cmpfunc_int(a,b). So I assume you meant to wrote something like this:

#define CFG_ADD(TYPE,NAME,STUFF) \
    static int compare_ ## NAME(const void * a, const void * b) \
    { \
        return _inline_cmpfunc_ ## TYPE(a,b); \
    }

This is great - except that now, the compiler may legally generate one compare function per NAME. Assuming a configuration file with 10 ints and 10 shorts, the compiler could emit 20 compare functions, 10 for int and 10 for short, and all would be in use. That's even worse than the current situation: Two compare functions for int and short, and eight unused functions for other types.

I think gcc should be able to optimize and ignore the inline. But then, gcc should also be able to optimize the eight unused functions away. Does it optimize in that way? And will it still optimize that way when we update to a gcc version three major version numbers away from now?

And here we are at a point that I did not mention. Having to answer "It depends" to a yes-or-no question is not acceptable in that project. The system will run in a highly regulated environment, and we need predictable behaviour.

Regarding warnings: you can suppress unused warnings on individual basis with an __attribute__((__unused__)).

Right. But the warning is not just cosmetic in this case. The compiler generates code that should not be generated. I could not care less if the target system was a PC from this century. But my target system is an embedded microcontroller with limited program memory.

Running some primitive tests, it seems that gcc not only warns about unused functions, but actually removes them, even when optimizing is disabled (-O0).

I think this is the way to go. I'll wrap __attribute__((__unused__)) in a macro named MAYBE_UNUSED (or similar), and at a lot of documentation explaining why and how this works together with gcc. The nice thing about this solution is that it is testable. We can simply look into the map file and search for the name of the functions that should not be there.

THANKS!

Alexander

[Anonymous Monk]

But then, gcc should also be able to optimize the eight unused functions away. Does it optimize in that way? And will it still optimize that way when we update to a gcc version three major version numbers away from now?

Functions declared with static have "internal linkage". When unused (have no direct calls nor reference taken), the compiler can eliminate those functions as dead code. Depending on this optimization is quite reasonable.

I'd wholeheartedly suggest exploring the effect of different options on generated code and looking at the assembly in general. Gcc with -O1 or -Os is usually okay, except gcc -Os does not optimize constant divisions via muls (e.g. x = x / 10).

Keeping a basic tally of object sizes is also advisable. I've used makefile rules to objdump -t foo.o > foo.syms and a simple symcmp.pl to report a sorted diff between object directories so that any big regressions with a new compiler will stick out in the noise.