README for the glibc Python pretty printers
===========================================

Pretty printers are gdb extensions that allow it to print useful, human-readable
information about a program's variables.  For example, for a pthread_mutex_t
gdb would usually output something like this:

(gdb) print mutex
$1 = {
  __data = {
    __lock = 22020096,
    __count = 0,
    __owner = 0,
    __nusers = 0,
    __kind = 576,
    __spins = 0,
    __elision = 0,
    __list = {
      __prev = 0x0,
      __next = 0x0
    }
  },
  __size = "\000\000P\001", '\000' <repeats 12 times>, "@\002", '\000' <repeats 21 times>,
  __align = 22020096
}

However, with a pretty printer gdb will output something like this:

(gdb) print mutex
$1 = pthread_mutex_t = {
  Type = Normal,
  Status = Not acquired,
  Robust = No,
  Shared = No,
  Protocol = Priority protect,
  Priority ceiling = 42
}

Before printing a value, gdb will first check if there's a pretty printer
registered for it.  If there is, it'll use it, otherwise it'll print the value
as usual.  Pretty printers can be registered in various ways; for our purposes
we register them for the current objfile by calling
gdb.printing.register_pretty_printer().

Currently our printers are based on gdb.RegexpCollectionPrettyPrinter, which
means they'll be triggered if the type of the variable we're printing matches
a given regular expression.  For example, MutexPrinter will be triggered if
our variable's type matches the regexp '^pthread_mutex_t$'.

Besides the printers themselves, each module may have a constants file which the
printers will import.  These constants are generated from C headers during the
build process, and need to be in the Python search path when loading the
printers.


Installing and loading
----------------------

The pretty printers and their constant files may be installed in different paths
for each distro, though gdb should be able to automatically load them by itself.
When in doubt, you can use the 'info pretty-printer' gdb command to list the
loaded pretty printers.

If the printers aren't automatically loaded for some reason, you should add the
following to your .gdbinit:

python
import sys
sys.path.insert(0, '/path/to/constants/file/directory')
end

source /path/to/printers.py

If you're building glibc manually, '/path/to/constants/file/directory' should be
'/path/to/glibc-build/submodule', where 'submodule' is e.g. nptl.


Testing
-------

The pretty printers come with a small test suite based on PExpect, which is a
Python module with Expect-like features for spawning and controlling interactive
programs.  Each printer has a corresponding C program and a Python script
that uses PExpect to drive gdb through the program and compare its output to
the expected printer's.

The tests run on the glibc host, which is assumed to have both gdb and PExpect;
if any of those is absent the tests will fail with code 77 (UNSUPPORTED).
Native builds can be tested simply by doing 'make check'; cross builds must use
cross-test-ssh.sh as test-wrapper, like this:

make test-wrapper='/path/to/scripts/cross-test-ssh.sh user@host' check

(Remember to share the build system's filesystem with the glibc host's through
NFS or something similar).

Running 'make check' on a cross build will only compile the test programs,
without running the scripts.


Adding new pretty printers
--------------------------

Adding new pretty printers to glibc requires following these steps:

1. Identify which constants must be generated from C headers, and write the
corresponding .pysym file.  See scripts/gen-as-const.py for more information
on how this works.  The name of the .pysym file must be added to the
'gen-py-const-headers' variable in your submodule's Makefile (without the .pysym
extension).

2. Write the pretty printer code itself.  For this you can follow the gdb
Python API documentation, and use the existing printers as examples.  The printer
code must import the generated constants file (which will have the same name
as your .pysym file).  The names of the pretty printer files must be added
to the 'pretty-printers' variable in your submodule's Makefile (without the .py
extension).

3. Write the unit tests for your pretty printers.  The build system calls each
test script passing it the paths to the test program source, the test program
binary, and the printer files you added to 'pretty-printers' in the previous
step.  The test scripts, in turn, must import scripts/test_printers_common
and call the init_test function passing it, among other things, the name of the
set of pretty printers to enable (as seen by running 'info pretty-printer').
You can use the existing unit tests as examples.

4. Add the names of the pretty printer tests to the 'tests-printers' variable
in your submodule's Makefile (without extensions).  In addition, for each test
program you must define a corresponding CFLAGS-* and CPPFLAGS-* variable and
set it to $(CFLAGS-printers-tests) to ensure they're compiled correctly.  For
example, test-foo-printer.c requires the following:

CFLAGS-test-foo-printer.c := $(CFLAGS-printers-tests)
CPPFLAGS-test-foo-printer.c := $(CFLAGS-printers-tests)

Finally, if your programs need to be linked with a specific library, you can add
its name to the 'tests-printers-libs' variable in your submodule's Makefile.


Known issues
------------

* Pretty printers are inherently coupled to the code they're targetting, thus
any changes to the target code must also update the corresponding printers.
On the plus side, the printer code itself may serve as a kind of documentation
for the target code.

* There's no guarantee that the information the pretty printers provide is
complete, i.e. some details might be left off.  For example, the pthread_mutex_t
printers won't report whether a thread is spin-waiting in an attempt to acquire
the mutex.

* Older versions of the gdb Python API have a bug where
gdb.RegexpCollectionPrettyPrinter would not be able to get a value's real type
if it was typedef'd.  This would cause gdb to ignore the pretty printers for
types like pthread_mutex_t, which is defined as:

typedef union
{
  ...
} pthread_mutex_t;

This was fixed in commit 1b588015839caafc608a6944a78aea170f5fb2f6, and released
as part of gdb 7.8.  However, typedef'ing an already typedef'd type may cause
a similar issue, e.g.:

typedef pthread_mutex_t mutex;
mutex a_mutex;

Here, trying to print a_mutex won't trigger the pthread_mutex_t printer.

* The test programs must be compiled without optimizations.  This is necessary
because the test scripts rely on the C code structure being preserved when
stepping through the programs.  Things like aggressive instruction reordering
or optimizing variables out may make this kind of testing impossible.

			TUNABLE FRAMEWORK
			=================

Tunables is a feature in the GNU C Library that allows application authors and
distribution maintainers to alter the runtime library behaviour to match their
workload.

The tunable framework allows modules within glibc to register variables that
may be tweaked through an environment variable.  It aims to enforce a strict
namespace rule to bring consistency to naming of these tunable environment
variables across the project.  This document is a guide for glibc developers to
add tunables to the framework.

ADDING A NEW TUNABLE
--------------------

The TOP_NAMESPACE macro is defined by default as 'glibc'.  If distributions
intend to add their own tunables, they should do so in a different top
namespace by overriding the TOP_NAMESPACE macro for that tunable.  Downstream
implementations are discouraged from using the 'glibc' top namespace for
tunables they don't already have consensus to push upstream.

There are three steps to adding a tunable:

1. Add a tunable to the list and fully specify its properties:

For each tunable you want to add, make an entry in elf/dl-tunables.list.  The
format of the file is as follows:

TOP_NAMESPACE {
  NAMESPACE1 {
    TUNABLE1 {
      # tunable attributes, one per line
    }
    # A tunable with default attributes, i.e. string variable.
    TUNABLE2
    TUNABLE3 {
      # its attributes
    }
  }
  NAMESPACE2 {
    ...
  }
}

The list of allowed attributes are:

- type:			Data type.  Defaults to STRING.  Allowed types are:
			INT_32, UINT_64, SIZE_T and STRING.  Numeric types may
			be in octal or hexadecimal format too.

- minval:		Optional minimum acceptable value.  For a string type
			this is the minimum length of the value.

- maxval:		Optional maximum acceptable value.  For a string type
			this is the maximum length of the value.

- default:		Specify an optional default value for the tunable.

- env_alias:		An alias environment variable

- security_level:	Specify security level of the tunable for AT_SECURE
			binaries.  Valid values are:

			SXID_ERASE: (default) Do not read and do not pass on to
			child processes.
			SXID_IGNORE: Do not read, but retain for non-AT_SECURE
			child processes.
			NONE: Read all the time.

2. Use TUNABLE_GET/TUNABLE_SET/TUNABLE_SET_WITH_BOUNDS to get and set tunables.

3. OPTIONAL: If tunables in a namespace are being used multiple times within a
   specific module, set the TUNABLE_NAMESPACE macro to reduce the amount of
   typing.

GETTING AND SETTING TUNABLES
----------------------------

When the TUNABLE_NAMESPACE macro is defined, one may get tunables in that
module using the TUNABLE_GET macro as follows:

  val = TUNABLE_GET (check, int32_t, TUNABLE_CALLBACK (check_callback))

where 'check' is the tunable name, 'int32_t' is the C type of the tunable and
'check_callback' is the function to call if the tunable got initialized to a
non-default value.  The macro returns the value as type 'int32_t'.

The callback function should be defined as follows:

  void
  TUNABLE_CALLBACK (check_callback) (int32_t *valp)
  {
  ...
  }

where it can expect the tunable value to be passed in VALP.

Tunables in the module can be updated using:

  TUNABLE_SET (check, val)

where 'check' is the tunable name and 'val' is a value of same type.

To get and set tunables in a different namespace from that module, use the full
form of the macros as follows:

  val = TUNABLE_GET_FULL (glibc, cpu, hwcap_mask, uint64_t, NULL)

  TUNABLE_SET_FULL (glibc, cpu, hwcap_mask, val)

where 'glibc' is the top namespace, 'cpu' is the tunable namespace and the
remaining arguments are the same as the short form macros.

The minimum and maximum values can updated together with the tunable value
using:

  TUNABLE_SET_WITH_BOUNDS (check, val, min, max)

where 'check' is the tunable name, 'val' is a value of same type, 'min' and
'max' are the minimum and maximum values of the tunable.

To set the minimum and maximum values of tunables in a different namespace
from that module, use the full form of the macros as follows:

  val = TUNABLE_GET_FULL (glibc, cpu, hwcap_mask, uint64_t, NULL)

  TUNABLE_SET_WITH_BOUNDS_FULL (glibc, cpu, hwcap_mask, val, min, max)

where 'glibc' is the top namespace, 'cpu' is the tunable namespace and the
remaining arguments are the same as the short form macros.

When TUNABLE_NAMESPACE is not defined in a module, TUNABLE_GET is equivalent to
TUNABLE_GET_FULL, so you will need to provide full namespace information for
both macros.  Likewise for TUNABLE_SET, TUNABLE_SET_FULL,
TUNABLE_SET_WITH_BOUNDS and TUNABLE_SET_WITH_BOUNDS_FULL.

** IMPORTANT NOTE **

The tunable list is set as read-only after the dynamic linker relocates itself,
so setting tunable values must be limited only to tunables within the dynamic
linker, that too before relocation.

FUTURE WORK
-----------

The framework currently only allows a one-time initialization of variables
through environment variables and in some cases, modification of variables via
an API call.  A future goals for this project include:

- Setting system-wide and user-wide defaults for tunables through some
  mechanism like a configuration file.

- Allow tweaking of some tunables at runtime