1=============================
2Guidance for writing policies
3=============================
4
5Try to keep transactionality out of it.  The core is careful to
6avoid asking about anything that is migrating.  This is a pain, but
7makes it easier to write the policies.
8
9Mappings are loaded into the policy at construction time.
10
11Every bio that is mapped by the target is referred to the policy.
12The policy can return a simple HIT or MISS or issue a migration.
13
14Currently there's no way for the policy to issue background work,
15e.g. to start writing back dirty blocks that are going to be evicted
16soon.
17
18Because we map bios, rather than requests it's easy for the policy
19to get fooled by many small bios.  For this reason the core target
20issues periodic ticks to the policy.  It's suggested that the policy
21doesn't update states (eg, hit counts) for a block more than once
22for each tick.  The core ticks by watching bios complete, and so
23trying to see when the io scheduler has let the ios run.
24
25
26Overview of supplied cache replacement policies
27===============================================
28
29multiqueue (mq)
30---------------
31
32This policy is now an alias for smq (see below).
33
34The following tunables are accepted, but have no effect::
35
36	'sequential_threshold <#nr_sequential_ios>'
37	'random_threshold <#nr_random_ios>'
38	'read_promote_adjustment <value>'
39	'write_promote_adjustment <value>'
40	'discard_promote_adjustment <value>'
41
42Stochastic multiqueue (smq)
43---------------------------
44
45This policy is the default.
46
47The stochastic multi-queue (smq) policy addresses some of the problems
48with the multiqueue (mq) policy.
49
50The smq policy (vs mq) offers the promise of less memory utilization,
51improved performance and increased adaptability in the face of changing
52workloads.  smq also does not have any cumbersome tuning knobs.
53
54Users may switch from "mq" to "smq" simply by appropriately reloading a
55DM table that is using the cache target.  Doing so will cause all of the
56mq policy's hints to be dropped.  Also, performance of the cache may
57degrade slightly until smq recalculates the origin device's hotspots
58that should be cached.
59
60Memory usage
61^^^^^^^^^^^^
62
63The mq policy used a lot of memory; 88 bytes per cache block on a 64
64bit machine.
65
66smq uses 28bit indexes to implement its data structures rather than
67pointers.  It avoids storing an explicit hit count for each block.  It
68has a 'hotspot' queue, rather than a pre-cache, which uses a quarter of
69the entries (each hotspot block covers a larger area than a single
70cache block).
71
72All this means smq uses ~25bytes per cache block.  Still a lot of
73memory, but a substantial improvement nonetheless.
74
75Level balancing
76^^^^^^^^^^^^^^^
77
78mq placed entries in different levels of the multiqueue structures
79based on their hit count (~ln(hit count)).  This meant the bottom
80levels generally had the most entries, and the top ones had very
81few.  Having unbalanced levels like this reduced the efficacy of the
82multiqueue.
83
84smq does not maintain a hit count, instead it swaps hit entries with
85the least recently used entry from the level above.  The overall
86ordering being a side effect of this stochastic process.  With this
87scheme we can decide how many entries occupy each multiqueue level,
88resulting in better promotion/demotion decisions.
89
90Adaptability:
91The mq policy maintained a hit count for each cache block.  For a
92different block to get promoted to the cache its hit count has to
93exceed the lowest currently in the cache.  This meant it could take a
94long time for the cache to adapt between varying IO patterns.
95
96smq doesn't maintain hit counts, so a lot of this problem just goes
97away.  In addition it tracks performance of the hotspot queue, which
98is used to decide which blocks to promote.  If the hotspot queue is
99performing badly then it starts moving entries more quickly between
100levels.  This lets it adapt to new IO patterns very quickly.
101
102Performance
103^^^^^^^^^^^
104
105Testing smq shows substantially better performance than mq.
106
107cleaner
108-------
109
110The cleaner writes back all dirty blocks in a cache to decommission it.
111
112Examples
113========
114
115The syntax for a table is::
116
117	cache <metadata dev> <cache dev> <origin dev> <block size>
118	<#feature_args> [<feature arg>]*
119	<policy> <#policy_args> [<policy arg>]*
120
121The syntax to send a message using the dmsetup command is::
122
123	dmsetup message <mapped device> 0 sequential_threshold 1024
124	dmsetup message <mapped device> 0 random_threshold 8
125
126Using dmsetup::
127
128	dmsetup create blah --table "0 268435456 cache /dev/sdb /dev/sdc \
129	    /dev/sdd 512 0 mq 4 sequential_threshold 1024 random_threshold 8"
130	creates a 128GB large mapped device named 'blah' with the
131	sequential threshold set to 1024 and the random_threshold set to 8.
132