Support reliable, cross-host, cross-process backtraces

[djs55]

See https://github.com/djs55/xen-api/blob/experimental-exceptions/docs/design/backtraces.md

This needs to be simultaneously merged with:

• [Use Debug.with_thread_named rather than the manual functions. xen-api-libs-transitional#14]
• [Support reliable, cross-host, cross-process backtraces stdext#9]
• [Support reliable, cross-host, cross-process backtraces xenopsd#147]
• [Support reliable, cross-host, cross-process backtraces xen-api#1999]
• [Update to use new API xcp-rrdd#24]
• [Update to new Debug.with_thread_associated API jonludlam/vhdd#7]
• xen-dist-ocaml.hg: not on github (yet)

NB this removes the manual 'name_thread' functions which were leaking in a few places. Please now use Debug.with_thread_named.

[djs55]

Example of backtrace you now get:

$ sudo xe vm-start vm=fedora-qcow2 --trace
Error code: SCRIPT_FAILED
Error parameters: /usr/lib/xapi-storage-script/datapath/file/Datapath.attach, non-zero exit, 1,
Raised Server_error(SCRIPT_FAILED, [ /usr/lib/xapi-storage-script/datapath/file/Datapath.attach; non-zero exit; 1;  ])
Backtrace:
0/50 Called from file /usr/lib/xapi-storage-script/datapath/file/Datapath.attach, line 14
1/50 Called from file /usr/lib/python2.7/dist-packages/xapi/d.py, line 33
2/50 Raised at file /usr/lib/xapi-storage-script/datapath/file/Datapath.attach, line 32
3/50 xenopsd-xenlight @ st30 Raised by primitive operation at file "lib/storage.ml", line 32, characters 3-26
4/50 xenopsd-xenlight @ st30 Called from file "lib/task_server.ml", line 176, characters 15-19
5/50 xenopsd-xenlight @ st30 Raised at file "lib/task_server.ml", line 184, characters 8-9
6/50 xenopsd-xenlight @ st30 Called from file "lib/storage.ml", line 54, characters 2-165
7/50 xenopsd-xenlight @ st30 Called from file "xl/xenops_server_xenlight.ml", line 341, characters 15-63
8/50 xenopsd-xenlight @ st30 Called from file "xl/xenops_server_xenlight.ml", line 947, characters 12-78
9/50 xenopsd-xenlight @ st30 Called from file "xl/xenops_server_xenlight.ml", line 2085, characters 11-45
10/50 xenopsd-xenlight @ st30 Called from file "list.ml", line 55, characters 20-23
11/50 xenopsd-xenlight @ st30 Called from file "xl/xenops_server_xenlight.ml", line 2084, characters 40-230
12/50 xenopsd-xenlight @ st30 Called from file "lib/xenops_utils.ml", line 80, characters 15-19
13/50 xenopsd-xenlight @ st30 Raised at file "lib/xenops_utils.ml", line 86, characters 8-9
14/50 xenopsd-xenlight @ st30 Called from file "lib/xenctrl.ml", line 127, characters 13-17
15/50 xenopsd-xenlight @ st30 Re-raised at file "lib/xenctrl.ml", line 127, characters 56-59
16/50 xenopsd-xenlight @ st30 Called from file "lib/xenops_utils.ml", line 80, characters 15-19
17/50 xenopsd-xenlight @ st30 Raised at file "lib/xenops_utils.ml", line 86, characters 8-9
18/50 xenopsd-xenlight @ st30 Called from file "lib/xenops_server.ml", line 1103, characters 4-71
19/50 xenopsd-xenlight @ st30 Called from file "list.ml", line 84, characters 24-34
20/50 xenopsd-xenlight @ st30 Called from file "lib/xenops_server.ml", line 1098, characters 2-367
21/50 xenopsd-xenlight @ st30 Called from file "lib/xenops_server.ml", line 1203, characters 3-46
22/50 xenopsd-xenlight @ st30 Called from file "lib/xenops_server.ml", line 1441, characters 3-9
23/50 xenopsd-xenlight @ st30 Raised at file "lib/xenops_server.ml", line 1452, characters 9-10
24/50 xenopsd-xenlight @ st30 Called from file "lib/xenops_server.ml", line 1458, characters 48-60
25/50 xenopsd-xenlight @ st30 Called from file "lib/task_server.ml", line 151, characters 15-26
26/50 xapi @ st30 Raised at file "xapi_xenops.ml", line 1713, characters 11-14
27/50 xapi @ st30 Called from file "lib/pervasiveext.ml", line 22, characters 3-9
28/50 xapi @ st30 Raised at file "xapi_xenops.ml", line 1997, characters 13-14
29/50 xapi @ st30 Called from file "lib/pervasiveext.ml", line 22, characters 3-9
30/50 xapi @ st30 Raised at file "xapi_xenops.ml", line 1779, characters 15-16
31/50 xapi @ st30 Called from file "message_forwarding.ml", line 233, characters 25-44
32/50 xapi @ st30 Called from file "message_forwarding.ml", line 915, characters 15-67
33/50 xapi @ st30 Called from file "lib/pervasiveext.ml", line 22, characters 3-9
34/50 xapi @ st30 Raised at file "lib/pervasiveext.ml", line 26, characters 9-12
35/50 xapi @ st30 Called from file "message_forwarding.ml", line 1205, characters 21-199
36/50 xapi @ st30 Called from file "lib/pervasiveext.ml", line 22, characters 3-9
37/50 xapi @ st30 Raised at file "lib/pervasiveext.ml", line 26, characters 9-12
38/50 xapi @ st30 Called from file "lib/pervasiveext.ml", line 22, characters 3-9
39/50 xapi @ st30 Raised at file "rbac.ml", line 236, characters 10-15
40/50 xapi @ st30 Called from file "server_helpers.ml", line 75, characters 11-41
41/50 xapi @ st30 Raised at file "cli_util.ml", line 78, characters 9-12
42/50 xapi @ st30 Called from file "lib/pervasiveext.ml", line 22, characters 3-9
43/50 xapi @ st30 Raised at file "lib/pervasiveext.ml", line 26, characters 9-12
44/50 xapi @ st30 Called from file "cli_operations.ml", line 1889, characters 2-6
45/50 xapi @ st30 Re-raised at file "cli_operations.ml", line 1898, characters 10-11
46/50 xapi @ st30 Called from file "cli_operations.ml", line 1821, characters 14-18
47/50 xapi @ st30 Called from file "cli_operations.ml", line 2109, characters 7-526
48/50 xapi @ st30 Called from file "xapi_cli.ml", line 113, characters 18-56
49/50 xapi @ st30 Called from file "lib/pervasiveext.ml", line 22, characters 3-9

[johnelse]

How exciting :)

[djs55]

OK to merge after chat with @jonludlam. We do need to think more about storage interface backward compatibility though