Max level shown:
12024-08-06T04:30:01.048ZINFOcrucible-pantry (dropshot): request completed latency_us = 3035 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = b0d0fdb3-f964-4268-8138-ea86e26bbf18 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
22024-08-06T04:30:01.158ZINFOcrucible-pantry (dropshot): request completed latency_us = 2942 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 752bfd06-d72a-4bbc-8dd6-0cc64e291b1e response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
32024-08-06T04:30:01.270ZINFOcrucible-pantry (dropshot): request completed latency_us = 2619 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = d13f9431-1de0-4c42-985a-384e00dff000 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
42024-08-06T04:30:01.751ZINFOcrucible-pantry (dropshot): request completed latency_us = 2985 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 7c036580-d4c5-460a-944d-8bc9eae7f494 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
52024-08-06T04:30:02.248ZINFOcrucible-pantry (dropshot): request completed latency_us = 2507 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 9d02d4f5-4bd6-4abf-a73f-768ca155f6a8 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
62024-08-06T04:30:02.340ZINFOcrucible-pantry (dropshot): request completed latency_us = 2709 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 62f8bda0-6677-47aa-9a33-bb1693986872 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
72024-08-06T04:30:02.353ZINFOcrucible-pantry (dropshot): request completed latency_us = 2752 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 457404f8-617a-4af8-afd8-572846c33051 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
82024-08-06T04:30:03.018ZINFOcrucible-pantry (dropshot): request completed latency_us = 2559 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = a5761df8-cfd5-464f-a7cd-0cebfa82777d response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
92024-08-06T04:30:03.116ZINFOcrucible-pantry (dropshot): request completed latency_us = 3028 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 681ea857-7923-4556-9022-9cd6c2a4f816 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
102024-08-06T04:30:04.028ZINFOcrucible-pantry (dropshot): request completed latency_us = 3049 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = af2f2738-622a-4b6d-a705-e6e1ebbe802e response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
112024-08-06T04:30:04.174ZINFOcrucible-pantry (dropshot): request completed latency_us = 2938 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 75e48bf4-0a6d-4f10-ad6b-b25a7e8764ab response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
122024-08-06T04:30:04.407ZINFOcrucible-pantry (dropshot): request completed latency_us = 2899 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = e338ddf4-d3d8-4cf3-8dea-08d89b6a771b response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
132024-08-06T04:30:05.081ZINFOcrucible-pantry (dropshot): request completed latency_us = 2624 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 366f9a87-c024-41d4-ad95-c00742bbba22 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
142024-08-06T04:30:05.181ZINFOcrucible-pantry (dropshot): request completed latency_us = 3261 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 975578ca-58a1-4caf-9acf-ff5fcea5ef8c response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
152024-08-06T04:30:05.312ZINFOcrucible-pantry (dropshot): request completed latency_us = 2727 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 1e6bcc9a-7b04-4719-8076-ffb8983d5d51 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
162024-08-06T04:30:05.685ZINFOcrucible-pantry (dropshot): request completed latency_us = 2909 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = c233b733-229c-46b4-a94c-b7fd4713e7a9 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
172024-08-06T04:30:06.588ZINFOcrucible-pantry (dropshot): request completed latency_us = 3844 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = e4ad2f9b-8877-4a37-8cde-c51c946f23bf response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
182024-08-06T04:30:07.415ZINFOcrucible-pantry (dropshot): request completed latency_us = 3105 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = ba66ad6e-550b-401c-9c41-69d22f3e6211 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
192024-08-06T04:30:08.676ZINFOcrucible-pantry (dropshot): request completed latency_us = 2652 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 60684c96-2105-4d52-b402-9ba9193844fa response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
202024-08-06T04:30:09.503ZINFOcrucible-pantry (dropshot): request completed latency_us = 3027 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = fda4a7c9-b4fb-4649-8013-b238f0b95891 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
212024-08-06T04:30:09.958ZINFOcrucible-pantry (dropshot): request completed latency_us = 2769 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 10767fa5-a585-48f4-bc24-ac68a9735315 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
222024-08-06T04:30:11.642ZINFOcrucible-pantry (dropshot): request completed latency_us = 2874 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:33502 req_id = 638d8617-6660-47c4-a30a-b7630534ec0b response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/bulk-write
232024-08-06T04:30:13.006ZINFOcrucible-pantry (dropshot): accepted connection local_addr = [fd00:1122:3344:101::f]:17000 remote_addr = [fd00:1122:3344:101::a]:62271
242024-08-06T04:30:13.006ZINFOcrucible-pantry (datafile): flush with snap requested session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
252024-08-06T04:30:13.058ZINFOcrucible-pantry (dropshot): request completed latency_us = 51329 local_addr = [fd00:1122:3344:101::f]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:62271 req_id = 0ea0fc0b-e91b-4fcd-b5df-257afa1c838b response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2/snapshot
262024-08-06T04:30:13.643ZINFOcrucible-pantry (dropshot): accepted connection local_addr = [fd00:1122:3344:101::f]:17000 remote_addr = [fd00:1122:3344:101::a]:53040
272024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): detach removing entry for volume a308e702-0683-492c-a35c-6cbe8b0de4a2
282024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): detaching volume a308e702-0683-492c-a35c-6cbe8b0de4a2
292024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): Request to deactivate this guest session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
302024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): checking for deactivation session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
312024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): [0] deactivate, no work so YES = downstairs session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
322024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): ds_transition from Active to Deactivated = downstairs client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
332024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): [0] Transition from Active to Deactivated = downstairs client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
342024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): deactivated client 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
352024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): [1] deactivate, no work so YES = downstairs session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
362024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): ds_transition from Active to Deactivated = downstairs client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
372024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): [1] Transition from Active to Deactivated = downstairs client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
382024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): deactivated client 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
392024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): [2] deactivate, no work so YES = downstairs session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
402024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): ds_transition from Active to Deactivated = downstairs client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
412024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): [2] Transition from Active to Deactivated = downstairs client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
422024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): deactivated client 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
432024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): not ready to deactivate due to state Deactivated = downstairs client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
442024-08-06T04:30:13.643ZWARNcrucible-pantry (datafile): client task is sending Done(RequestedStop(Deactivated)) = io task client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
452024-08-06T04:30:13.643ZWARNcrucible-pantry (datafile): downstairs task for 2 stopped due to RequestedStop(Deactivated) session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
462024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): Gone missing, transition from Deactivated to Disconnected = downstairs client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
472024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): Disconnected -> New = downstairs client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
482024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): checking for deactivation session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
492024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): not ready to deactivate due to state Deactivated = downstairs client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
502024-08-06T04:30:13.643ZWARNcrucible-pantry (datafile): client task is sending Done(RequestedStop(Deactivated)) = io task client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
512024-08-06T04:30:13.643ZWARNcrucible-pantry (datafile): downstairs task for 0 stopped due to RequestedStop(Deactivated) session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
522024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): Gone missing, transition from Deactivated to Disconnected = downstairs client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
532024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): Disconnected -> New = downstairs client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
542024-08-06T04:30:13.643ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
552024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): checking for deactivation session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
562024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): ready to deactivate from state New = downstairs client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
572024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): not ready to deactivate due to state Deactivated = downstairs client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
582024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): client task is sending Done(RequestedStop(Deactivated)) = io task client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
592024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
602024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): downstairs task for 1 stopped due to RequestedStop(Deactivated) session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
612024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): Gone missing, transition from Deactivated to Disconnected = downstairs client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
622024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): Disconnected -> New = downstairs client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
632024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): checking for deactivation session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
642024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): ready to deactivate from state New = downstairs client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
652024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): ready to deactivate from state New = downstairs client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
662024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): ready to deactivate from state New = downstairs client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
672024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): All DS in the proper state! -> INIT session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
682024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
692024-08-06T04:30:13.646ZINFOcrucible-pantry (dropshot): request completed latency_us = 265 local_addr = [fd00:1122:3344:101::f]:17000 method = DELETE remote_addr = [fd00:1122:3344:101::a]:53040 req_id = 4fe78dff-21b7-46b6-8b23-7da1446e5bf8 response_code = 204 uri = /crucible/pantry/0/volume/a308e702-0683-492c-a35c-6cbe8b0de4a2
702024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): Guest handle has been dropped
712024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): client IO task stopped during sleep = io task client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
722024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): client_stop_rx closed unexpectedly: RecvError(()) = io task client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
732024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): client task is sending Done(QueueClosed) = io task client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
742024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): client task could not reply to main task; shutting down? = io task client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
752024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 0 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
762024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): client IO task stopped during sleep = io task client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
772024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): client_stop_rx closed unexpectedly: RecvError(()) = io task client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
782024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): client task is sending Done(QueueClosed) = io task client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
792024-08-06T04:30:13.646ZWARNcrucible-pantry (datafile): client task could not reply to main task; shutting down? = io task client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
802024-08-06T04:30:13.646ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 1 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
812024-08-06T04:30:13.649ZWARNcrucible-pantry (datafile): client IO task stopped during sleep = io task client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
822024-08-06T04:30:13.649ZWARNcrucible-pantry (datafile): client_stop_rx closed unexpectedly: RecvError(()) = io task client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
832024-08-06T04:30:13.649ZWARNcrucible-pantry (datafile): client task is sending Done(QueueClosed) = io task client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
842024-08-06T04:30:13.649ZWARNcrucible-pantry (datafile): client task could not reply to main task; shutting down? = io task client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73
852024-08-06T04:30:13.649ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 2 session_id = 6e3792f1-3057-487d-8cfb-c017f4e57e73