Max level shown:
12024-07-26T02:45:01.159ZINFOcrucible-pantry (dropshot): request completed latency_us = 2887 local_addr = [fd00:1122:3344:101::11]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:51126 req_id = fd75c83d-ac8b-4407-bf3e-027a2cf8e82c response_code = 204 uri = /crucible/pantry/0/volume/2ffe1f80-f3b4-473e-960d-bdf57a25f014/bulk-write
22024-07-26T02:45:02.863ZINFOcrucible-pantry (dropshot): request completed latency_us = 2712 local_addr = [fd00:1122:3344:101::11]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:51126 req_id = 3de577d9-8423-499b-8bc8-c194bf4ffe1b response_code = 204 uri = /crucible/pantry/0/volume/2ffe1f80-f3b4-473e-960d-bdf57a25f014/bulk-write
32024-07-26T02:45:04.563ZINFOcrucible-pantry (dropshot): request completed latency_us = 2698 local_addr = [fd00:1122:3344:101::11]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:51126 req_id = f5d438ce-d843-4fef-8156-23eca0d4f154 response_code = 204 uri = /crucible/pantry/0/volume/2ffe1f80-f3b4-473e-960d-bdf57a25f014/bulk-write
42024-07-26T02:45:04.660ZINFOcrucible-pantry (dropshot): request completed latency_us = 2669 local_addr = [fd00:1122:3344:101::11]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:51126 req_id = 54a3d2ba-4d1d-4b2c-9306-6730cb9c184c response_code = 204 uri = /crucible/pantry/0/volume/2ffe1f80-f3b4-473e-960d-bdf57a25f014/bulk-write
52024-07-26T02:45:04.761ZINFOcrucible-pantry (dropshot): request completed latency_us = 2489 local_addr = [fd00:1122:3344:101::11]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:51126 req_id = a3b009b1-f241-4474-b102-1b5d758b264c response_code = 204 uri = /crucible/pantry/0/volume/2ffe1f80-f3b4-473e-960d-bdf57a25f014/bulk-write
62024-07-26T02:45:04.859ZINFOcrucible-pantry (dropshot): request completed latency_us = 2757 local_addr = [fd00:1122:3344:101::11]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:51126 req_id = 4bf5ca28-def7-4a0f-aef6-85574ebc8699 response_code = 204 uri = /crucible/pantry/0/volume/2ffe1f80-f3b4-473e-960d-bdf57a25f014/bulk-write
72024-07-26T02:45:04.958ZINFOcrucible-pantry (dropshot): request completed latency_us = 3382 local_addr = [fd00:1122:3344:101::11]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:51126 req_id = 108ff88e-21ad-4c31-934f-7f345a4f0d08 response_code = 204 uri = /crucible/pantry/0/volume/2ffe1f80-f3b4-473e-960d-bdf57a25f014/bulk-write
82024-07-26T02:45:05.058ZINFOcrucible-pantry (dropshot): request completed latency_us = 3187 local_addr = [fd00:1122:3344:101::11]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:51126 req_id = 91985d86-6c31-4a5a-a86a-b8bfbbbcaf0b response_code = 204 uri = /crucible/pantry/0/volume/2ffe1f80-f3b4-473e-960d-bdf57a25f014/bulk-write
92024-07-26T02:45:06.668ZINFOcrucible-pantry (dropshot): accepted connection local_addr = [fd00:1122:3344:101::11]:17000 remote_addr = [fd00:1122:3344:101::a]:50493
102024-07-26T02:45:06.668ZINFOcrucible-pantry (datafile): flush with snap requested session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
112024-07-26T02:45:06.724ZINFOcrucible-pantry (dropshot): request completed latency_us = 55822 local_addr = [fd00:1122:3344:101::11]:17000 method = POST remote_addr = [fd00:1122:3344:101::a]:50493 req_id = 87b8dbd1-9a43-4dcb-9836-4a586ba22d6d response_code = 204 uri = /crucible/pantry/0/volume/2ffe1f80-f3b4-473e-960d-bdf57a25f014/snapshot
122024-07-26T02:45:07.370ZINFOcrucible-pantry (dropshot): accepted connection local_addr = [fd00:1122:3344:101::11]:17000 remote_addr = [fd00:1122:3344:101::a]:49706
132024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): detach removing entry for volume 2ffe1f80-f3b4-473e-960d-bdf57a25f014
142024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): detaching volume 2ffe1f80-f3b4-473e-960d-bdf57a25f014
152024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): Request to deactivate this guest session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
162024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): checking for deactivation session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
172024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): [0] deactivate, no work so YES = downstairs session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
182024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): ds_transition from Active to Deactivated = downstairs client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
192024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): [0] Transition from Active to Deactivated = downstairs client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
202024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): deactivated client 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
212024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): [1] deactivate, no work so YES = downstairs session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
222024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): ds_transition from Active to Deactivated = downstairs client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
232024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): [1] Transition from Active to Deactivated = downstairs client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
242024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): deactivated client 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
252024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): [2] deactivate, no work so YES = downstairs session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
262024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): ds_transition from Active to Deactivated = downstairs client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
272024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): [2] Transition from Active to Deactivated = downstairs client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
282024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): deactivated client 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
292024-07-26T02:45:07.373ZINFOcrucible-pantry (datafile): not ready to deactivate due to state Deactivated = downstairs client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
302024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client task is sending Done(RequestedStop(Deactivated)) = io task client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
312024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client task is sending Done(RequestedStop(Deactivated)) = io task client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
322024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client task is sending Done(RequestedStop(Deactivated)) = io task client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
332024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): downstairs task for 0 stopped due to RequestedStop(Deactivated) session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
342024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): Gone missing, transition from Deactivated to Disconnected = downstairs client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
352024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): Disconnected -> New = downstairs client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
362024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): checking for deactivation session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
372024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): ready to deactivate from state New = downstairs client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
382024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): not ready to deactivate due to state Deactivated = downstairs client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
392024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): downstairs task for 1 stopped due to RequestedStop(Deactivated) session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
402024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): Gone missing, transition from Deactivated to Disconnected = downstairs client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
412024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): Disconnected -> New = downstairs client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
422024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): checking for deactivation session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
432024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): ready to deactivate from state New = downstairs client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
442024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): ready to deactivate from state New = downstairs client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
452024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): not ready to deactivate due to state Deactivated = downstairs client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
462024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): downstairs task for 2 stopped due to RequestedStop(Deactivated) session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
472024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): Gone missing, transition from Deactivated to Disconnected = downstairs client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
482024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): Disconnected -> New = downstairs client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
492024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
502024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): checking for deactivation session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
512024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): ready to deactivate from state New = downstairs client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
522024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): ready to deactivate from state New = downstairs client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
532024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): ready to deactivate from state New = downstairs client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
542024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): All DS in the proper state! -> INIT session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
552024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
562024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
572024-07-26T02:45:07.376ZINFOcrucible-pantry (dropshot): request completed latency_us = 5386 local_addr = [fd00:1122:3344:101::11]:17000 method = DELETE remote_addr = [fd00:1122:3344:101::a]:49706 req_id = 927ba8c4-a506-44b5-89d7-24210099d27f response_code = 204 uri = /crucible/pantry/0/volume/2ffe1f80-f3b4-473e-960d-bdf57a25f014
582024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): Guest handle has been dropped
592024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client IO task stopped during sleep = io task client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
602024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client_stop_rx closed unexpectedly: RecvError(()) = io task client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
612024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client task is sending Done(QueueClosed) = io task client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
622024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client task could not reply to main task; shutting down? = io task client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
632024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 1 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
642024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client IO task stopped during sleep = io task client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
652024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client_stop_rx closed unexpectedly: RecvError(()) = io task client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
662024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client task is sending Done(QueueClosed) = io task client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
672024-07-26T02:45:07.376ZWARNcrucible-pantry (datafile): client task could not reply to main task; shutting down? = io task client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
682024-07-26T02:45:07.376ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 0 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
692024-07-26T02:45:07.384ZWARNcrucible-pantry (datafile): client IO task stopped during sleep = io task client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
702024-07-26T02:45:07.384ZWARNcrucible-pantry (datafile): client_stop_rx closed unexpectedly: RecvError(()) = io task client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
712024-07-26T02:45:07.384ZWARNcrucible-pantry (datafile): client task is sending Done(QueueClosed) = io task client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
722024-07-26T02:45:07.384ZWARNcrucible-pantry (datafile): client task could not reply to main task; shutting down? = io task client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246
732024-07-26T02:45:07.384ZINFOcrucible-pantry (datafile): client task is exiting = io task client = 2 session_id = f2e1ec53-8a47-4fc1-86e9-d81fd526f246