Max level shown:
12025-11-21T18:19:56.672ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::c]:53807
22025-11-21T18:19:56.672ZINFOcrucible-agent (datafile): region 659440ab-5346-4992-a810-45c680d7cd8f state: Requested
32025-11-21T18:19:56.674ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-11-21T18:19:56.674ZINFOcrucible-agent (dropshot): request completed latency_us = 1734 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:53807 req_id = ca0ae9ae-346d-4a0b-a82f-bac164fb3bec response_code = 200 uri = /crucible/0/regions
52025-11-21T18:19:56.686ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/659440ab-5346-4992-a810-45c680d7cd8f
62025-11-21T18:19:56.686ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/659440ab-5346-4992-a810-45c680d7cd8f
72025-11-21T18:19:56.757ZINFOcrucible-agent (worker): creating region Region { id: RegionId("659440ab-5346-4992-a810-45c680d7cd8f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/659440ab-5346-4992-a810-45c680d7cd8f" region = 659440ab-5346-4992-a810-45c680d7cd8f
82025-11-21T18:19:56.757ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "659440ab-5346-4992-a810-45c680d7cd8f" "--data" "/data/regions/659440ab-5346-4992-a810-45c680d7cd8f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 659440ab-5346-4992-a810-45c680d7cd8f
92025-11-21T18:19:56.941ZINFOcrucible-agent (worker): region files created ok region = 659440ab-5346-4992-a810-45c680d7cd8f
102025-11-21T18:19:56.942ZINFOcrucible-agent (datafile): region 659440ab-5346-4992-a810-45c680d7cd8f state: Requested -> Created
112025-11-21T18:19:56.942ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-11-21T18:19:56.942ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:53807 req_id = b68ec8e1-7829-4840-9f84-9e19f27de630 response_code = 200 uri = /crucible/0/regions
132025-11-21T18:19:56.964ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-659440ab-5346-4992-a810-45c680d7cd8f
142025-11-21T18:19:56.978ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-659440ab-5346-4992-a810-45c680d7cd8f
152025-11-21T18:19:57.005ZINFOcrucible-agent (worker): creating config property group
162025-11-21T18:19:57.011ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-659440ab-5346-4992-a810-45c680d7cd8f
172025-11-21T18:19:57.011ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/659440ab-5346-4992-a810-45c680d7cd8f
182025-11-21T18:19:57.011ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-11-21T18:19:57.011ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
202025-11-21T18:19:57.011ZINFOcrucible-agent (worker): commit
212025-11-21T18:19:57.020ZINFOcrucible-agent (worker): ok!
222025-11-21T18:19:57.028ZINFOcrucible-agent (worker): SMF ok!
232025-11-21T18:19:58.966ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:53807 req_id = 820c3993-4e95-4fee-9708-082244df7930 response_code = 200 uri = /crucible/0/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1
242025-11-21T18:19:58.969ZINFOcrucible-agent (datafile): path is "/data/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1"
252025-11-21T18:19:58.976ZINFOcrucible-agent (datafile): dataset is oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1
262025-11-21T18:19:58.985ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1@adb7f5b4-9085-41c0-aae9-58c567c85b63
272025-11-21T18:19:58.985ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1", "adb7f5b4-9085-41c0-aae9-58c567c85b63"]
282025-11-21T18:19:58.997ZINFOcrucible-agent (dropshot): request completed latency_us = 27847 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:53807 req_id = 5810fdf1-4761-456a-8dff-ffe1d14e0727 response_code = 200 uri = /crucible/0/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1/snapshots/adb7f5b4-9085-41c0-aae9-58c567c85b63
292025-11-21T18:19:59.001ZINFOcrucible-agent (datafile): path is "/data/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1"
302025-11-21T18:19:59.009ZINFOcrucible-agent (datafile): dataset is oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1
312025-11-21T18:19:59.016ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1@adb7f5b4-9085-41c0-aae9-58c567c85b63
322025-11-21T18:19:59.016ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1", "adb7f5b4-9085-41c0-aae9-58c567c85b63"]
332025-11-21T18:19:59.027ZINFOcrucible-agent (datafile): requesting running snapshot 1577d104-5dcc-4a98-94fc-a9196a08e1c1-adb7f5b4-9085-41c0-aae9-58c567c85b63 state: Requested
342025-11-21T18:19:59.028ZINFOcrucible-agent (dropshot): request completed latency_us = 26102 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:53807 req_id = aded5328-bac8-41c7-954b-2241523e4744 response_code = 200 uri = /crucible/0/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1/snapshots/adb7f5b4-9085-41c0-aae9-58c567c85b63/run
352025-11-21T18:19:59.028ZINFOcrucible-agent (worker): applying SMF actions for region 1577d104-5dcc-4a98-94fc-a9196a08e1c1 running snapshot adb7f5b4-9085-41c0-aae9-58c567c85b63 (state Requested)...
362025-11-21T18:19:59.035ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-1577d104-5dcc-4a98-94fc-a9196a08e1c1-adb7f5b4-9085-41c0-aae9-58c567c85b63
372025-11-21T18:19:59.038ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-1577d104-5dcc-4a98-94fc-a9196a08e1c1-adb7f5b4-9085-41c0-aae9-58c567c85b63
382025-11-21T18:19:59.050ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-1577d104-5dcc-4a98-94fc-a9196a08e1c1-adb7f5b4-9085-41c0-aae9-58c567c85b63
392025-11-21T18:19:59.050ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1/.zfs/snapshot/adb7f5b4-9085-41c0-aae9-58c567c85b63
402025-11-21T18:19:59.050ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-11-21T18:19:59.050ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-11-21T18:19:59.050ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432025-11-21T18:19:59.050ZINFOcrucible-agent (worker): commit
442025-11-21T18:19:59.055ZINFOcrucible-agent (worker): ok!
452025-11-21T18:19:59.060ZINFOcrucible-agent (worker): SMF ok!
462025-11-21T18:19:59.060ZINFOcrucible-agent (datafile): region 1577d104-5dcc-4a98-94fc-a9196a08e1c1 running snapshot adb7f5b4-9085-41c0-aae9-58c567c85b63 state: Requested -> Created
472025-11-21T18:19:59.350ZINFOcrucible-agent (datafile): path is "/data/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1"
482025-11-21T18:19:59.357ZINFOcrucible-agent (datafile): dataset is oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1
492025-11-21T18:19:59.363ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1@adb7f5b4-9085-41c0-aae9-58c567c85b63
502025-11-21T18:19:59.363ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f3cc082c-2bdb-40cb-9c49-ac104a39b635/crucible/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1", "adb7f5b4-9085-41c0-aae9-58c567c85b63"]
512025-11-21T18:19:59.369ZINFOcrucible-agent (dropshot): request completed latency_us = 19368 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:53807 req_id = f157b63d-e767-4752-9417-7713d6499981 response_code = 200 uri = /crucible/0/regions/1577d104-5dcc-4a98-94fc-a9196a08e1c1/snapshots/adb7f5b4-9085-41c0-aae9-58c567c85b63/run