Max level shown:
12025-12-18T05:40:55.743ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::c]:55654
22025-12-18T05:40:55.743ZINFOcrucible-agent (datafile): region 7df77af3-0259-47e9-b3ec-6aefd9874c17 state: Requested
32025-12-18T05:40:55.744ZINFOcrucible-agent (dropshot): request completed latency_us = 1077 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:55654 req_id = 44fa28f9-f13c-441a-a774-6bd2a1fcc5ee response_code = 200 uri = /crucible/0/regions
42025-12-18T05:40:55.744ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52025-12-18T05:40:55.755ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/7df77af3-0259-47e9-b3ec-6aefd9874c17
62025-12-18T05:40:55.755ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/7df77af3-0259-47e9-b3ec-6aefd9874c17
72025-12-18T05:40:55.801ZINFOcrucible-agent (worker): creating region Region { id: RegionId("7df77af3-0259-47e9-b3ec-6aefd9874c17"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/7df77af3-0259-47e9-b3ec-6aefd9874c17" region = 7df77af3-0259-47e9-b3ec-6aefd9874c17
82025-12-18T05:40:55.801ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "7df77af3-0259-47e9-b3ec-6aefd9874c17" "--data" "/data/regions/7df77af3-0259-47e9-b3ec-6aefd9874c17" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 7df77af3-0259-47e9-b3ec-6aefd9874c17
92025-12-18T05:40:55.881ZINFOcrucible-agent (worker): region files created ok region = 7df77af3-0259-47e9-b3ec-6aefd9874c17
102025-12-18T05:40:55.882ZINFOcrucible-agent (datafile): region 7df77af3-0259-47e9-b3ec-6aefd9874c17 state: Requested -> Created
112025-12-18T05:40:55.882ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-12-18T05:40:55.882ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-7df77af3-0259-47e9-b3ec-6aefd9874c17
132025-12-18T05:40:56.003ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-7df77af3-0259-47e9-b3ec-6aefd9874c17
142025-12-18T05:40:56.114ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:55654 req_id = cc05a88f-6a4c-4a15-b447-21721610ffbf response_code = 200 uri = /crucible/0/regions
152025-12-18T05:40:56.217ZINFOcrucible-agent (worker): creating config property group
162025-12-18T05:40:56.289ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-7df77af3-0259-47e9-b3ec-6aefd9874c17
172025-12-18T05:40:56.289ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/7df77af3-0259-47e9-b3ec-6aefd9874c17
182025-12-18T05:40:56.289ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-12-18T05:40:56.289ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
202025-12-18T05:40:56.289ZINFOcrucible-agent (worker): commit
212025-12-18T05:40:56.473ZINFOcrucible-agent (worker): ok!
222025-12-18T05:40:56.873ZINFOcrucible-agent (worker): SMF ok!
232025-12-18T05:41:05.067ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:55654 req_id = c306c188-61ae-4b43-9fed-f0744994ee2f response_code = 200 uri = /crucible/0/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c
242025-12-18T05:41:05.070ZINFOcrucible-agent (datafile): path is "/data/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c"
252025-12-18T05:41:05.076ZINFOcrucible-agent (datafile): dataset is oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c
262025-12-18T05:41:05.083ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c@6ad72dfe-9953-404c-a213-c07607a5fe06
272025-12-18T05:41:05.083ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c", "6ad72dfe-9953-404c-a213-c07607a5fe06"]
282025-12-18T05:41:05.091ZINFOcrucible-agent (dropshot): request completed latency_us = 20785 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:55654 req_id = ce980fe3-515c-4b02-a69d-fe90b740540a response_code = 200 uri = /crucible/0/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c/snapshots/6ad72dfe-9953-404c-a213-c07607a5fe06
292025-12-18T05:41:05.095ZINFOcrucible-agent (datafile): path is "/data/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c"
302025-12-18T05:41:05.101ZINFOcrucible-agent (datafile): dataset is oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c
312025-12-18T05:41:05.107ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c@6ad72dfe-9953-404c-a213-c07607a5fe06
322025-12-18T05:41:05.107ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c", "6ad72dfe-9953-404c-a213-c07607a5fe06"]
332025-12-18T05:41:05.115ZINFOcrucible-agent (datafile): requesting running snapshot bbd0eb8f-b0da-415c-9f04-6544f9c6b71c-6ad72dfe-9953-404c-a213-c07607a5fe06 state: Requested
342025-12-18T05:41:05.115ZINFOcrucible-agent (dropshot): request completed latency_us = 19891 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:55654 req_id = ef0281b0-d409-4061-991e-9a8ca18d8af4 response_code = 200 uri = /crucible/0/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c/snapshots/6ad72dfe-9953-404c-a213-c07607a5fe06/run
352025-12-18T05:41:05.115ZINFOcrucible-agent (worker): applying SMF actions for region bbd0eb8f-b0da-415c-9f04-6544f9c6b71c running snapshot 6ad72dfe-9953-404c-a213-c07607a5fe06 (state Requested)...
362025-12-18T05:41:05.119ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-bbd0eb8f-b0da-415c-9f04-6544f9c6b71c-6ad72dfe-9953-404c-a213-c07607a5fe06
372025-12-18T05:41:05.129ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-bbd0eb8f-b0da-415c-9f04-6544f9c6b71c-6ad72dfe-9953-404c-a213-c07607a5fe06
382025-12-18T05:41:05.139ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-bbd0eb8f-b0da-415c-9f04-6544f9c6b71c-6ad72dfe-9953-404c-a213-c07607a5fe06
392025-12-18T05:41:05.140ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c/.zfs/snapshot/6ad72dfe-9953-404c-a213-c07607a5fe06
402025-12-18T05:41:05.140ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-12-18T05:41:05.140ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-12-18T05:41:05.140ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432025-12-18T05:41:05.140ZINFOcrucible-agent (worker): commit
442025-12-18T05:41:05.144ZINFOcrucible-agent (worker): ok!
452025-12-18T05:41:05.148ZINFOcrucible-agent (worker): SMF ok!
462025-12-18T05:41:05.148ZINFOcrucible-agent (datafile): region bbd0eb8f-b0da-415c-9f04-6544f9c6b71c running snapshot 6ad72dfe-9953-404c-a213-c07607a5fe06 state: Requested -> Created
472025-12-18T05:41:05.327ZINFOcrucible-agent (datafile): path is "/data/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c"
482025-12-18T05:41:05.333ZINFOcrucible-agent (datafile): dataset is oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c
492025-12-18T05:41:05.339ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c@6ad72dfe-9953-404c-a213-c07607a5fe06
502025-12-18T05:41:05.339ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7f54cd20-4093-4e05-ba2f-3214ce50ba92/crucible/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c", "6ad72dfe-9953-404c-a213-c07607a5fe06"]
512025-12-18T05:41:05.344ZINFOcrucible-agent (dropshot): request completed latency_us = 17303 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:55654 req_id = 8da32cc0-9fff-4f88-9354-a4aa0482e4d9 response_code = 200 uri = /crucible/0/regions/bbd0eb8f-b0da-415c-9f04-6544f9c6b71c/snapshots/6ad72dfe-9953-404c-a213-c07607a5fe06/run