Max level shown:
12026-08-31T15:56:07.340ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::b]:40292
22026-08-31T15:56:07.340ZINFOcrucible-agent (datafile): region 4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f state: Requested
32026-08-31T15:56:07.341ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-08-31T15:56:07.341ZINFOcrucible-agent (dropshot): request completed latency_us = 1091 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:40292 req_id = 2cbe70ac-3a04-41ce-b5b7-cdee49f888af response_code = 200 uri = /crucible/0/regions
52026-08-31T15:56:07.351ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f
62026-08-31T15:56:07.351ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f
72026-08-31T15:56:07.391ZINFOcrucible-agent (worker): creating region Region { id: RegionId("4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f" region = 4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f
82026-08-31T15:56:07.391ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f" "--data" "/data/regions/4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f
92026-08-31T15:56:07.438ZINFOcrucible-agent (worker): region files created ok region = 4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f
102026-08-31T15:56:07.438ZINFOcrucible-agent (datafile): region 4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f state: Requested -> Created
112026-08-31T15:56:07.438ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-08-31T15:56:07.439ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f
132026-08-31T15:56:07.443ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f
142026-08-31T15:56:07.453ZINFOcrucible-agent (worker): creating config property group
152026-08-31T15:56:07.454ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f
162026-08-31T15:56:07.454ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4c1b4d6a-49c6-41ab-91cd-61d14ab63b2f
172026-08-31T15:56:07.454ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-08-31T15:56:07.454ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
192026-08-31T15:56:07.454ZINFOcrucible-agent (worker): commit
202026-08-31T15:56:07.456ZINFOcrucible-agent (worker): ok!
212026-08-31T15:56:07.460ZINFOcrucible-agent (worker): SMF ok!
222026-08-31T15:56:07.519ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:40292 req_id = 151c39e6-c3f3-4baa-9db7-3e19de81e620 response_code = 200 uri = /crucible/0/regions
232026-08-31T15:56:08.801ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:40292 req_id = f721ac85-6058-427f-9614-e80cf32dd064 response_code = 200 uri = /crucible/0/regions/b3baf7a9-57a0-409d-a818-8f9966c66526
242026-08-31T15:56:08.805ZINFOcrucible-agent (datafile): path is "/data/regions/b3baf7a9-57a0-409d-a818-8f9966c66526"
252026-08-31T15:56:08.811ZINFOcrucible-agent (datafile): dataset is oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/b3baf7a9-57a0-409d-a818-8f9966c66526
262026-08-31T15:56:08.818ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/b3baf7a9-57a0-409d-a818-8f9966c66526@8110ecd3-78e3-466a-af15-01ac65559a06
272026-08-31T15:56:08.818ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/b3baf7a9-57a0-409d-a818-8f9966c66526", "8110ecd3-78e3-466a-af15-01ac65559a06"]
282026-08-31T15:56:08.828ZINFOcrucible-agent (dropshot): request completed latency_us = 22674 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:40292 req_id = 12d5bc48-9aa9-46e1-94c3-70028f0b7363 response_code = 200 uri = /crucible/0/regions/b3baf7a9-57a0-409d-a818-8f9966c66526/snapshots/8110ecd3-78e3-466a-af15-01ac65559a06
292026-08-31T15:56:08.831ZINFOcrucible-agent (datafile): path is "/data/regions/b3baf7a9-57a0-409d-a818-8f9966c66526"
302026-08-31T15:56:08.838ZINFOcrucible-agent (datafile): dataset is oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/b3baf7a9-57a0-409d-a818-8f9966c66526
312026-08-31T15:56:08.844ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/b3baf7a9-57a0-409d-a818-8f9966c66526@8110ecd3-78e3-466a-af15-01ac65559a06
322026-08-31T15:56:08.844ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/b3baf7a9-57a0-409d-a818-8f9966c66526", "8110ecd3-78e3-466a-af15-01ac65559a06"]
332026-08-31T15:56:08.850ZINFOcrucible-agent (datafile): requesting running snapshot b3baf7a9-57a0-409d-a818-8f9966c66526-8110ecd3-78e3-466a-af15-01ac65559a06 state: Requested
342026-08-31T15:56:08.855ZINFOcrucible-agent (dropshot): request completed latency_us = 23655 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:40292 req_id = 32fbe6dd-f1de-4d06-b0b8-3299b382f8b8 response_code = 200 uri = /crucible/0/regions/b3baf7a9-57a0-409d-a818-8f9966c66526/snapshots/8110ecd3-78e3-466a-af15-01ac65559a06/run
352026-08-31T15:56:08.855ZINFOcrucible-agent (worker): applying SMF actions for region b3baf7a9-57a0-409d-a818-8f9966c66526 running snapshot 8110ecd3-78e3-466a-af15-01ac65559a06 (state Requested)...
362026-08-31T15:56:08.857ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-b3baf7a9-57a0-409d-a818-8f9966c66526-8110ecd3-78e3-466a-af15-01ac65559a06
372026-08-31T15:56:08.864ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-b3baf7a9-57a0-409d-a818-8f9966c66526-8110ecd3-78e3-466a-af15-01ac65559a06
382026-08-31T15:56:08.868ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-b3baf7a9-57a0-409d-a818-8f9966c66526-8110ecd3-78e3-466a-af15-01ac65559a06
392026-08-31T15:56:08.869ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b3baf7a9-57a0-409d-a818-8f9966c66526/.zfs/snapshot/8110ecd3-78e3-466a-af15-01ac65559a06
402026-08-31T15:56:08.869ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-08-31T15:56:08.869ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-08-31T15:56:08.869ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
432026-08-31T15:56:08.869ZINFOcrucible-agent (worker): commit
442026-08-31T15:56:08.871ZINFOcrucible-agent (worker): ok!
452026-08-31T15:56:08.874ZINFOcrucible-agent (worker): SMF ok!
462026-08-31T15:56:08.874ZINFOcrucible-agent (datafile): region b3baf7a9-57a0-409d-a818-8f9966c66526 running snapshot 8110ecd3-78e3-466a-af15-01ac65559a06 state: Requested -> Created
472026-08-31T15:56:09.203ZINFOcrucible-agent (datafile): path is "/data/regions/b3baf7a9-57a0-409d-a818-8f9966c66526"
482026-08-31T15:56:09.210ZINFOcrucible-agent (datafile): dataset is oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/b3baf7a9-57a0-409d-a818-8f9966c66526
492026-08-31T15:56:09.216ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/b3baf7a9-57a0-409d-a818-8f9966c66526@8110ecd3-78e3-466a-af15-01ac65559a06
502026-08-31T15:56:09.216ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e6b66b59-9b1b-474b-87bd-0044d4040bff/crucible/regions/b3baf7a9-57a0-409d-a818-8f9966c66526", "8110ecd3-78e3-466a-af15-01ac65559a06"]
512026-08-31T15:56:09.221ZINFOcrucible-agent (dropshot): request completed latency_us = 18385 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:40292 req_id = e795a0b7-d58c-45f8-9117-cbcea5d18d92 response_code = 200 uri = /crucible/0/regions/b3baf7a9-57a0-409d-a818-8f9966c66526/snapshots/8110ecd3-78e3-466a-af15-01ac65559a06/run