Max level shown:
12026-02-01T23:03:33.266ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::c]:44015
22026-02-01T23:03:33.266ZINFOcrucible-agent (datafile): region be5f346d-ec9b-4b79-98fd-4b7065643a38 state: Requested
32026-02-01T23:03:33.267ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-02-01T23:03:33.267ZINFOcrucible-agent (dropshot): request completed latency_us = 998 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44015 req_id = 7bf47b2c-e697-4d03-9d5d-8ee00a9a0bbd response_code = 200 uri = /crucible/0/regions
52026-02-01T23:03:33.276ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/be5f346d-ec9b-4b79-98fd-4b7065643a38
62026-02-01T23:03:33.276ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/be5f346d-ec9b-4b79-98fd-4b7065643a38
72026-02-01T23:03:33.334ZINFOcrucible-agent (worker): creating region Region { id: RegionId("be5f346d-ec9b-4b79-98fd-4b7065643a38"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/be5f346d-ec9b-4b79-98fd-4b7065643a38" region = be5f346d-ec9b-4b79-98fd-4b7065643a38
82026-02-01T23:03:33.334ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "be5f346d-ec9b-4b79-98fd-4b7065643a38" "--data" "/data/regions/be5f346d-ec9b-4b79-98fd-4b7065643a38" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = be5f346d-ec9b-4b79-98fd-4b7065643a38
92026-02-01T23:03:33.440ZINFOcrucible-agent (worker): region files created ok region = be5f346d-ec9b-4b79-98fd-4b7065643a38
102026-02-01T23:03:33.440ZINFOcrucible-agent (datafile): region be5f346d-ec9b-4b79-98fd-4b7065643a38 state: Requested -> Created
112026-02-01T23:03:33.440ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-02-01T23:03:33.441ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-be5f346d-ec9b-4b79-98fd-4b7065643a38
132026-02-01T23:03:33.449ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-be5f346d-ec9b-4b79-98fd-4b7065643a38
142026-02-01T23:03:33.474ZINFOcrucible-agent (worker): creating config property group
152026-02-01T23:03:33.476ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-be5f346d-ec9b-4b79-98fd-4b7065643a38
162026-02-01T23:03:33.477ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/be5f346d-ec9b-4b79-98fd-4b7065643a38
172026-02-01T23:03:33.477ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-02-01T23:03:33.477ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
192026-02-01T23:03:33.477ZINFOcrucible-agent (worker): commit
202026-02-01T23:03:33.480ZINFOcrucible-agent (worker): ok!
212026-02-01T23:03:33.495ZINFOcrucible-agent (worker): SMF ok!
222026-02-01T23:03:33.564ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44015 req_id = f38a8b13-9ed1-45ba-9a2f-80274848220a response_code = 200 uri = /crucible/0/regions
232026-02-01T23:03:34.749ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:44015 req_id = 0a05b9cb-b4ac-42da-aceb-582118524abc response_code = 200 uri = /crucible/0/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a
242026-02-01T23:03:34.752ZINFOcrucible-agent (datafile): path is "/data/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a"
252026-02-01T23:03:34.758ZINFOcrucible-agent (datafile): dataset is oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a
262026-02-01T23:03:34.765ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a@fffe5950-bbf8-4a60-bf30-27b5999669f5
272026-02-01T23:03:34.765ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a", "fffe5950-bbf8-4a60-bf30-27b5999669f5"]
282026-02-01T23:03:34.773ZINFOcrucible-agent (dropshot): request completed latency_us = 21303 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:44015 req_id = 4fc1b04d-8fb6-48dd-ad96-8dd0de7d8e75 response_code = 200 uri = /crucible/0/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a/snapshots/fffe5950-bbf8-4a60-bf30-27b5999669f5
292026-02-01T23:03:34.776ZINFOcrucible-agent (datafile): path is "/data/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a"
302026-02-01T23:03:34.783ZINFOcrucible-agent (datafile): dataset is oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a
312026-02-01T23:03:34.789ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a@fffe5950-bbf8-4a60-bf30-27b5999669f5
322026-02-01T23:03:34.790ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a", "fffe5950-bbf8-4a60-bf30-27b5999669f5"]
332026-02-01T23:03:34.796ZINFOcrucible-agent (datafile): requesting running snapshot f2957ecc-66d5-444b-a96f-67fdf2aece8a-fffe5950-bbf8-4a60-bf30-27b5999669f5 state: Requested
342026-02-01T23:03:34.797ZINFOcrucible-agent (worker): applying SMF actions for region f2957ecc-66d5-444b-a96f-67fdf2aece8a running snapshot fffe5950-bbf8-4a60-bf30-27b5999669f5 (state Requested)...
352026-02-01T23:03:34.797ZINFOcrucible-agent (dropshot): request completed latency_us = 20081 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44015 req_id = 7cd8fa63-0dc5-4ed3-8fa6-769b636ebc80 response_code = 200 uri = /crucible/0/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a/snapshots/fffe5950-bbf8-4a60-bf30-27b5999669f5/run
362026-02-01T23:03:34.801ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-f2957ecc-66d5-444b-a96f-67fdf2aece8a-fffe5950-bbf8-4a60-bf30-27b5999669f5
372026-02-01T23:03:34.813ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-f2957ecc-66d5-444b-a96f-67fdf2aece8a-fffe5950-bbf8-4a60-bf30-27b5999669f5
382026-02-01T23:03:34.823ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-f2957ecc-66d5-444b-a96f-67fdf2aece8a-fffe5950-bbf8-4a60-bf30-27b5999669f5
392026-02-01T23:03:34.823ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a/.zfs/snapshot/fffe5950-bbf8-4a60-bf30-27b5999669f5
402026-02-01T23:03:34.823ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-02-01T23:03:34.823ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-02-01T23:03:34.823ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
432026-02-01T23:03:34.823ZINFOcrucible-agent (worker): commit
442026-02-01T23:03:34.827ZINFOcrucible-agent (worker): ok!
452026-02-01T23:03:34.833ZINFOcrucible-agent (worker): SMF ok!
462026-02-01T23:03:34.833ZINFOcrucible-agent (datafile): region f2957ecc-66d5-444b-a96f-67fdf2aece8a running snapshot fffe5950-bbf8-4a60-bf30-27b5999669f5 state: Requested -> Created
472026-02-01T23:03:35.162ZINFOcrucible-agent (datafile): path is "/data/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a"
482026-02-01T23:03:35.169ZINFOcrucible-agent (datafile): dataset is oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a
492026-02-01T23:03:35.175ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a@fffe5950-bbf8-4a60-bf30-27b5999669f5
502026-02-01T23:03:35.175ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0b3bd04e-5552-448a-a1ea-badd13cdb296/crucible/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a", "fffe5950-bbf8-4a60-bf30-27b5999669f5"]
512026-02-01T23:03:35.182ZINFOcrucible-agent (dropshot): request completed latency_us = 19684 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44015 req_id = 314c9bbb-22dd-4a31-ab11-972e2eabb753 response_code = 200 uri = /crucible/0/regions/f2957ecc-66d5-444b-a96f-67fdf2aece8a/snapshots/fffe5950-bbf8-4a60-bf30-27b5999669f5/run