Max level shown:
12025-02-11T20:45:27.189ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::c]:35088
22025-02-11T20:45:27.213ZINFOcrucible-agent (datafile): region fbca07da-d52e-4bd7-b0bb-e13df6976e8f state: Requested
32025-02-11T20:45:27.215ZINFOcrucible-agent (dropshot): request completed latency_us = 11142 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35088 req_id = a550d2fe-5bbd-4527-af5a-69218455a6d0 response_code = 200 uri = /crucible/0/regions
42025-02-11T20:45:27.218ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52025-02-11T20:45:27.227ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f
62025-02-11T20:45:27.227ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f
72025-02-11T20:45:27.388ZINFOcrucible-agent (worker): creating region Region { id: RegionId("fbca07da-d52e-4bd7-b0bb-e13df6976e8f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f" region = fbca07da-d52e-4bd7-b0bb-e13df6976e8f
82025-02-11T20:45:27.390ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "fbca07da-d52e-4bd7-b0bb-e13df6976e8f" "--data" "/data/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = fbca07da-d52e-4bd7-b0bb-e13df6976e8f
92025-02-11T20:45:27.562ZINFOcrucible-agent (dropshot): request completed latency_us = 78 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35088 req_id = 3c3a565f-8b4a-4787-a90c-500585a8a099 response_code = 200 uri = /crucible/0/regions
102025-02-11T20:45:27.681ZINFOcrucible-agent (worker): region files created ok region = fbca07da-d52e-4bd7-b0bb-e13df6976e8f
112025-02-11T20:45:27.681ZINFOcrucible-agent (datafile): region fbca07da-d52e-4bd7-b0bb-e13df6976e8f state: Requested -> Created
122025-02-11T20:45:27.681ZINFOcrucible-agent (worker): applying SMF actions post create...
132025-02-11T20:45:27.682ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-fbca07da-d52e-4bd7-b0bb-e13df6976e8f
142025-02-11T20:45:27.688ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-fbca07da-d52e-4bd7-b0bb-e13df6976e8f
152025-02-11T20:45:27.713ZINFOcrucible-agent (worker): creating config property group
162025-02-11T20:45:27.725ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-fbca07da-d52e-4bd7-b0bb-e13df6976e8f
172025-02-11T20:45:27.726ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f
182025-02-11T20:45:27.728ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192025-02-11T20:45:27.728ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
202025-02-11T20:45:27.728ZINFOcrucible-agent (worker): commit
212025-02-11T20:45:27.761ZINFOcrucible-agent (worker): ok!
222025-02-11T20:45:27.775ZINFOcrucible-agent (worker): SMF ok!
232025-02-11T20:45:27.917ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35088 req_id = 33e61447-33a8-4ca2-a0dc-be0cf6125424 response_code = 200 uri = /crucible/0/regions
242025-02-11T20:55:54.339ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::c]:62649
252025-02-11T20:55:54.339ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:62649 req_id = 7bda7873-2751-45a8-bd62-e55319fd41ac response_code = 200 uri = /crucible/0/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f
262025-02-11T20:55:54.348ZINFOcrucible-agent (datafile): path is "/data/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f"
272025-02-11T20:55:54.358ZINFOcrucible-agent (datafile): dataset is oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f
282025-02-11T20:55:54.367ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f@1728d5e2-46d0-499b-9fa3-43dc9f311787
292025-02-11T20:55:54.367ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f", "1728d5e2-46d0-499b-9fa3-43dc9f311787"]
302025-02-11T20:55:54.379ZINFOcrucible-agent (dropshot): request completed latency_us = 34635 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:62649 req_id = dfe5e808-7b94-4ebd-ad8f-6f73d6b7a5f4 response_code = 200 uri = /crucible/0/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f/snapshots/1728d5e2-46d0-499b-9fa3-43dc9f311787
312025-02-11T20:55:54.382ZINFOcrucible-agent (datafile): path is "/data/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f"
322025-02-11T20:55:54.390ZINFOcrucible-agent (datafile): dataset is oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f
332025-02-11T20:55:54.398ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f@1728d5e2-46d0-499b-9fa3-43dc9f311787
342025-02-11T20:55:54.398ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f", "1728d5e2-46d0-499b-9fa3-43dc9f311787"]
352025-02-11T20:55:54.408ZINFOcrucible-agent (datafile): requesting running snapshot fbca07da-d52e-4bd7-b0bb-e13df6976e8f-1728d5e2-46d0-499b-9fa3-43dc9f311787 state: Requested
362025-02-11T20:55:54.408ZINFOcrucible-agent (worker): applying SMF actions for region fbca07da-d52e-4bd7-b0bb-e13df6976e8f running snapshot 1728d5e2-46d0-499b-9fa3-43dc9f311787 (state Requested)...
372025-02-11T20:55:54.408ZINFOcrucible-agent (dropshot): request completed latency_us = 26070 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:62649 req_id = 534eaa52-fcfd-4941-aeea-9d33bcde7b9a response_code = 200 uri = /crucible/0/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f/snapshots/1728d5e2-46d0-499b-9fa3-43dc9f311787/run
382025-02-11T20:55:54.413ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f
392025-02-11T20:55:54.413ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
402025-02-11T20:55:54.413ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::12
412025-02-11T20:55:54.413ZINFOcrucible-agent (worker): reconfig required
422025-02-11T20:55:54.413ZINFOcrucible-agent (worker): using existing config property group
432025-02-11T20:55:54.413ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-fbca07da-d52e-4bd7-b0bb-e13df6976e8f
442025-02-11T20:55:54.413ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f
452025-02-11T20:55:54.413ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
462025-02-11T20:55:54.413ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
472025-02-11T20:55:54.413ZINFOcrucible-agent (worker): commit
482025-02-11T20:55:54.421ZINFOcrucible-agent (worker): ok!
492025-02-11T20:55:54.422ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-fbca07da-d52e-4bd7-b0bb-e13df6976e8f-1728d5e2-46d0-499b-9fa3-43dc9f311787
502025-02-11T20:55:54.424ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-fbca07da-d52e-4bd7-b0bb-e13df6976e8f-1728d5e2-46d0-499b-9fa3-43dc9f311787
512025-02-11T20:55:54.434ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-fbca07da-d52e-4bd7-b0bb-e13df6976e8f-1728d5e2-46d0-499b-9fa3-43dc9f311787
522025-02-11T20:55:54.434ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f/.zfs/snapshot/1728d5e2-46d0-499b-9fa3-43dc9f311787
532025-02-11T20:55:54.434ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
542025-02-11T20:55:54.434ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
552025-02-11T20:55:54.434ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
562025-02-11T20:55:54.434ZINFOcrucible-agent (worker): commit
572025-02-11T20:55:54.437ZINFOcrucible-agent (worker): ok!
582025-02-11T20:55:54.441ZINFOcrucible-agent (worker): SMF ok!
592025-02-11T20:55:54.441ZINFOcrucible-agent (datafile): region fbca07da-d52e-4bd7-b0bb-e13df6976e8f running snapshot 1728d5e2-46d0-499b-9fa3-43dc9f311787 state: Requested -> Created
602025-02-11T20:55:54.747ZINFOcrucible-agent (datafile): path is "/data/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f"
612025-02-11T20:55:54.756ZINFOcrucible-agent (datafile): dataset is oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f
622025-02-11T20:55:54.765ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f@1728d5e2-46d0-499b-9fa3-43dc9f311787
632025-02-11T20:55:54.765ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d5d0c478-01ca-45fb-b39c-0af3379fead2/crucible/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f", "1728d5e2-46d0-499b-9fa3-43dc9f311787"]
642025-02-11T20:55:54.774ZINFOcrucible-agent (dropshot): request completed latency_us = 26976 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:62649 req_id = 3543a7cd-1f4f-46aa-8064-9a9f9bcdd7ec response_code = 200 uri = /crucible/0/regions/fbca07da-d52e-4bd7-b0bb-e13df6976e8f/snapshots/1728d5e2-46d0-499b-9fa3-43dc9f311787/run