Max level shown:
12026-01-30T16:46:31.117ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::b]:62725
22026-01-30T16:46:31.117ZINFOcrucible-agent (datafile): region 5f425de8-0fad-44a2-91d7-72835038a7dd state: Requested
32026-01-30T16:46:31.118ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-01-30T16:46:31.118ZINFOcrucible-agent (dropshot): request completed latency_us = 1331 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:62725 req_id = fdc33708-4ec3-46b9-a5ec-9c5547a7bec8 response_code = 200 uri = /crucible/0/regions
52026-01-30T16:46:31.127ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/5f425de8-0fad-44a2-91d7-72835038a7dd
62026-01-30T16:46:31.127ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/5f425de8-0fad-44a2-91d7-72835038a7dd
72026-01-30T16:46:31.183ZINFOcrucible-agent (worker): creating region Region { id: RegionId("5f425de8-0fad-44a2-91d7-72835038a7dd"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/5f425de8-0fad-44a2-91d7-72835038a7dd" region = 5f425de8-0fad-44a2-91d7-72835038a7dd
82026-01-30T16:46:31.183ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "5f425de8-0fad-44a2-91d7-72835038a7dd" "--data" "/data/regions/5f425de8-0fad-44a2-91d7-72835038a7dd" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 5f425de8-0fad-44a2-91d7-72835038a7dd
92026-01-30T16:46:31.270ZINFOcrucible-agent (worker): region files created ok region = 5f425de8-0fad-44a2-91d7-72835038a7dd
102026-01-30T16:46:31.270ZINFOcrucible-agent (datafile): region 5f425de8-0fad-44a2-91d7-72835038a7dd state: Requested -> Created
112026-01-30T16:46:31.270ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-01-30T16:46:31.271ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-5f425de8-0fad-44a2-91d7-72835038a7dd
132026-01-30T16:46:31.280ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-5f425de8-0fad-44a2-91d7-72835038a7dd
142026-01-30T16:46:31.293ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:62725 req_id = 7285a613-57e1-49a1-9c80-4cc05c64dd2a response_code = 200 uri = /crucible/0/regions
152026-01-30T16:46:31.305ZINFOcrucible-agent (worker): creating config property group
162026-01-30T16:46:31.308ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-5f425de8-0fad-44a2-91d7-72835038a7dd
172026-01-30T16:46:31.308ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5f425de8-0fad-44a2-91d7-72835038a7dd
182026-01-30T16:46:31.308ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192026-01-30T16:46:31.309ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
202026-01-30T16:46:31.309ZINFOcrucible-agent (worker): commit
212026-01-30T16:46:31.312ZINFOcrucible-agent (worker): ok!
222026-01-30T16:46:31.329ZINFOcrucible-agent (worker): SMF ok!
232026-01-30T16:46:33.231ZINFOcrucible-agent (dropshot): request completed latency_us = 84 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:62725 req_id = 02f6b993-3289-4379-a034-e33835e48b88 response_code = 200 uri = /crucible/0/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f
242026-01-30T16:46:33.235ZINFOcrucible-agent (datafile): path is "/data/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f"
252026-01-30T16:46:33.244ZINFOcrucible-agent (datafile): dataset is oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f
262026-01-30T16:46:33.251ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f@b37c9f1e-c473-4967-bb6b-8d1939804129
272026-01-30T16:46:33.251ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f", "b37c9f1e-c473-4967-bb6b-8d1939804129"]
282026-01-30T16:46:33.263ZINFOcrucible-agent (dropshot): request completed latency_us = 28467 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:62725 req_id = c9346245-15b9-4ab0-b595-48e2127ff205 response_code = 200 uri = /crucible/0/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f/snapshots/b37c9f1e-c473-4967-bb6b-8d1939804129
292026-01-30T16:46:33.268ZINFOcrucible-agent (datafile): path is "/data/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f"
302026-01-30T16:46:33.275ZINFOcrucible-agent (datafile): dataset is oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f
312026-01-30T16:46:33.281ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f@b37c9f1e-c473-4967-bb6b-8d1939804129
322026-01-30T16:46:33.281ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f", "b37c9f1e-c473-4967-bb6b-8d1939804129"]
332026-01-30T16:46:33.290ZINFOcrucible-agent (datafile): requesting running snapshot ac8ae3f9-96f5-4fe4-bc32-7976ff62899f-b37c9f1e-c473-4967-bb6b-8d1939804129 state: Requested
342026-01-30T16:46:33.290ZINFOcrucible-agent (worker): applying SMF actions for region ac8ae3f9-96f5-4fe4-bc32-7976ff62899f running snapshot b37c9f1e-c473-4967-bb6b-8d1939804129 (state Requested)...
352026-01-30T16:46:33.290ZINFOcrucible-agent (dropshot): request completed latency_us = 22191 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:62725 req_id = f1d398d0-810c-4aa6-b3c0-2b38eb6b7db0 response_code = 200 uri = /crucible/0/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f/snapshots/b37c9f1e-c473-4967-bb6b-8d1939804129/run
362026-01-30T16:46:33.295ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-ac8ae3f9-96f5-4fe4-bc32-7976ff62899f-b37c9f1e-c473-4967-bb6b-8d1939804129
372026-01-30T16:46:33.305ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-ac8ae3f9-96f5-4fe4-bc32-7976ff62899f-b37c9f1e-c473-4967-bb6b-8d1939804129
382026-01-30T16:46:33.315ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-ac8ae3f9-96f5-4fe4-bc32-7976ff62899f-b37c9f1e-c473-4967-bb6b-8d1939804129
392026-01-30T16:46:33.315ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f/.zfs/snapshot/b37c9f1e-c473-4967-bb6b-8d1939804129
402026-01-30T16:46:33.316ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-01-30T16:46:33.316ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-01-30T16:46:33.316ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
432026-01-30T16:46:33.316ZINFOcrucible-agent (worker): commit
442026-01-30T16:46:33.319ZINFOcrucible-agent (worker): ok!
452026-01-30T16:46:33.324ZINFOcrucible-agent (worker): SMF ok!
462026-01-30T16:46:33.324ZINFOcrucible-agent (datafile): region ac8ae3f9-96f5-4fe4-bc32-7976ff62899f running snapshot b37c9f1e-c473-4967-bb6b-8d1939804129 state: Requested -> Created
472026-01-30T16:46:33.449ZINFOcrucible-agent (datafile): path is "/data/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f"
482026-01-30T16:46:33.456ZINFOcrucible-agent (datafile): dataset is oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f
492026-01-30T16:46:33.463ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f@b37c9f1e-c473-4967-bb6b-8d1939804129
502026-01-30T16:46:33.463ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3d3852d2-5917-4f0c-909d-b0fb44a1856c/crucible/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f", "b37c9f1e-c473-4967-bb6b-8d1939804129"]
512026-01-30T16:46:33.470ZINFOcrucible-agent (dropshot): request completed latency_us = 20501 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:62725 req_id = a413bdd3-87b1-4284-ab8d-87d246d13232 response_code = 200 uri = /crucible/0/regions/ac8ae3f9-96f5-4fe4-bc32-7976ff62899f/snapshots/b37c9f1e-c473-4967-bb6b-8d1939804129/run