Max level shown:
12026-01-05T22:00:43.071ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:57854
22026-01-05T22:00:43.097ZINFOcrucible-agent (datafile): region 3cbaca0a-e467-47bc-98de-d465c2cac0d8 state: Requested
32026-01-05T22:00:43.100ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-01-05T22:00:43.100ZINFOcrucible-agent (dropshot): request completed latency_us = 14981 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:57854 req_id = b01cb258-788c-47a0-b23f-e912fd84d84c response_code = 200 uri = /crucible/0/regions
52026-01-05T22:00:43.109ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8
62026-01-05T22:00:43.109ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8
72026-01-05T22:00:43.158ZINFOcrucible-agent (worker): creating region Region { id: RegionId("3cbaca0a-e467-47bc-98de-d465c2cac0d8"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8" region = 3cbaca0a-e467-47bc-98de-d465c2cac0d8
82026-01-05T22:00:43.159ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "3cbaca0a-e467-47bc-98de-d465c2cac0d8" "--data" "/data/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 3cbaca0a-e467-47bc-98de-d465c2cac0d8
92026-01-05T22:00:43.431ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:57854 req_id = c09ff2f8-47b1-4e75-b52f-5e62854ad71b response_code = 200 uri = /crucible/0/regions
102026-01-05T22:00:43.516ZINFOcrucible-agent (worker): region files created ok region = 3cbaca0a-e467-47bc-98de-d465c2cac0d8
112026-01-05T22:00:43.516ZINFOcrucible-agent (datafile): region 3cbaca0a-e467-47bc-98de-d465c2cac0d8 state: Requested -> Created
122026-01-05T22:00:43.516ZINFOcrucible-agent (worker): applying SMF actions post create...
132026-01-05T22:00:43.516ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-3cbaca0a-e467-47bc-98de-d465c2cac0d8
142026-01-05T22:00:43.521ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-3cbaca0a-e467-47bc-98de-d465c2cac0d8
152026-01-05T22:00:43.535ZINFOcrucible-agent (worker): creating config property group
162026-01-05T22:00:43.537ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-3cbaca0a-e467-47bc-98de-d465c2cac0d8
172026-01-05T22:00:43.538ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8
182026-01-05T22:00:43.538ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192026-01-05T22:00:43.538ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
202026-01-05T22:00:43.538ZINFOcrucible-agent (worker): commit
212026-01-05T22:00:43.542ZINFOcrucible-agent (worker): ok!
222026-01-05T22:00:43.546ZINFOcrucible-agent (worker): SMF ok!
232026-01-05T22:00:44.090ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:57854 req_id = 3ec700d5-0e63-4632-955b-3e1a70f4e7df response_code = 200 uri = /crucible/0/regions
242026-01-05T22:11:33.270ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:38760
252026-01-05T22:11:33.270ZINFOcrucible-agent (dropshot): request completed latency_us = 67 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38760 req_id = d46e5292-7d0c-4b14-a150-f1a1ac741909 response_code = 200 uri = /crucible/0/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8
262026-01-05T22:11:33.273ZINFOcrucible-agent (datafile): path is "/data/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8"
272026-01-05T22:11:33.280ZINFOcrucible-agent (datafile): dataset is oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8
282026-01-05T22:11:33.287ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8@2e842058-547c-4b6f-8fac-b6c73ffa0528
292026-01-05T22:11:33.287ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8", "2e842058-547c-4b6f-8fac-b6c73ffa0528"]
302026-01-05T22:11:33.300ZINFOcrucible-agent (dropshot): request completed latency_us = 26852 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38760 req_id = 3ae11325-6873-4e45-85c7-de30ed838175 response_code = 200 uri = /crucible/0/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8/snapshots/2e842058-547c-4b6f-8fac-b6c73ffa0528
312026-01-05T22:11:33.305ZINFOcrucible-agent (datafile): path is "/data/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8"
322026-01-05T22:11:33.311ZINFOcrucible-agent (datafile): dataset is oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8
332026-01-05T22:11:33.318ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8@2e842058-547c-4b6f-8fac-b6c73ffa0528
342026-01-05T22:11:33.318ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8", "2e842058-547c-4b6f-8fac-b6c73ffa0528"]
352026-01-05T22:11:33.328ZINFOcrucible-agent (datafile): requesting running snapshot 3cbaca0a-e467-47bc-98de-d465c2cac0d8-2e842058-547c-4b6f-8fac-b6c73ffa0528 state: Requested
362026-01-05T22:11:33.329ZINFOcrucible-agent (worker): applying SMF actions for region 3cbaca0a-e467-47bc-98de-d465c2cac0d8 running snapshot 2e842058-547c-4b6f-8fac-b6c73ffa0528 (state Requested)...
372026-01-05T22:11:33.329ZINFOcrucible-agent (dropshot): request completed latency_us = 24362 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38760 req_id = 515cab3a-bb3c-44ff-a7a1-ab4267b374e6 response_code = 200 uri = /crucible/0/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8/snapshots/2e842058-547c-4b6f-8fac-b6c73ffa0528/run
382026-01-05T22:11:33.354ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-3cbaca0a-e467-47bc-98de-d465c2cac0d8-2e842058-547c-4b6f-8fac-b6c73ffa0528
392026-01-05T22:11:33.371ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-3cbaca0a-e467-47bc-98de-d465c2cac0d8-2e842058-547c-4b6f-8fac-b6c73ffa0528
402026-01-05T22:11:33.392ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-3cbaca0a-e467-47bc-98de-d465c2cac0d8-2e842058-547c-4b6f-8fac-b6c73ffa0528
412026-01-05T22:11:33.393ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8/.zfs/snapshot/2e842058-547c-4b6f-8fac-b6c73ffa0528
422026-01-05T22:11:33.393ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
432026-01-05T22:11:33.393ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
442026-01-05T22:11:33.393ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
452026-01-05T22:11:33.393ZINFOcrucible-agent (worker): commit
462026-01-05T22:11:33.399ZINFOcrucible-agent (worker): ok!
472026-01-05T22:11:33.407ZINFOcrucible-agent (worker): SMF ok!
482026-01-05T22:11:33.407ZINFOcrucible-agent (datafile): region 3cbaca0a-e467-47bc-98de-d465c2cac0d8 running snapshot 2e842058-547c-4b6f-8fac-b6c73ffa0528 state: Requested -> Created
492026-01-05T22:11:33.597ZINFOcrucible-agent (datafile): path is "/data/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8"
502026-01-05T22:11:33.604ZINFOcrucible-agent (datafile): dataset is oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8
512026-01-05T22:11:33.610ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8@2e842058-547c-4b6f-8fac-b6c73ffa0528
522026-01-05T22:11:33.610ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8329a4ea-3572-4c1b-ac9a-7469e7db8092/crucible/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8", "2e842058-547c-4b6f-8fac-b6c73ffa0528"]
532026-01-05T22:11:33.617ZINFOcrucible-agent (dropshot): request completed latency_us = 20403 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38760 req_id = 1eb7c288-f8d6-4aea-b6ed-463706fe93a4 response_code = 200 uri = /crucible/0/regions/3cbaca0a-e467-47bc-98de-d465c2cac0d8/snapshots/2e842058-547c-4b6f-8fac-b6c73ffa0528/run