Max level shown:
1 [ Aug 28 21:33:04 Enabled. ]
2 [ Aug 28 21:33:04 Rereading configuration. ]
3 [ Aug 28 21:33:07 Rereading configuration. ]
4 [ Aug 28 21:33:15 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_fffc6471-073e-4b9d-ad3c-51e8980e27cf/crucible -l [fd00:1122:3344:101::17]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-08-28T21:33:16.125ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 779775d5130ff7a4836f52f48b7e64d1479ee104 Commit timestamp: 2025-08-09T00:27:59.000000000Z branch: main rustc: 1.86.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72025-08-28T21:33:16.128ZINFOcrucible-agent: dataset: "oxp_fffc6471-073e-4b9d-ad3c-51e8980e27cf/crucible"
82025-08-28T21:33:16.128ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::17]:32345
92025-08-28T21:33:16.128ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102025-08-28T21:33:16.145ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112025-08-28T21:33:16.240ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::17]:32345
122025-08-28T21:34:12.445ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:40163
132025-08-28T21:34:12.480ZINFOcrucible-agent (datafile): region 07ecae3b-74e3-42cf-b2df-a2f00210b78f state: Requested
142025-08-28T21:34:12.485ZINFOcrucible-agent (dropshot): request completed latency_us = 18541 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:40163 req_id = 7e697a54-6c5d-4376-8541-76a776e58a52 response_code = 200 uri = /crucible/0/regions
152025-08-28T21:34:12.487ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
162025-08-28T21:34:12.498ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_fffc6471-073e-4b9d-ad3c-51e8980e27cf/crucible/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f
172025-08-28T21:34:12.498ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_fffc6471-073e-4b9d-ad3c-51e8980e27cf/crucible/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f
182025-08-28T21:34:12.552ZINFOcrucible-agent (worker): creating region Region { id: RegionId("07ecae3b-74e3-42cf-b2df-a2f00210b78f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f" region = 07ecae3b-74e3-42cf-b2df-a2f00210b78f
192025-08-28T21:34:12.554ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "07ecae3b-74e3-42cf-b2df-a2f00210b78f" "--data" "/data/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 07ecae3b-74e3-42cf-b2df-a2f00210b78f
202025-08-28T21:34:12.739ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:40163 req_id = 05c337d4-878f-4eb7-b20e-e6910230a7f3 response_code = 200 uri = /crucible/0/regions
212025-08-28T21:34:12.847ZINFOcrucible-agent (worker): region files created ok region = 07ecae3b-74e3-42cf-b2df-a2f00210b78f
222025-08-28T21:34:12.847ZINFOcrucible-agent (datafile): region 07ecae3b-74e3-42cf-b2df-a2f00210b78f state: Requested -> Created
232025-08-28T21:34:12.848ZINFOcrucible-agent (worker): applying SMF actions post create...
242025-08-28T21:34:12.848ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-07ecae3b-74e3-42cf-b2df-a2f00210b78f
252025-08-28T21:34:12.853ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-07ecae3b-74e3-42cf-b2df-a2f00210b78f
262025-08-28T21:34:12.868ZINFOcrucible-agent (worker): creating config property group
272025-08-28T21:34:12.872ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-07ecae3b-74e3-42cf-b2df-a2f00210b78f
282025-08-28T21:34:12.873ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f
292025-08-28T21:34:12.881ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302025-08-28T21:34:12.881ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
312025-08-28T21:34:12.881ZINFOcrucible-agent (worker): commit
322025-08-28T21:34:12.886ZINFOcrucible-agent (worker): ok!
332025-08-28T21:34:12.891ZINFOcrucible-agent (worker): SMF ok!
342025-08-28T21:34:13.220ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:40163 req_id = d272f046-7184-4e31-92e6-97b7e254ca31 response_code = 200 uri = /crucible/0/regions
352025-08-28T21:34:13.714ZINFOcrucible-agent (dropshot): request completed latency_us = 85 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:40163 req_id = d4f9594e-bced-49cf-94e5-b9d4a88bc8f8 response_code = 200 uri = /crucible/0/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f
362025-08-28T21:34:13.717ZINFOcrucible-agent (datafile): path is "/data/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f"
372025-08-28T21:34:13.730ZINFOcrucible-agent (datafile): dataset is oxp_fffc6471-073e-4b9d-ad3c-51e8980e27cf/crucible/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f
382025-08-28T21:34:13.742ZINFOcrucible-agent (datafile): region 07ecae3b-74e3-42cf-b2df-a2f00210b78f state: Created -> Tombstoned
392025-08-28T21:34:13.742ZINFOcrucible-agent (worker): applying SMF actions before removal...
402025-08-28T21:34:13.742ZINFOcrucible-agent (dropshot): request completed latency_us = 25582 local_addr = [fd00:1122:3344:101::17]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:40163 req_id = 8adf52ed-27cd-4927-ae7f-3eb84bf3f1be response_code = 204 uri = /crucible/0/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f
412025-08-28T21:34:13.743ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-07ecae3b-74e3-42cf-b2df-a2f00210b78f (instance states: (Some(Online), None))
422025-08-28T21:34:13.747ZINFOcrucible-agent (worker): SMF ok!
432025-08-28T21:34:13.751ZINFOcrucible-agent (dropshot): request completed latency_us = 697 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:40163 req_id = 3534b6ad-a2d4-4aa5-9012-4ed018bbc1c6 response_code = 200 uri = /crucible/0/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f
442025-08-28T21:34:13.758ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_fffc6471-073e-4b9d-ad3c-51e8980e27cf/crucible/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f" region = 07ecae3b-74e3-42cf-b2df-a2f00210b78f
452025-08-28T21:34:13.816ZINFOcrucible-agent (datafile): region 07ecae3b-74e3-42cf-b2df-a2f00210b78f state: Tombstoned -> Destroyed
462025-08-28T21:34:13.830ZINFOcrucible-agent (dropshot): request completed latency_us = 45 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:40163 req_id = f420c247-c677-4dad-bfa2-23c51659f758 response_code = 200 uri = /crucible/0/regions/07ecae3b-74e3-42cf-b2df-a2f00210b78f
472025-08-28T21:44:37.912ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:46194
482025-08-28T21:44:37.913ZINFOcrucible-agent (datafile): region 7247ed88-2060-4d7b-b999-695f7a55b302 state: Requested
492025-08-28T21:44:37.913ZINFOcrucible-agent (dropshot): request completed latency_us = 801 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46194 req_id = 8b6c72e8-f8d9-472a-b8a3-f09d95e6e0e3 response_code = 200 uri = /crucible/0/regions
502025-08-28T21:44:37.913ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
512025-08-28T21:44:37.925ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_fffc6471-073e-4b9d-ad3c-51e8980e27cf/crucible/regions/7247ed88-2060-4d7b-b999-695f7a55b302
522025-08-28T21:44:37.925ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_fffc6471-073e-4b9d-ad3c-51e8980e27cf/crucible/regions/7247ed88-2060-4d7b-b999-695f7a55b302
532025-08-28T21:44:37.974ZINFOcrucible-agent (worker): creating region Region { id: RegionId("7247ed88-2060-4d7b-b999-695f7a55b302"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/7247ed88-2060-4d7b-b999-695f7a55b302" region = 7247ed88-2060-4d7b-b999-695f7a55b302
542025-08-28T21:44:37.976ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "7247ed88-2060-4d7b-b999-695f7a55b302" "--data" "/data/regions/7247ed88-2060-4d7b-b999-695f7a55b302" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 7247ed88-2060-4d7b-b999-695f7a55b302
552025-08-28T21:44:38.135ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46194 req_id = 7d6b1025-800b-434d-8900-2da3c80b0c60 response_code = 200 uri = /crucible/0/regions
562025-08-28T21:44:38.302ZINFOcrucible-agent (worker): region files created ok region = 7247ed88-2060-4d7b-b999-695f7a55b302
572025-08-28T21:44:38.302ZINFOcrucible-agent (datafile): region 7247ed88-2060-4d7b-b999-695f7a55b302 state: Requested -> Created
582025-08-28T21:44:38.302ZINFOcrucible-agent (worker): applying SMF actions post create...
592025-08-28T21:44:38.304ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-7247ed88-2060-4d7b-b999-695f7a55b302
602025-08-28T21:44:38.313ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-7247ed88-2060-4d7b-b999-695f7a55b302
612025-08-28T21:44:38.334ZINFOcrucible-agent (worker): creating config property group
622025-08-28T21:44:38.336ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-7247ed88-2060-4d7b-b999-695f7a55b302
632025-08-28T21:44:38.337ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/7247ed88-2060-4d7b-b999-695f7a55b302
642025-08-28T21:44:38.337ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
652025-08-28T21:44:38.337ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
662025-08-28T21:44:38.337ZINFOcrucible-agent (worker): commit
672025-08-28T21:44:38.340ZINFOcrucible-agent (worker): ok!
682025-08-28T21:44:38.344ZINFOcrucible-agent (worker): SMF ok!
692025-08-28T21:44:38.574ZINFOcrucible-agent (dropshot): request completed latency_us = 45 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46194 req_id = ab9d92b3-c361-41bb-9607-f4fa2a8d8969 response_code = 200 uri = /crucible/0/regions