Max level shown:
1 [ Feb 28 02:09:36 Enabled. ]
2 [ Feb 28 02:09:36 Rereading configuration. ]
3 [ Feb 28 02:09:40 Rereading configuration. ]
4 [ Feb 28 02:09:51 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_cbba9e4a-2c5a-45f6-a2bf-a3a7a30df21e/crucible -l [fd00:1122:3344:101::1a]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-02-28T02:09:52.119ZINFOcrucible-agent: dataset: "oxp_cbba9e4a-2c5a-45f6-a2bf-a3a7a30df21e/crucible"
72025-02-28T02:09:52.122ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::1a]:32345
82025-02-28T02:09:52.122ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-02-28T02:09:52.136ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-02-28T02:09:52.372ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::1a]:32345
112025-02-28T02:10:22.835ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::a]:52216
122025-02-28T02:10:22.887ZINFOcrucible-agent (datafile): region d2f0d992-e09f-4d38-897f-c08a547c69b1 state: Requested
132025-02-28T02:10:22.892ZINFOcrucible-agent (dropshot): request completed latency_us = 26316 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52216 req_id = 0906cd33-4241-4edc-996a-64b0e4d3d712 response_code = 200 uri = /crucible/0/regions
142025-02-28T02:10:22.892ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-02-28T02:10:22.900ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_cbba9e4a-2c5a-45f6-a2bf-a3a7a30df21e/crucible/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1
162025-02-28T02:10:22.900ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_cbba9e4a-2c5a-45f6-a2bf-a3a7a30df21e/crucible/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1
172025-02-28T02:10:22.950ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d2f0d992-e09f-4d38-897f-c08a547c69b1"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1" region = d2f0d992-e09f-4d38-897f-c08a547c69b1
182025-02-28T02:10:22.950ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d2f0d992-e09f-4d38-897f-c08a547c69b1" "--data" "/data/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = d2f0d992-e09f-4d38-897f-c08a547c69b1
192025-02-28T02:10:23.160ZINFOcrucible-agent (dropshot): request completed latency_us = 85 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52216 req_id = d30a316f-607f-4ea9-bb44-e6bd5843c2f6 response_code = 200 uri = /crucible/0/regions
202025-02-28T02:10:23.195ZINFOcrucible-agent (worker): region files created ok region = d2f0d992-e09f-4d38-897f-c08a547c69b1
212025-02-28T02:10:23.195ZINFOcrucible-agent (datafile): region d2f0d992-e09f-4d38-897f-c08a547c69b1 state: Requested -> Created
222025-02-28T02:10:23.195ZINFOcrucible-agent (worker): applying SMF actions post create...
232025-02-28T02:10:23.195ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d2f0d992-e09f-4d38-897f-c08a547c69b1
242025-02-28T02:10:23.201ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d2f0d992-e09f-4d38-897f-c08a547c69b1
252025-02-28T02:10:23.214ZINFOcrucible-agent (worker): creating config property group
262025-02-28T02:10:23.220ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d2f0d992-e09f-4d38-897f-c08a547c69b1
272025-02-28T02:10:23.220ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1
282025-02-28T02:10:23.220ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292025-02-28T02:10:23.221ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
302025-02-28T02:10:23.221ZINFOcrucible-agent (worker): commit
312025-02-28T02:10:23.225ZINFOcrucible-agent (worker): ok!
322025-02-28T02:10:23.240ZINFOcrucible-agent (worker): SMF ok!
332025-02-28T02:10:23.688ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52216 req_id = 73cc7073-f521-439a-82d7-39b4502f3b19 response_code = 200 uri = /crucible/0/regions
342025-02-28T02:10:24.017ZINFOcrucible-agent (dropshot): request completed latency_us = 2019 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:52216 req_id = dd4977df-eaf6-416f-b451-b5430b5dbcbd response_code = 200 uri = /crucible/0/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1
352025-02-28T02:10:24.019ZINFOcrucible-agent (datafile): path is "/data/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1"
362025-02-28T02:10:24.030ZINFOcrucible-agent (datafile): dataset is oxp_cbba9e4a-2c5a-45f6-a2bf-a3a7a30df21e/crucible/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1
372025-02-28T02:10:24.040ZINFOcrucible-agent (datafile): region d2f0d992-e09f-4d38-897f-c08a547c69b1 state: Created -> Tombstoned
382025-02-28T02:10:24.040ZINFOcrucible-agent (worker): applying SMF actions before removal...
392025-02-28T02:10:24.040ZINFOcrucible-agent (dropshot): request completed latency_us = 20886 local_addr = [fd00:1122:3344:101::1a]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:52216 req_id = 19a18dca-08f9-4cc2-b1c5-1f4e1323313b response_code = 204 uri = /crucible/0/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1
402025-02-28T02:10:24.041ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-d2f0d992-e09f-4d38-897f-c08a547c69b1 (instance states: (Some(Online), None))
412025-02-28T02:10:24.045ZINFOcrucible-agent (dropshot): request completed latency_us = 30 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:52216 req_id = 98cc25df-5541-4903-96ef-8ead04d525a3 response_code = 200 uri = /crucible/0/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1
422025-02-28T02:10:24.053ZINFOcrucible-agent (worker): SMF ok!
432025-02-28T02:10:24.065ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_cbba9e4a-2c5a-45f6-a2bf-a3a7a30df21e/crucible/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1" region = d2f0d992-e09f-4d38-897f-c08a547c69b1
442025-02-28T02:10:24.130ZINFOcrucible-agent (datafile): region d2f0d992-e09f-4d38-897f-c08a547c69b1 state: Tombstoned -> Destroyed
452025-02-28T02:10:24.165ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:52216 req_id = 810aee58-b9e5-4174-96d2-bf4e1a2f71e5 response_code = 200 uri = /crucible/0/regions/d2f0d992-e09f-4d38-897f-c08a547c69b1
462025-02-28T02:10:26.354ZINFOcrucible-agent (datafile): region bdbef397-d8d6-4782-9afd-327e4a528c95 state: Requested
472025-02-28T02:10:26.354ZINFOcrucible-agent (dropshot): request completed latency_us = 229 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52216 req_id = 75224773-4b1f-4469-805a-db66073a8966 response_code = 200 uri = /crucible/0/regions
482025-02-28T02:10:26.354ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492025-02-28T02:10:26.362ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_cbba9e4a-2c5a-45f6-a2bf-a3a7a30df21e/crucible/regions/bdbef397-d8d6-4782-9afd-327e4a528c95
502025-02-28T02:10:26.362ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_cbba9e4a-2c5a-45f6-a2bf-a3a7a30df21e/crucible/regions/bdbef397-d8d6-4782-9afd-327e4a528c95
512025-02-28T02:10:26.410ZINFOcrucible-agent (worker): creating region Region { id: RegionId("bdbef397-d8d6-4782-9afd-327e4a528c95"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/bdbef397-d8d6-4782-9afd-327e4a528c95" region = bdbef397-d8d6-4782-9afd-327e4a528c95
522025-02-28T02:10:26.412ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "bdbef397-d8d6-4782-9afd-327e4a528c95" "--data" "/data/regions/bdbef397-d8d6-4782-9afd-327e4a528c95" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = bdbef397-d8d6-4782-9afd-327e4a528c95
532025-02-28T02:10:26.514ZINFOcrucible-agent (worker): region files created ok region = bdbef397-d8d6-4782-9afd-327e4a528c95
542025-02-28T02:10:26.514ZINFOcrucible-agent (datafile): region bdbef397-d8d6-4782-9afd-327e4a528c95 state: Requested -> Created
552025-02-28T02:10:26.514ZINFOcrucible-agent (worker): applying SMF actions post create...
562025-02-28T02:10:26.515ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-bdbef397-d8d6-4782-9afd-327e4a528c95
572025-02-28T02:10:26.518ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-bdbef397-d8d6-4782-9afd-327e4a528c95
582025-02-28T02:10:26.527ZINFOcrucible-agent (worker): creating config property group
592025-02-28T02:10:26.530ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-bdbef397-d8d6-4782-9afd-327e4a528c95
602025-02-28T02:10:26.531ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/bdbef397-d8d6-4782-9afd-327e4a528c95
612025-02-28T02:10:26.531ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
622025-02-28T02:10:26.531ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
632025-02-28T02:10:26.531ZINFOcrucible-agent (worker): commit
642025-02-28T02:10:26.535ZINFOcrucible-agent (worker): ok!
652025-02-28T02:10:26.541ZINFOcrucible-agent (worker): SMF ok!
662025-02-28T02:10:26.729ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52216 req_id = e2ae9fb2-a139-4f63-8dea-3691855a4fed response_code = 200 uri = /crucible/0/regions