| 1 | 2026-08-05T05:18:34.878Z | INFO | crucible-agent (dropshot): accepted connection
local_addr = [fd00:1122:3344:101::19]:32345
remote_addr = [fd00:1122:3344:101::b]:39208
|
| 2 | 2026-08-05T05:18:34.878Z | INFO | crucible-agent (dropshot): request completed
latency_us = 43
local_addr = [fd00:1122:3344:101::19]:32345
method = GET
remote_addr = [fd00:1122:3344:101::b]:39208
req_id = f8692659-d4ff-4b3f-9979-6cb710a5cb8c
response_code = 200
uri = /crucible/0/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab
|
| 3 | 2026-08-05T05:18:34.881Z | INFO | crucible-agent (datafile): path is "/data/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab"
|
| 4 | 2026-08-05T05:18:34.887Z | INFO | crucible-agent (datafile): dataset is oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab
|
| 5 | 2026-08-05T05:18:34.895Z | INFO | crucible-agent (ZfsSnapshotInterface): snapshot is oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab@b62b5337-9c23-497d-99e7-6e8a97e3df3f
|
| 6 | 2026-08-05T05:18:34.895Z | INFO | crucible-agent (ZfsSnapshotInterface): parts is ["oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab", "b62b5337-9c23-497d-99e7-6e8a97e3df3f"]
|
| 7 | 2026-08-05T05:18:34.906Z | INFO | crucible-agent (dropshot): request completed
latency_us = 25679
local_addr = [fd00:1122:3344:101::19]:32345
method = GET
remote_addr = [fd00:1122:3344:101::b]:39208
req_id = 68cb0bb1-4996-4ae9-9a11-e588f0bc322c
response_code = 200
uri = /crucible/0/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab/snapshots/b62b5337-9c23-497d-99e7-6e8a97e3df3f
|
| 8 | 2026-08-05T05:18:34.917Z | INFO | crucible-agent (datafile): path is "/data/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab"
|
| 9 | 2026-08-05T05:18:34.924Z | INFO | crucible-agent (datafile): dataset is oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab
|
| 10 | 2026-08-05T05:18:34.932Z | INFO | crucible-agent (ZfsSnapshotInterface): snapshot is oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab@b62b5337-9c23-497d-99e7-6e8a97e3df3f
|
| 11 | 2026-08-05T05:18:34.932Z | INFO | crucible-agent (ZfsSnapshotInterface): parts is ["oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab", "b62b5337-9c23-497d-99e7-6e8a97e3df3f"]
|
| 12 | 2026-08-05T05:18:34.945Z | INFO | crucible-agent (datafile): requesting running snapshot 3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab-b62b5337-9c23-497d-99e7-6e8a97e3df3f state: Requested
|
| 13 | 2026-08-05T05:18:34.953Z | INFO | crucible-agent (worker): applying SMF actions for region 3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab running snapshot b62b5337-9c23-497d-99e7-6e8a97e3df3f (state Requested)...
|
| 14 | 2026-08-05T05:18:34.953Z | INFO | crucible-agent (dropshot): request completed
latency_us = 35281
local_addr = [fd00:1122:3344:101::19]:32345
method = POST
remote_addr = [fd00:1122:3344:101::b]:39208
req_id = e8ebc28d-6398-4faa-98ab-f6c9bef5e64c
response_code = 200
uri = /crucible/0/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab/snapshots/b62b5337-9c23-497d-99e7-6e8a97e3df3f/run
|
| 15 | 2026-08-05T05:18:34.963Z | INFO | crucible-agent (worker): creating missing snapshot instance snapshot-3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab-b62b5337-9c23-497d-99e7-6e8a97e3df3f
|
| 16 | 2026-08-05T05:18:34.975Z | INFO | crucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab-b62b5337-9c23-497d-99e7-6e8a97e3df3f
|
| 17 | 2026-08-05T05:18:34.987Z | INFO | crucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab-b62b5337-9c23-497d-99e7-6e8a97e3df3f
|
| 18 | 2026-08-05T05:18:34.987Z | INFO | crucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab/.zfs/snapshot/b62b5337-9c23-497d-99e7-6e8a97e3df3f
|
| 19 | 2026-08-05T05:18:34.987Z | INFO | crucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
|
| 20 | 2026-08-05T05:18:34.987Z | INFO | crucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
|
| 21 | 2026-08-05T05:18:34.987Z | INFO | crucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
|
| 22 | 2026-08-05T05:18:34.987Z | INFO | crucible-agent (worker): commit
|
| 23 | 2026-08-05T05:18:34.990Z | INFO | crucible-agent (worker): ok!
|
| 24 | 2026-08-05T05:18:34.994Z | INFO | crucible-agent (worker): SMF ok!
|
| 25 | 2026-08-05T05:18:34.994Z | INFO | crucible-agent (datafile): region 3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab running snapshot b62b5337-9c23-497d-99e7-6e8a97e3df3f state: Requested -> Created
|
| 26 | 2026-08-05T05:18:35.251Z | INFO | crucible-agent (datafile): path is "/data/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab"
|
| 27 | 2026-08-05T05:18:35.258Z | INFO | crucible-agent (datafile): dataset is oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab
|
| 28 | 2026-08-05T05:18:35.264Z | INFO | crucible-agent (ZfsSnapshotInterface): snapshot is oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab@b62b5337-9c23-497d-99e7-6e8a97e3df3f
|
| 29 | 2026-08-05T05:18:35.264Z | INFO | crucible-agent (ZfsSnapshotInterface): parts is ["oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab", "b62b5337-9c23-497d-99e7-6e8a97e3df3f"]
|
| 30 | 2026-08-05T05:18:35.269Z | INFO | crucible-agent (dropshot): request completed
latency_us = 17523
local_addr = [fd00:1122:3344:101::19]:32345
method = POST
remote_addr = [fd00:1122:3344:101::b]:39208
req_id = 48053738-1f3d-4c9c-aeed-e03f12b21bf4
response_code = 200
uri = /crucible/0/regions/3eeef639-6ad9-4c6f-99e9-2ca402f9a2ab/snapshots/b62b5337-9c23-497d-99e7-6e8a97e3df3f/run
|
| 31 | 2026-08-05T05:18:40.268Z | INFO | crucible-agent (datafile): region 0c357648-f492-4f53-bfa4-73e8533190b7 state: Requested
|
| 32 | 2026-08-05T05:18:40.268Z | INFO | crucible-agent (dropshot): request completed
latency_us = 217
local_addr = [fd00:1122:3344:101::19]:32345
method = POST
remote_addr = [fd00:1122:3344:101::b]:39208
req_id = be188d7e-3b03-4137-bf7a-5dc591028914
response_code = 200
uri = /crucible/0/regions
|
| 33 | 2026-08-05T05:18:40.268Z | INFO | crucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
|
| 34 | 2026-08-05T05:18:40.275Z | INFO | crucible-agent (worker): zfs set reservation of 2684354560 for oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/0c357648-f492-4f53-bfa4-73e8533190b7
|
| 35 | 2026-08-05T05:18:40.275Z | INFO | crucible-agent (worker): zfs set quota of 6442450944 for oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/0c357648-f492-4f53-bfa4-73e8533190b7
|
| 36 | 2026-08-05T05:18:40.324Z | INFO | crucible-agent (worker): creating region Region { id: RegionId("0c357648-f492-4f53-bfa4-73e8533190b7"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/0c357648-f492-4f53-bfa4-73e8533190b7"
region = 0c357648-f492-4f53-bfa4-73e8533190b7
|
| 37 | 2026-08-05T05:18:40.324Z | INFO | crucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "0c357648-f492-4f53-bfa4-73e8533190b7" "--data" "/data/regions/0c357648-f492-4f53-bfa4-73e8533190b7" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted"
region = 0c357648-f492-4f53-bfa4-73e8533190b7
|
| 38 | 2026-08-05T05:18:40.368Z | INFO | crucible-agent (worker): region files created ok
region = 0c357648-f492-4f53-bfa4-73e8533190b7
|
| 39 | 2026-08-05T05:18:40.368Z | INFO | crucible-agent (datafile): region 0c357648-f492-4f53-bfa4-73e8533190b7 state: Requested -> Created
|
| 40 | 2026-08-05T05:18:40.368Z | INFO | crucible-agent (worker): applying SMF actions post create...
|
| 41 | 2026-08-05T05:18:40.369Z | INFO | crucible-agent (worker): creating missing downstairs instance downstairs-0c357648-f492-4f53-bfa4-73e8533190b7
|
| 42 | 2026-08-05T05:18:40.370Z | INFO | crucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-0c357648-f492-4f53-bfa4-73e8533190b7
|
| 43 | 2026-08-05T05:18:40.373Z | INFO | crucible-agent (worker): creating config property group
|
| 44 | 2026-08-05T05:18:40.375Z | INFO | crucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-0c357648-f492-4f53-bfa4-73e8533190b7
|
| 45 | 2026-08-05T05:18:40.375Z | INFO | crucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/0c357648-f492-4f53-bfa4-73e8533190b7
|
| 46 | 2026-08-05T05:18:40.375Z | INFO | crucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
|
| 47 | 2026-08-05T05:18:40.375Z | INFO | crucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
|
| 48 | 2026-08-05T05:18:40.375Z | INFO | crucible-agent (worker): commit
|
| 49 | 2026-08-05T05:18:40.377Z | INFO | crucible-agent (worker): ok!
|
| 50 | 2026-08-05T05:18:40.384Z | INFO | crucible-agent (worker): SMF ok!
|
| 51 | 2026-08-05T05:18:40.604Z | INFO | crucible-agent (dropshot): request completed
latency_us = 82
local_addr = [fd00:1122:3344:101::19]:32345
method = POST
remote_addr = [fd00:1122:3344:101::b]:39208
req_id = c5583d4f-ac2a-4a4b-9e5f-5cc5e7b920c9
response_code = 200
uri = /crucible/0/regions
|
| 52 | 2026-08-05T05:19:06.020Z | INFO | crucible-agent (dropshot): request completed
latency_us = 48
local_addr = [fd00:1122:3344:101::19]:32345
method = GET
remote_addr = [fd00:1122:3344:101::b]:39208
req_id = da219ad3-3716-48d0-b710-42f56df18d4f
response_code = 200
uri = /crucible/0/regions/0c357648-f492-4f53-bfa4-73e8533190b7
|
| 53 | 2026-08-05T05:19:06.024Z | INFO | crucible-agent (datafile): path is "/data/regions/0c357648-f492-4f53-bfa4-73e8533190b7"
|
| 54 | 2026-08-05T05:19:06.033Z | INFO | crucible-agent (datafile): dataset is oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/0c357648-f492-4f53-bfa4-73e8533190b7
|
| 55 | 2026-08-05T05:19:06.040Z | INFO | crucible-agent (datafile): region 0c357648-f492-4f53-bfa4-73e8533190b7 state: Created -> Tombstoned
|
| 56 | 2026-08-05T05:19:06.040Z | INFO | crucible-agent (dropshot): request completed
latency_us = 15853
local_addr = [fd00:1122:3344:101::19]:32345
method = DELETE
remote_addr = [fd00:1122:3344:101::b]:39208
req_id = c4aae899-6e4e-4a77-933e-a35eb2d183d2
response_code = 204
uri = /crucible/0/regions/0c357648-f492-4f53-bfa4-73e8533190b7
|
| 57 | 2026-08-05T05:19:06.040Z | INFO | crucible-agent (worker): applying SMF actions before removal...
|
| 58 | 2026-08-05T05:19:06.042Z | INFO | crucible-agent (worker): disabling downstairs instance: downstairs-0c357648-f492-4f53-bfa4-73e8533190b7 (instance states: (Some(Online), None))
|
| 59 | 2026-08-05T05:19:06.043Z | INFO | crucible-agent (dropshot): request completed
latency_us = 32
local_addr = [fd00:1122:3344:101::19]:32345
method = GET
remote_addr = [fd00:1122:3344:101::b]:39208
req_id = 905c0a25-0ae4-43cb-87e2-10a6f5e47f06
response_code = 200
uri = /crucible/0/regions/0c357648-f492-4f53-bfa4-73e8533190b7
|
| 60 | 2026-08-05T05:19:06.046Z | INFO | crucible-agent (worker): SMF ok!
|
| 61 | 2026-08-05T05:19:06.055Z | INFO | crucible-agent (worker): deleting zfs dataset "oxp_1df1fe12-a1e9-4510-9962-d31acf3f3273/crucible/regions/0c357648-f492-4f53-bfa4-73e8533190b7"
region = 0c357648-f492-4f53-bfa4-73e8533190b7
|
| 62 | 2026-08-05T05:19:06.100Z | INFO | crucible-agent (datafile): region 0c357648-f492-4f53-bfa4-73e8533190b7 state: Tombstoned -> Destroyed
|
| 63 | 2026-08-05T05:19:06.165Z | INFO | crucible-agent (dropshot): request completed
latency_us = 58
local_addr = [fd00:1122:3344:101::19]:32345
method = GET
remote_addr = [fd00:1122:3344:101::b]:39208
req_id = db9a9ce0-2353-41de-8362-fea40a993edb
response_code = 200
uri = /crucible/0/regions/0c357648-f492-4f53-bfa4-73e8533190b7
|