Max level shown:
12025-04-24T06:49:01.535ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:54281
22025-04-24T06:49:01.535ZINFOcrucible-agent (datafile): region 6c4d1050-89ff-49f2-9fe5-abf525ccdbc5 state: Requested
32025-04-24T06:49:01.535ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-04-24T06:49:01.535ZINFOcrucible-agent (dropshot): request completed latency_us = 682 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:54281 req_id = a10ab040-e86e-4f7e-8384-2cb6fd619061 response_code = 200 uri = /crucible/0/regions
52025-04-24T06:49:01.547ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/6c4d1050-89ff-49f2-9fe5-abf525ccdbc5
62025-04-24T06:49:01.547ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/6c4d1050-89ff-49f2-9fe5-abf525ccdbc5
72025-04-24T06:49:01.627ZINFOcrucible-agent (worker): creating region Region { id: RegionId("6c4d1050-89ff-49f2-9fe5-abf525ccdbc5"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/6c4d1050-89ff-49f2-9fe5-abf525ccdbc5" region = 6c4d1050-89ff-49f2-9fe5-abf525ccdbc5
82025-04-24T06:49:01.629ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "6c4d1050-89ff-49f2-9fe5-abf525ccdbc5" "--data" "/data/regions/6c4d1050-89ff-49f2-9fe5-abf525ccdbc5" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 6c4d1050-89ff-49f2-9fe5-abf525ccdbc5
92025-04-24T06:49:01.897ZINFOcrucible-agent (worker): region files created ok region = 6c4d1050-89ff-49f2-9fe5-abf525ccdbc5
102025-04-24T06:49:01.897ZINFOcrucible-agent (datafile): region 6c4d1050-89ff-49f2-9fe5-abf525ccdbc5 state: Requested -> Created
112025-04-24T06:49:01.897ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-04-24T06:49:01.908ZINFOcrucible-agent (dropshot): request completed latency_us = 92 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:54281 req_id = ef636879-d812-4877-9489-3fd9d3143401 response_code = 200 uri = /crucible/0/regions
132025-04-24T06:49:01.926ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-6c4d1050-89ff-49f2-9fe5-abf525ccdbc5
142025-04-24T06:49:01.955ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-6c4d1050-89ff-49f2-9fe5-abf525ccdbc5
152025-04-24T06:49:01.973ZINFOcrucible-agent (worker): creating config property group
162025-04-24T06:49:01.990ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-6c4d1050-89ff-49f2-9fe5-abf525ccdbc5
172025-04-24T06:49:01.990ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/6c4d1050-89ff-49f2-9fe5-abf525ccdbc5
182025-04-24T06:49:01.990ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-04-24T06:49:01.990ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
202025-04-24T06:49:01.990ZINFOcrucible-agent (worker): commit
212025-04-24T06:49:02.002ZINFOcrucible-agent (worker): ok!
222025-04-24T06:49:02.013ZINFOcrucible-agent (worker): SMF ok!
232025-04-24T06:49:03.280ZINFOcrucible-agent (dropshot): request completed latency_us = 47 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:54281 req_id = 4e2d7b58-a746-49b1-b4a8-667fd6e35c1d response_code = 200 uri = /crucible/0/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0
242025-04-24T06:49:03.289ZINFOcrucible-agent (datafile): path is "/data/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0"
252025-04-24T06:49:03.300ZINFOcrucible-agent (datafile): dataset is oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0
262025-04-24T06:49:03.308ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0@fe402274-71cb-4663-b868-094fc9c911d8
272025-04-24T06:49:03.308ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0", "fe402274-71cb-4663-b868-094fc9c911d8"]
282025-04-24T06:49:03.326ZINFOcrucible-agent (dropshot): request completed latency_us = 42759 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:54281 req_id = 31b3aa6d-7ca1-40b6-8d81-88ce17aac907 response_code = 200 uri = /crucible/0/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0/snapshots/fe402274-71cb-4663-b868-094fc9c911d8
292025-04-24T06:49:03.329ZINFOcrucible-agent (datafile): path is "/data/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0"
302025-04-24T06:49:03.338ZINFOcrucible-agent (datafile): dataset is oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0
312025-04-24T06:49:03.347ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0@fe402274-71cb-4663-b868-094fc9c911d8
322025-04-24T06:49:03.347ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0", "fe402274-71cb-4663-b868-094fc9c911d8"]
332025-04-24T06:49:03.360ZINFOcrucible-agent (datafile): requesting running snapshot 3ca18dd8-30e8-4b2e-9b6a-6115300d18f0-fe402274-71cb-4663-b868-094fc9c911d8 state: Requested
342025-04-24T06:49:03.360ZINFOcrucible-agent (worker): applying SMF actions for region 3ca18dd8-30e8-4b2e-9b6a-6115300d18f0 running snapshot fe402274-71cb-4663-b868-094fc9c911d8 (state Requested)...
352025-04-24T06:49:03.360ZINFOcrucible-agent (dropshot): request completed latency_us = 31182 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:54281 req_id = e319f1bd-30a7-4151-b8cb-0d22d959fa83 response_code = 200 uri = /crucible/0/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0/snapshots/fe402274-71cb-4663-b868-094fc9c911d8/run
362025-04-24T06:49:03.363ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-3ca18dd8-30e8-4b2e-9b6a-6115300d18f0-fe402274-71cb-4663-b868-094fc9c911d8
372025-04-24T06:49:03.384ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-3ca18dd8-30e8-4b2e-9b6a-6115300d18f0-fe402274-71cb-4663-b868-094fc9c911d8
382025-04-24T06:49:03.414ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-3ca18dd8-30e8-4b2e-9b6a-6115300d18f0-fe402274-71cb-4663-b868-094fc9c911d8
392025-04-24T06:49:03.415ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0/.zfs/snapshot/fe402274-71cb-4663-b868-094fc9c911d8
402025-04-24T06:49:03.415ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-04-24T06:49:03.415ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-04-24T06:49:03.415ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
432025-04-24T06:49:03.415ZINFOcrucible-agent (worker): commit
442025-04-24T06:49:03.426ZINFOcrucible-agent (worker): ok!
452025-04-24T06:49:03.437ZINFOcrucible-agent (worker): SMF ok!
462025-04-24T06:49:03.437ZINFOcrucible-agent (datafile): region 3ca18dd8-30e8-4b2e-9b6a-6115300d18f0 running snapshot fe402274-71cb-4663-b868-094fc9c911d8 state: Requested -> Created
472025-04-24T06:49:03.732ZINFOcrucible-agent (datafile): path is "/data/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0"
482025-04-24T06:49:03.741ZINFOcrucible-agent (datafile): dataset is oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0
492025-04-24T06:49:03.749ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0@fe402274-71cb-4663-b868-094fc9c911d8
502025-04-24T06:49:03.749ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8d813973-46ec-459a-bf10-547f8bb041f6/crucible/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0", "fe402274-71cb-4663-b868-094fc9c911d8"]
512025-04-24T06:49:03.758ZINFOcrucible-agent (dropshot): request completed latency_us = 25470 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:54281 req_id = 282dae46-0dfe-44dd-b0ae-98e2c5b3e854 response_code = 200 uri = /crucible/0/regions/3ca18dd8-30e8-4b2e-9b6a-6115300d18f0/snapshots/fe402274-71cb-4663-b868-094fc9c911d8/run