Max level shown:
12025-03-04T06:09:38.940ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::b]:61290
22025-03-04T06:09:38.940ZINFOcrucible-agent (datafile): region 0a784496-c43d-48c5-8473-85d81e383afc state: Requested
32025-03-04T06:09:38.941ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-03-04T06:09:38.941ZINFOcrucible-agent (dropshot): request completed latency_us = 1310 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:61290 req_id = 091947b4-1433-4124-8351-33dee4ce7a1d response_code = 200 uri = /crucible/0/regions
52025-03-04T06:09:38.952ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/0a784496-c43d-48c5-8473-85d81e383afc
62025-03-04T06:09:38.952ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/0a784496-c43d-48c5-8473-85d81e383afc
72025-03-04T06:09:39.007ZINFOcrucible-agent (worker): creating region Region { id: RegionId("0a784496-c43d-48c5-8473-85d81e383afc"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/0a784496-c43d-48c5-8473-85d81e383afc" region = 0a784496-c43d-48c5-8473-85d81e383afc
82025-03-04T06:09:39.007ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "0a784496-c43d-48c5-8473-85d81e383afc" "--data" "/data/regions/0a784496-c43d-48c5-8473-85d81e383afc" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 0a784496-c43d-48c5-8473-85d81e383afc
92025-03-04T06:09:39.093ZINFOcrucible-agent (worker): region files created ok region = 0a784496-c43d-48c5-8473-85d81e383afc
102025-03-04T06:09:39.093ZINFOcrucible-agent (datafile): region 0a784496-c43d-48c5-8473-85d81e383afc state: Requested -> Created
112025-03-04T06:09:39.093ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-03-04T06:09:39.093ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-0a784496-c43d-48c5-8473-85d81e383afc
132025-03-04T06:09:39.099ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-0a784496-c43d-48c5-8473-85d81e383afc
142025-03-04T06:09:39.111ZINFOcrucible-agent (worker): creating config property group
152025-03-04T06:09:39.113ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-0a784496-c43d-48c5-8473-85d81e383afc
162025-03-04T06:09:39.114ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/0a784496-c43d-48c5-8473-85d81e383afc
172025-03-04T06:09:39.114ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-03-04T06:09:39.114ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
192025-03-04T06:09:39.114ZINFOcrucible-agent (worker): commit
202025-03-04T06:09:39.117ZINFOcrucible-agent (worker): ok!
212025-03-04T06:09:39.122ZINFOcrucible-agent (worker): SMF ok!
222025-03-04T06:09:39.240ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:61290 req_id = 44f008fd-2ae4-4e09-9f66-56429b1cbca3 response_code = 200 uri = /crucible/0/regions
232025-03-04T06:09:40.131ZINFOcrucible-agent (dropshot): request completed latency_us = 2717 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:61290 req_id = 66749156-18ee-4808-8b28-96be68451b68 response_code = 200 uri = /crucible/0/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1
242025-03-04T06:09:40.137ZINFOcrucible-agent (datafile): path is "/data/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1"
252025-03-04T06:09:40.147ZINFOcrucible-agent (datafile): dataset is oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1
262025-03-04T06:09:40.155ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1@837a2c65-e2cf-43e9-8ee5-8788e74ee743
272025-03-04T06:09:40.155ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1", "837a2c65-e2cf-43e9-8ee5-8788e74ee743"]
282025-03-04T06:09:40.167ZINFOcrucible-agent (dropshot): request completed latency_us = 33496 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:61290 req_id = 03f63f56-f222-4623-b7e0-c6cda6ddb2c0 response_code = 200 uri = /crucible/0/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1/snapshots/837a2c65-e2cf-43e9-8ee5-8788e74ee743
292025-03-04T06:09:40.173ZINFOcrucible-agent (datafile): path is "/data/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1"
302025-03-04T06:09:40.181ZINFOcrucible-agent (datafile): dataset is oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1
312025-03-04T06:09:40.189ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1@837a2c65-e2cf-43e9-8ee5-8788e74ee743
322025-03-04T06:09:40.189ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1", "837a2c65-e2cf-43e9-8ee5-8788e74ee743"]
332025-03-04T06:09:40.198ZINFOcrucible-agent (datafile): requesting running snapshot 257c40bb-a9c3-4d27-ae30-7f5940d337b1-837a2c65-e2cf-43e9-8ee5-8788e74ee743 state: Requested
342025-03-04T06:09:40.198ZINFOcrucible-agent (worker): applying SMF actions for region 257c40bb-a9c3-4d27-ae30-7f5940d337b1 running snapshot 837a2c65-e2cf-43e9-8ee5-8788e74ee743 (state Requested)...
352025-03-04T06:09:40.198ZINFOcrucible-agent (dropshot): request completed latency_us = 25360 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:61290 req_id = ed0f4e5e-1218-4d0b-ad55-5c5dfc662b45 response_code = 200 uri = /crucible/0/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1/snapshots/837a2c65-e2cf-43e9-8ee5-8788e74ee743/run
362025-03-04T06:09:40.200ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-257c40bb-a9c3-4d27-ae30-7f5940d337b1-837a2c65-e2cf-43e9-8ee5-8788e74ee743
372025-03-04T06:09:40.207ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-257c40bb-a9c3-4d27-ae30-7f5940d337b1-837a2c65-e2cf-43e9-8ee5-8788e74ee743
382025-03-04T06:09:40.216ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-257c40bb-a9c3-4d27-ae30-7f5940d337b1-837a2c65-e2cf-43e9-8ee5-8788e74ee743
392025-03-04T06:09:40.217ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1/.zfs/snapshot/837a2c65-e2cf-43e9-8ee5-8788e74ee743
402025-03-04T06:09:40.217ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-03-04T06:09:40.217ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-03-04T06:09:40.217ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
432025-03-04T06:09:40.217ZINFOcrucible-agent (worker): commit
442025-03-04T06:09:40.221ZINFOcrucible-agent (worker): ok!
452025-03-04T06:09:40.225ZINFOcrucible-agent (worker): SMF ok!
462025-03-04T06:09:40.225ZINFOcrucible-agent (datafile): region 257c40bb-a9c3-4d27-ae30-7f5940d337b1 running snapshot 837a2c65-e2cf-43e9-8ee5-8788e74ee743 state: Requested -> Created
472025-03-04T06:09:40.368ZINFOcrucible-agent (datafile): path is "/data/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1"
482025-03-04T06:09:40.377ZINFOcrucible-agent (datafile): dataset is oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1
492025-03-04T06:09:40.385ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1@837a2c65-e2cf-43e9-8ee5-8788e74ee743
502025-03-04T06:09:40.385ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_875c83eb-f73c-48c5-be95-6a75e92d3a3b/crucible/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1", "837a2c65-e2cf-43e9-8ee5-8788e74ee743"]
512025-03-04T06:09:40.393ZINFOcrucible-agent (dropshot): request completed latency_us = 25548 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:61290 req_id = c4261a52-26c2-4e3d-829d-cf7475bc6df1 response_code = 200 uri = /crucible/0/regions/257c40bb-a9c3-4d27-ae30-7f5940d337b1/snapshots/837a2c65-e2cf-43e9-8ee5-8788e74ee743/run