Max level shown:
1 [ Jul 18 19:14:56 Enabled. ]
2 [ Jul 18 19:14:56 Rereading configuration. ]
3 [ Jul 18 19:14:59 Rereading configuration. ]
4 [ Jul 18 19:15:05 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible -l [fd00:1122:3344:101::18]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-07-18T19:15:05.349ZINFOcrucible-agent: dataset: "oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible"
72025-07-18T19:15:05.353ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::18]:32345
82025-07-18T19:15:05.353ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-07-18T19:15:05.368ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-07-18T19:15:05.507ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::18]:32345
112025-07-18T19:15:30.999ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:65175
122025-07-18T19:15:31.024ZINFOcrucible-agent (datafile): region 3e77c582-6607-416d-904d-d725fc796064 state: Requested
132025-07-18T19:15:31.028ZINFOcrucible-agent (dropshot): request completed latency_us = 19023 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65175 req_id = ad7fb912-b864-41d7-a528-df27084d0f85 response_code = 200 uri = /crucible/0/regions
142025-07-18T19:15:31.031ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
152025-07-18T19:15:31.040ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064
162025-07-18T19:15:31.040ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064
172025-07-18T19:15:31.119ZINFOcrucible-agent (worker): creating region Region { id: RegionId("3e77c582-6607-416d-904d-d725fc796064"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/3e77c582-6607-416d-904d-d725fc796064" region = 3e77c582-6607-416d-904d-d725fc796064
182025-07-18T19:15:31.121ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "3e77c582-6607-416d-904d-d725fc796064" "--data" "/data/regions/3e77c582-6607-416d-904d-d725fc796064" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 3e77c582-6607-416d-904d-d725fc796064
192025-07-18T19:15:31.194ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65175 req_id = ac27d7a9-9600-45d9-b2aa-008483fb1d5d response_code = 200 uri = /crucible/0/regions
202025-07-18T19:15:31.439ZINFOcrucible-agent (worker): region files created ok region = 3e77c582-6607-416d-904d-d725fc796064
212025-07-18T19:15:31.439ZINFOcrucible-agent (datafile): region 3e77c582-6607-416d-904d-d725fc796064 state: Requested -> Created
222025-07-18T19:15:31.439ZINFOcrucible-agent (worker): applying SMF actions post create...
232025-07-18T19:15:31.440ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-3e77c582-6607-416d-904d-d725fc796064
242025-07-18T19:15:31.445ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-3e77c582-6607-416d-904d-d725fc796064
252025-07-18T19:15:31.460ZINFOcrucible-agent (worker): creating config property group
262025-07-18T19:15:31.463ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-3e77c582-6607-416d-904d-d725fc796064
272025-07-18T19:15:31.463ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3e77c582-6607-416d-904d-d725fc796064
282025-07-18T19:15:31.463ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292025-07-18T19:15:31.463ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
302025-07-18T19:15:31.463ZINFOcrucible-agent (worker): commit
312025-07-18T19:15:31.468ZINFOcrucible-agent (worker): ok!
322025-07-18T19:15:31.481ZINFOcrucible-agent (worker): SMF ok!
332025-07-18T19:15:31.814ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65175 req_id = f32d1bf3-f930-42c8-acf7-871335ba9296 response_code = 200 uri = /crucible/0/regions
342025-07-18T19:26:15.941ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:65404
352025-07-18T19:26:15.945ZINFOcrucible-agent (dropshot): request completed latency_us = 4448 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:65404 req_id = 88c304ec-3e10-430b-a409-63c406d4aeea response_code = 200 uri = /crucible/0/regions/3e77c582-6607-416d-904d-d725fc796064
362025-07-18T19:26:15.948ZINFOcrucible-agent (datafile): path is "/data/regions/3e77c582-6607-416d-904d-d725fc796064"
372025-07-18T19:26:15.957ZINFOcrucible-agent (datafile): dataset is oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064
382025-07-18T19:26:15.967ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064@116a239b-954f-4639-80ac-bb95bc129003
392025-07-18T19:26:15.967ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064", "116a239b-954f-4639-80ac-bb95bc129003"]
402025-07-18T19:26:15.978ZINFOcrucible-agent (dropshot): request completed latency_us = 29584 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:65404 req_id = 4536448e-70e7-4248-b332-cd2dca6ea2a8 response_code = 200 uri = /crucible/0/regions/3e77c582-6607-416d-904d-d725fc796064/snapshots/116a239b-954f-4639-80ac-bb95bc129003
412025-07-18T19:26:15.982ZINFOcrucible-agent (datafile): path is "/data/regions/3e77c582-6607-416d-904d-d725fc796064"
422025-07-18T19:26:15.990ZINFOcrucible-agent (datafile): dataset is oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064
432025-07-18T19:26:15.998ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064@116a239b-954f-4639-80ac-bb95bc129003
442025-07-18T19:26:15.998ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064", "116a239b-954f-4639-80ac-bb95bc129003"]
452025-07-18T19:26:16.008ZINFOcrucible-agent (datafile): requesting running snapshot 3e77c582-6607-416d-904d-d725fc796064-116a239b-954f-4639-80ac-bb95bc129003 state: Requested
462025-07-18T19:26:16.008ZINFOcrucible-agent (worker): applying SMF actions for region 3e77c582-6607-416d-904d-d725fc796064 running snapshot 116a239b-954f-4639-80ac-bb95bc129003 (state Requested)...
472025-07-18T19:26:16.008ZINFOcrucible-agent (dropshot): request completed latency_us = 26701 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65404 req_id = 0d5f81ab-517d-4639-9408-fe7d90eb6e69 response_code = 200 uri = /crucible/0/regions/3e77c582-6607-416d-904d-d725fc796064/snapshots/116a239b-954f-4639-80ac-bb95bc129003/run
482025-07-18T19:26:16.026ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-3e77c582-6607-416d-904d-d725fc796064-116a239b-954f-4639-80ac-bb95bc129003
492025-07-18T19:26:16.042ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-3e77c582-6607-416d-904d-d725fc796064-116a239b-954f-4639-80ac-bb95bc129003
502025-07-18T19:26:16.062ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-3e77c582-6607-416d-904d-d725fc796064-116a239b-954f-4639-80ac-bb95bc129003
512025-07-18T19:26:16.062ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3e77c582-6607-416d-904d-d725fc796064/.zfs/snapshot/116a239b-954f-4639-80ac-bb95bc129003
522025-07-18T19:26:16.062ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
532025-07-18T19:26:16.062ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
542025-07-18T19:26:16.062ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
552025-07-18T19:26:16.062ZINFOcrucible-agent (worker): commit
562025-07-18T19:26:16.067ZINFOcrucible-agent (worker): ok!
572025-07-18T19:26:16.072ZINFOcrucible-agent (worker): SMF ok!
582025-07-18T19:26:16.072ZINFOcrucible-agent (datafile): region 3e77c582-6607-416d-904d-d725fc796064 running snapshot 116a239b-954f-4639-80ac-bb95bc129003 state: Requested -> Created
592025-07-18T19:26:16.225ZINFOcrucible-agent (datafile): path is "/data/regions/3e77c582-6607-416d-904d-d725fc796064"
602025-07-18T19:26:16.233ZINFOcrucible-agent (datafile): dataset is oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064
612025-07-18T19:26:16.240ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064@116a239b-954f-4639-80ac-bb95bc129003
622025-07-18T19:26:16.240ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_8e508f4c-2811-49cd-ab16-1715133cdbf4/crucible/regions/3e77c582-6607-416d-904d-d725fc796064", "116a239b-954f-4639-80ac-bb95bc129003"]
632025-07-18T19:26:16.247ZINFOcrucible-agent (dropshot): request completed latency_us = 22885 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65404 req_id = 46801475-ba3d-46d7-8279-08f0a0fef043 response_code = 200 uri = /crucible/0/regions/3e77c582-6607-416d-904d-d725fc796064/snapshots/116a239b-954f-4639-80ac-bb95bc129003/run