Max level shown:
1 [ Dec 21 00:47:34 Enabled. ]
2 [ Dec 21 00:47:34 Rereading configuration. ]
3 [ Dec 21 00:47:36 Rereading configuration. ]
4 [ Dec 21 00:47:40 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible -l [fd00:1122:3344:101::12]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-12-21T00:47:40.430ZINFOcrucible-agent: dataset: "oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible"
72024-12-21T00:47:40.433ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::12]:32345
82024-12-21T00:47:40.433ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-12-21T00:47:40.450ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-12-21T00:47:40.553ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::12]:32345
112024-12-21T00:48:08.690ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:54636
122024-12-21T00:48:08.722ZINFOcrucible-agent (datafile): region 3d93ccd1-6827-47ba-b053-605b0ed644d1 state: Requested
132024-12-21T00:48:08.726ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
142024-12-21T00:48:08.731ZINFOcrucible-agent (dropshot): request completed latency_us = 12956 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:54636 req_id = 4aae7324-ccc6-4f60-9871-9c139e320bed response_code = 200 uri = /crucible/0/regions
152024-12-21T00:48:08.738ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1
162024-12-21T00:48:08.738ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1
172024-12-21T00:48:08.830ZINFOcrucible-agent (worker): creating region Region { id: RegionId("3d93ccd1-6827-47ba-b053-605b0ed644d1"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1" region = 3d93ccd1-6827-47ba-b053-605b0ed644d1
182024-12-21T00:48:08.832ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "3d93ccd1-6827-47ba-b053-605b0ed644d1" "--data" "/data/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 3d93ccd1-6827-47ba-b053-605b0ed644d1
192024-12-21T00:48:08.925ZINFOcrucible-agent (dropshot): request completed latency_us = 88 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:54636 req_id = 32fb304b-1184-485a-8f4e-f9b54168f072 response_code = 200 uri = /crucible/0/regions
202024-12-21T00:48:09.075ZINFOcrucible-agent (worker): region files created ok region = 3d93ccd1-6827-47ba-b053-605b0ed644d1
212024-12-21T00:48:09.075ZINFOcrucible-agent (datafile): region 3d93ccd1-6827-47ba-b053-605b0ed644d1 state: Requested -> Created
222024-12-21T00:48:09.075ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-12-21T00:48:09.075ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-3d93ccd1-6827-47ba-b053-605b0ed644d1
242024-12-21T00:48:09.082ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-3d93ccd1-6827-47ba-b053-605b0ed644d1
252024-12-21T00:48:09.093ZINFOcrucible-agent (worker): creating config property group
262024-12-21T00:48:09.097ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-3d93ccd1-6827-47ba-b053-605b0ed644d1
272024-12-21T00:48:09.098ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1
282024-12-21T00:48:09.098ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-12-21T00:48:09.098ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
302024-12-21T00:48:09.098ZINFOcrucible-agent (worker): commit
312024-12-21T00:48:09.104ZINFOcrucible-agent (worker): ok!
322024-12-21T00:48:09.112ZINFOcrucible-agent (worker): SMF ok!
332024-12-21T00:48:09.575ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:54636 req_id = 225d9dc7-0662-4497-8c57-59c929f48300 response_code = 200 uri = /crucible/0/regions
342024-12-21T00:58:17.348ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:37049
352024-12-21T00:58:17.352ZINFOcrucible-agent (dropshot): request completed latency_us = 4502 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:37049 req_id = 5432b1dc-6fe2-492c-bd64-4f780f7520d1 response_code = 200 uri = /crucible/0/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1
362024-12-21T00:58:17.355ZINFOcrucible-agent (datafile): path is "/data/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1"
372024-12-21T00:58:17.365ZINFOcrucible-agent (datafile): dataset is oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1
382024-12-21T00:58:17.373ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1@99d9a509-237d-43cd-9c13-86aab0a26b82
392024-12-21T00:58:17.373ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1", "99d9a509-237d-43cd-9c13-86aab0a26b82"]
402024-12-21T00:58:17.387ZINFOcrucible-agent (dropshot): request completed latency_us = 31161 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:37049 req_id = 645514c7-bdf1-40a0-8a38-b611bac13697 response_code = 200 uri = /crucible/0/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1/snapshots/99d9a509-237d-43cd-9c13-86aab0a26b82
412024-12-21T00:58:17.391ZINFOcrucible-agent (datafile): path is "/data/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1"
422024-12-21T00:58:17.399ZINFOcrucible-agent (datafile): dataset is oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1
432024-12-21T00:58:17.407ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1@99d9a509-237d-43cd-9c13-86aab0a26b82
442024-12-21T00:58:17.407ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1", "99d9a509-237d-43cd-9c13-86aab0a26b82"]
452024-12-21T00:58:17.416ZINFOcrucible-agent (datafile): requesting running snapshot 3d93ccd1-6827-47ba-b053-605b0ed644d1-99d9a509-237d-43cd-9c13-86aab0a26b82 state: Requested
462024-12-21T00:58:17.420ZINFOcrucible-agent (worker): applying SMF actions for region 3d93ccd1-6827-47ba-b053-605b0ed644d1 running snapshot 99d9a509-237d-43cd-9c13-86aab0a26b82 (state Requested)...
472024-12-21T00:58:17.420ZINFOcrucible-agent (dropshot): request completed latency_us = 29868 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:37049 req_id = 36354967-2fe7-4b76-b354-b4bb2f769fa8 response_code = 200 uri = /crucible/0/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1/snapshots/99d9a509-237d-43cd-9c13-86aab0a26b82/run
482024-12-21T00:58:17.430ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-3d93ccd1-6827-47ba-b053-605b0ed644d1-99d9a509-237d-43cd-9c13-86aab0a26b82
492024-12-21T00:58:17.438ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-3d93ccd1-6827-47ba-b053-605b0ed644d1-99d9a509-237d-43cd-9c13-86aab0a26b82
502024-12-21T00:58:17.464ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-3d93ccd1-6827-47ba-b053-605b0ed644d1-99d9a509-237d-43cd-9c13-86aab0a26b82
512024-12-21T00:58:17.464ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1/.zfs/snapshot/99d9a509-237d-43cd-9c13-86aab0a26b82
522024-12-21T00:58:17.464ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
532024-12-21T00:58:17.464ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
542024-12-21T00:58:17.464ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
552024-12-21T00:58:17.464ZINFOcrucible-agent (worker): commit
562024-12-21T00:58:17.473ZINFOcrucible-agent (worker): ok!
572024-12-21T00:58:17.487ZINFOcrucible-agent (worker): SMF ok!
582024-12-21T00:58:17.487ZINFOcrucible-agent (datafile): region 3d93ccd1-6827-47ba-b053-605b0ed644d1 running snapshot 99d9a509-237d-43cd-9c13-86aab0a26b82 state: Requested -> Created
592024-12-21T00:58:17.594ZINFOcrucible-agent (datafile): path is "/data/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1"
602024-12-21T00:58:17.603ZINFOcrucible-agent (datafile): dataset is oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1
612024-12-21T00:58:17.611ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1@99d9a509-237d-43cd-9c13-86aab0a26b82
622024-12-21T00:58:17.611ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_590ffd4f-3c08-4aaa-9320-324eceef269f/crucible/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1", "99d9a509-237d-43cd-9c13-86aab0a26b82"]
632024-12-21T00:58:17.620ZINFOcrucible-agent (dropshot): request completed latency_us = 26476 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:37049 req_id = 02872fb9-e11c-4e71-bcfb-b12c617ee4fd response_code = 200 uri = /crucible/0/regions/3d93ccd1-6827-47ba-b053-605b0ed644d1/snapshots/99d9a509-237d-43cd-9c13-86aab0a26b82/run