Max level shown:
1 [ Jan 7 05:07:54 Enabled. ]
2 [ Jan 7 05:07:54 Rereading configuration. ]
3 [ Jan 7 05:07:57 Rereading configuration. ]
4 [ Jan 7 05:08:02 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_cc33774c-0d42-4a7d-aa14-02e0ff49f2d2/crucible -l [fd00:1122:3344:101::19]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-01-07T05:08:02.126ZINFOcrucible-agent: dataset: "oxp_cc33774c-0d42-4a7d-aa14-02e0ff49f2d2/crucible"
72025-01-07T05:08:02.130ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::19]:32345
82025-01-07T05:08:02.130ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-01-07T05:08:02.152ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-01-07T05:08:02.209ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::19]:32345
112025-01-07T05:09:03.851ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::c]:48367
122025-01-07T05:09:03.877ZINFOcrucible-agent (datafile): region 646e59ee-cdc8-4ed1-bd7b-2771a97cfd68 state: Requested
132025-01-07T05:09:03.882ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142025-01-07T05:09:03.887ZINFOcrucible-agent (dropshot): request completed latency_us = 11657 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48367 req_id = 88588079-d673-47a6-a4a1-2d382703613b response_code = 200 uri = /crucible/0/regions
152025-01-07T05:09:03.894ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_cc33774c-0d42-4a7d-aa14-02e0ff49f2d2/crucible/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
162025-01-07T05:09:03.894ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_cc33774c-0d42-4a7d-aa14-02e0ff49f2d2/crucible/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
172025-01-07T05:09:03.954ZINFOcrucible-agent (worker): creating region Region { id: RegionId("646e59ee-cdc8-4ed1-bd7b-2771a97cfd68"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68" region = 646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
182025-01-07T05:09:03.956ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "646e59ee-cdc8-4ed1-bd7b-2771a97cfd68" "--data" "/data/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
192025-01-07T05:09:04.139ZINFOcrucible-agent (dropshot): request completed latency_us = 78 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48367 req_id = 1ffbce43-47b4-44ad-a561-bf610f88ca9a response_code = 200 uri = /crucible/0/regions
202025-01-07T05:09:04.170ZINFOcrucible-agent (worker): region files created ok region = 646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
212025-01-07T05:09:04.170ZINFOcrucible-agent (datafile): region 646e59ee-cdc8-4ed1-bd7b-2771a97cfd68 state: Requested -> Created
222025-01-07T05:09:04.171ZINFOcrucible-agent (worker): applying SMF actions post create...
232025-01-07T05:09:04.171ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
242025-01-07T05:09:04.175ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
252025-01-07T05:09:04.185ZINFOcrucible-agent (worker): creating config property group
262025-01-07T05:09:04.190ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
272025-01-07T05:09:04.191ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
282025-01-07T05:09:04.191ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292025-01-07T05:09:04.191ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
302025-01-07T05:09:04.191ZINFOcrucible-agent (worker): commit
312025-01-07T05:09:04.197ZINFOcrucible-agent (worker): ok!
322025-01-07T05:09:04.201ZINFOcrucible-agent (worker): SMF ok!
332025-01-07T05:09:04.535ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48367 req_id = 98a4bc8f-5cdd-4f09-9484-a72b81d13401 response_code = 200 uri = /crucible/0/regions
342025-01-07T05:09:04.909ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:48367 req_id = 4705254c-51fb-4057-8d82-bee4255de7c3 response_code = 200 uri = /crucible/0/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
352025-01-07T05:09:04.913ZINFOcrucible-agent (datafile): path is "/data/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68"
362025-01-07T05:09:04.925ZINFOcrucible-agent (datafile): dataset is oxp_cc33774c-0d42-4a7d-aa14-02e0ff49f2d2/crucible/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
372025-01-07T05:09:04.935ZINFOcrucible-agent (datafile): region 646e59ee-cdc8-4ed1-bd7b-2771a97cfd68 state: Created -> Tombstoned
382025-01-07T05:09:04.935ZINFOcrucible-agent (worker): applying SMF actions before removal...
392025-01-07T05:09:04.935ZINFOcrucible-agent (dropshot): request completed latency_us = 21841 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:48367 req_id = 26fe7e01-b10f-4c54-b392-a9ff18bf2c99 response_code = 204 uri = /crucible/0/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
402025-01-07T05:09:04.938ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-646e59ee-cdc8-4ed1-bd7b-2771a97cfd68 (instance states: (Some(Online), None))
412025-01-07T05:09:04.938ZINFOcrucible-agent (dropshot): request completed latency_us = 29 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:48367 req_id = adb45980-9b68-4987-983c-f1c62023d5dd response_code = 200 uri = /crucible/0/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
422025-01-07T05:09:04.944ZINFOcrucible-agent (worker): SMF ok!
432025-01-07T05:09:04.955ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_cc33774c-0d42-4a7d-aa14-02e0ff49f2d2/crucible/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68" region = 646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
442025-01-07T05:09:05.015ZINFOcrucible-agent (datafile): region 646e59ee-cdc8-4ed1-bd7b-2771a97cfd68 state: Tombstoned -> Destroyed
452025-01-07T05:09:05.082ZINFOcrucible-agent (dropshot): request completed latency_us = 81 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:48367 req_id = 81fe2190-3924-4e44-984c-cc109aa102f0 response_code = 200 uri = /crucible/0/regions/646e59ee-cdc8-4ed1-bd7b-2771a97cfd68
462025-01-07T05:09:11.143ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:63276
472025-01-07T05:09:11.143ZINFOcrucible-agent (datafile): region 8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0 state: Requested
482025-01-07T05:09:11.143ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492025-01-07T05:09:11.143ZINFOcrucible-agent (dropshot): request completed latency_us = 214 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63276 req_id = b8398185-f2a0-44ca-97bd-8119cf81b7fd response_code = 200 uri = /crucible/0/regions
502025-01-07T05:09:11.152ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_cc33774c-0d42-4a7d-aa14-02e0ff49f2d2/crucible/regions/8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0
512025-01-07T05:09:11.152ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_cc33774c-0d42-4a7d-aa14-02e0ff49f2d2/crucible/regions/8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0
522025-01-07T05:09:11.201ZINFOcrucible-agent (worker): creating region Region { id: RegionId("8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0" region = 8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0
532025-01-07T05:09:11.203ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0" "--data" "/data/regions/8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0
542025-01-07T05:09:11.281ZINFOcrucible-agent (worker): region files created ok region = 8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0
552025-01-07T05:09:11.281ZINFOcrucible-agent (datafile): region 8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0 state: Requested -> Created
562025-01-07T05:09:11.281ZINFOcrucible-agent (worker): applying SMF actions post create...
572025-01-07T05:09:11.283ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0
582025-01-07T05:09:11.286ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0
592025-01-07T05:09:11.288ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63276 req_id = 83178dc1-f55d-49f9-9eb5-1113d66b7fe9 response_code = 200 uri = /crucible/0/regions
602025-01-07T05:09:11.293ZINFOcrucible-agent (worker): creating config property group
612025-01-07T05:09:11.297ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0
622025-01-07T05:09:11.297ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8e13d8ee-ef5e-497c-b4d7-4bd02d0af3d0
632025-01-07T05:09:11.297ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
642025-01-07T05:09:11.297ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
652025-01-07T05:09:11.297ZINFOcrucible-agent (worker): commit
662025-01-07T05:09:11.305ZINFOcrucible-agent (worker): ok!
672025-01-07T05:09:11.311ZINFOcrucible-agent (worker): SMF ok!