Max level shown:
12024-05-15T07:18:11.143ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::a]:65290
22024-05-15T07:18:11.143ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:65290 req_id = ed747513-308e-468b-bd9c-249fcdbb7cc8 response_code = 200 uri = /crucible/0/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94
32024-05-15T07:18:11.146ZINFOcrucible-agent (datafile): path is "/data/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94"
42024-05-15T07:18:11.155ZINFOcrucible-agent (datafile): dataset is oxp_12d428b9-eedd-40c0-8928-1d92a73f181d/crucible/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94
52024-05-15T07:18:11.164ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_12d428b9-eedd-40c0-8928-1d92a73f181d/crucible/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94@2284b6d3-8a75-4527-b2c7-5464667ad0d9
62024-05-15T07:18:11.164ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_12d428b9-eedd-40c0-8928-1d92a73f181d/crucible/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94", "2284b6d3-8a75-4527-b2c7-5464667ad0d9"]
72024-05-15T07:18:11.178ZINFOcrucible-agent (dropshot): request completed latency_us = 32577 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:65290 req_id = 482784bd-01a3-4199-b262-c9bd1e3098e0 response_code = 200 uri = /crucible/0/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94/snapshots/2284b6d3-8a75-4527-b2c7-5464667ad0d9
82024-05-15T07:18:11.180ZINFOcrucible-agent (datafile): path is "/data/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94"
92024-05-15T07:18:11.189ZINFOcrucible-agent (datafile): dataset is oxp_12d428b9-eedd-40c0-8928-1d92a73f181d/crucible/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94
102024-05-15T07:18:11.197ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_12d428b9-eedd-40c0-8928-1d92a73f181d/crucible/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94@2284b6d3-8a75-4527-b2c7-5464667ad0d9
112024-05-15T07:18:11.197ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_12d428b9-eedd-40c0-8928-1d92a73f181d/crucible/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94", "2284b6d3-8a75-4527-b2c7-5464667ad0d9"]
122024-05-15T07:18:11.206ZINFOcrucible-agent (datafile): requesting running snapshot 249ae8f3-9e9e-4810-9977-8cb50b0cfa94-2284b6d3-8a75-4527-b2c7-5464667ad0d9 state: Requested
132024-05-15T07:18:11.206ZINFOcrucible-agent (worker): applying SMF actions for region 249ae8f3-9e9e-4810-9977-8cb50b0cfa94 running snapshot 2284b6d3-8a75-4527-b2c7-5464667ad0d9 (state Requested)...
142024-05-15T07:18:11.206ZINFOcrucible-agent (dropshot): request completed latency_us = 25707 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:65290 req_id = 32d6cd3d-9f4f-4383-9a6a-95e59f9f4bcc response_code = 200 uri = /crucible/0/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94/snapshots/2284b6d3-8a75-4527-b2c7-5464667ad0d9/run
152024-05-15T07:18:11.214ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-249ae8f3-9e9e-4810-9977-8cb50b0cfa94-2284b6d3-8a75-4527-b2c7-5464667ad0d9
162024-05-15T07:18:11.220ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-249ae8f3-9e9e-4810-9977-8cb50b0cfa94-2284b6d3-8a75-4527-b2c7-5464667ad0d9
172024-05-15T07:18:11.231ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-249ae8f3-9e9e-4810-9977-8cb50b0cfa94-2284b6d3-8a75-4527-b2c7-5464667ad0d9
182024-05-15T07:18:11.232ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/249ae8f3-9e9e-4810-9977-8cb50b0cfa94/.zfs/snapshot/2284b6d3-8a75-4527-b2c7-5464667ad0d9
192024-05-15T07:18:11.232ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
202024-05-15T07:18:11.232ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
212024-05-15T07:18:11.232ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
222024-05-15T07:18:11.232ZINFOcrucible-agent (worker): commit
232024-05-15T07:18:11.235ZINFOcrucible-agent (worker): ok!
242024-05-15T07:18:11.239ZINFOcrucible-agent (worker): SMF ok!
252024-05-15T07:18:11.239ZINFOcrucible-agent (datafile): region 249ae8f3-9e9e-4810-9977-8cb50b0cfa94 running snapshot 2284b6d3-8a75-4527-b2c7-5464667ad0d9 state: Requested -> Created
262024-05-15T07:18:14.143ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::a]:64277
272024-05-15T07:18:14.143ZINFOcrucible-agent (datafile): region 94cb7f1d-af2d-4f0a-a702-960d11798a0e state: Requested
282024-05-15T07:18:14.143ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
292024-05-15T07:18:14.143ZINFOcrucible-agent (dropshot): request completed latency_us = 209 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64277 req_id = 87b74f36-e8f6-4c76-9ec3-26e9d8a7ff37 response_code = 200 uri = /crucible/0/regions
302024-05-15T07:18:14.153ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_12d428b9-eedd-40c0-8928-1d92a73f181d/crucible/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e
312024-05-15T07:18:14.153ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_12d428b9-eedd-40c0-8928-1d92a73f181d/crucible/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e
322024-05-15T07:18:14.222ZINFOcrucible-agent (worker): creating region Region { id: RegionId("94cb7f1d-af2d-4f0a-a702-960d11798a0e"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e" region = 94cb7f1d-af2d-4f0a-a702-960d11798a0e
332024-05-15T07:18:14.222ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "94cb7f1d-af2d-4f0a-a702-960d11798a0e" "--data" "/data/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 94cb7f1d-af2d-4f0a-a702-960d11798a0e
342024-05-15T07:18:14.287ZINFOcrucible-agent (worker): region files created ok region = 94cb7f1d-af2d-4f0a-a702-960d11798a0e
352024-05-15T07:18:14.287ZINFOcrucible-agent (datafile): region 94cb7f1d-af2d-4f0a-a702-960d11798a0e state: Requested -> Created
362024-05-15T07:18:14.287ZINFOcrucible-agent (worker): applying SMF actions post create...
372024-05-15T07:18:14.289ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-94cb7f1d-af2d-4f0a-a702-960d11798a0e
382024-05-15T07:18:14.292ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-94cb7f1d-af2d-4f0a-a702-960d11798a0e
392024-05-15T07:18:14.299ZINFOcrucible-agent (worker): creating config property group
402024-05-15T07:18:14.302ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-94cb7f1d-af2d-4f0a-a702-960d11798a0e
412024-05-15T07:18:14.302ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e
422024-05-15T07:18:14.302ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
432024-05-15T07:18:14.302ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
442024-05-15T07:18:14.302ZINFOcrucible-agent (worker): commit
452024-05-15T07:18:14.305ZINFOcrucible-agent (worker): ok!
462024-05-15T07:18:14.311ZINFOcrucible-agent (worker): SMF ok!
472024-05-15T07:18:14.431ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64277 req_id = 92be8bb1-ae93-4447-8187-6e0640f635cb response_code = 200 uri = /crucible/0/regions
482024-05-15T07:18:38.061ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::a]:60171
492024-05-15T07:18:38.061ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60171 req_id = 06a01382-a86b-402d-9d73-72c38e3c2c79 response_code = 200 uri = /crucible/0/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e
502024-05-15T07:18:38.061ZINFOcrucible-agent (datafile): path is "/data/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e"
512024-05-15T07:18:38.073ZINFOcrucible-agent (datafile): dataset is oxp_12d428b9-eedd-40c0-8928-1d92a73f181d/crucible/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e
522024-05-15T07:18:38.081ZINFOcrucible-agent (datafile): region 94cb7f1d-af2d-4f0a-a702-960d11798a0e state: Created -> Tombstoned
532024-05-15T07:18:38.081ZINFOcrucible-agent (worker): applying SMF actions before removal...
542024-05-15T07:18:38.082ZINFOcrucible-agent (dropshot): request completed latency_us = 20534 local_addr = [fd00:1122:3344:101::13]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:60171 req_id = e885987f-6d0e-4fa0-bb13-99b21c281a7d response_code = 204 uri = /crucible/0/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e
552024-05-15T07:18:38.082ZINFOcrucible-agent (dropshot): request completed latency_us = 22 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60171 req_id = e1e3a434-9f69-41dd-87b7-b7c30e8c93ad response_code = 200 uri = /crucible/0/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e
562024-05-15T07:18:38.083ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-94cb7f1d-af2d-4f0a-a702-960d11798a0e (instance states: (Some(Online), None))
572024-05-15T07:18:38.089ZINFOcrucible-agent (worker): SMF ok!
582024-05-15T07:18:38.099ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_12d428b9-eedd-40c0-8928-1d92a73f181d/crucible/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e" region = 94cb7f1d-af2d-4f0a-a702-960d11798a0e
592024-05-15T07:18:38.167ZINFOcrucible-agent (datafile): region 94cb7f1d-af2d-4f0a-a702-960d11798a0e state: Tombstoned -> Destroyed
602024-05-15T07:18:38.197ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60171 req_id = 699ce062-166a-4db5-b570-ad58b29583da response_code = 200 uri = /crucible/0/regions/94cb7f1d-af2d-4f0a-a702-960d11798a0e