Max level shown:
12024-02-15T06:05:09.635ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:35813
22024-02-15T06:05:09.635ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35813 req_id = 12b87381-e419-4d31-a1ab-404315f5d273 response_code = 200 uri = /crucible/0/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639
32024-02-15T06:05:09.639ZINFOcrucible-agent (datafile): path is "/data/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639"
42024-02-15T06:05:09.648ZINFOcrucible-agent (datafile): dataset is oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639
52024-02-15T06:05:09.657ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639@26054eb5-467c-4ad3-988b-6fd144d2b756
62024-02-15T06:05:09.657ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639", "26054eb5-467c-4ad3-988b-6fd144d2b756"]
72024-02-15T06:05:09.669ZINFOcrucible-agent (dropshot): request completed latency_us = 29268 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35813 req_id = 7b6f85ff-2afe-413f-922f-2bbb9a7f30e8 response_code = 200 uri = /crucible/0/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639/snapshots/26054eb5-467c-4ad3-988b-6fd144d2b756
82024-02-15T06:05:09.669ZINFOcrucible-agent (datafile): path is "/data/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639"
92024-02-15T06:05:09.677ZINFOcrucible-agent (datafile): dataset is oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639
102024-02-15T06:05:09.684ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639@26054eb5-467c-4ad3-988b-6fd144d2b756
112024-02-15T06:05:09.684ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639", "26054eb5-467c-4ad3-988b-6fd144d2b756"]
122024-02-15T06:05:09.693ZINFOcrucible-agent (datafile): requesting running snapshot d81eaef2-ffde-4bdf-bacb-f25f75065639-26054eb5-467c-4ad3-988b-6fd144d2b756 state: Requested
132024-02-15T06:05:09.693ZINFOcrucible-agent (worker): applying SMF actions for region d81eaef2-ffde-4bdf-bacb-f25f75065639 running snapshot 26054eb5-467c-4ad3-988b-6fd144d2b756 (state Requested)...
142024-02-15T06:05:09.693ZINFOcrucible-agent (dropshot): request completed latency_us = 24602 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35813 req_id = 47832331-9be8-47aa-addb-45f41e80acd1 response_code = 200 uri = /crucible/0/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639/snapshots/26054eb5-467c-4ad3-988b-6fd144d2b756/run
152024-02-15T06:05:09.704ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-d81eaef2-ffde-4bdf-bacb-f25f75065639-26054eb5-467c-4ad3-988b-6fd144d2b756
162024-02-15T06:05:09.709ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-d81eaef2-ffde-4bdf-bacb-f25f75065639-26054eb5-467c-4ad3-988b-6fd144d2b756
172024-02-15T06:05:09.720ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-d81eaef2-ffde-4bdf-bacb-f25f75065639-26054eb5-467c-4ad3-988b-6fd144d2b756
182024-02-15T06:05:09.720ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d81eaef2-ffde-4bdf-bacb-f25f75065639/.zfs/snapshot/26054eb5-467c-4ad3-988b-6fd144d2b756
192024-02-15T06:05:09.720ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
202024-02-15T06:05:09.720ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
212024-02-15T06:05:09.720ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
222024-02-15T06:05:09.721ZINFOcrucible-agent (worker): commit
232024-02-15T06:05:09.725ZINFOcrucible-agent (worker): ok!
242024-02-15T06:05:09.731ZINFOcrucible-agent (worker): SMF ok!
252024-02-15T06:05:09.731ZINFOcrucible-agent (datafile): region d81eaef2-ffde-4bdf-bacb-f25f75065639 running snapshot 26054eb5-467c-4ad3-988b-6fd144d2b756 state: Requested -> Created
262024-02-15T06:05:13.126ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:61812
272024-02-15T06:05:13.126ZINFOcrucible-agent (datafile): region ce1e4a26-d199-4e3b-8aa1-836674544624 state: Requested
282024-02-15T06:05:13.126ZINFOcrucible-agent (dropshot): request completed latency_us = 246 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61812 req_id = 84a72d7b-8ac0-4ba2-a542-6b688abd540d response_code = 200 uri = /crucible/0/regions
292024-02-15T06:05:13.126ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
302024-02-15T06:05:13.136ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible/regions/ce1e4a26-d199-4e3b-8aa1-836674544624
312024-02-15T06:05:13.136ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible/regions/ce1e4a26-d199-4e3b-8aa1-836674544624
322024-02-15T06:05:13.212ZINFOcrucible-agent (worker): creating region Region { id: RegionId("ce1e4a26-d199-4e3b-8aa1-836674544624"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None } at "/data/regions/ce1e4a26-d199-4e3b-8aa1-836674544624" region = ce1e4a26-d199-4e3b-8aa1-836674544624
332024-02-15T06:05:13.283ZINFOcrucible-agent (worker): region files created ok region = ce1e4a26-d199-4e3b-8aa1-836674544624
342024-02-15T06:05:13.283ZINFOcrucible-agent (datafile): region ce1e4a26-d199-4e3b-8aa1-836674544624 state: Requested -> Created
352024-02-15T06:05:13.283ZINFOcrucible-agent (worker): applying SMF actions post create...
362024-02-15T06:05:13.284ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-ce1e4a26-d199-4e3b-8aa1-836674544624
372024-02-15T06:05:13.291ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-ce1e4a26-d199-4e3b-8aa1-836674544624
382024-02-15T06:05:13.300ZINFOcrucible-agent (worker): creating config property group
392024-02-15T06:05:13.304ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-ce1e4a26-d199-4e3b-8aa1-836674544624
402024-02-15T06:05:13.304ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ce1e4a26-d199-4e3b-8aa1-836674544624
412024-02-15T06:05:13.304ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
422024-02-15T06:05:13.304ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
432024-02-15T06:05:13.304ZINFOcrucible-agent (worker): commit
442024-02-15T06:05:13.308ZINFOcrucible-agent (worker): ok!
452024-02-15T06:05:13.319ZINFOcrucible-agent (worker): SMF ok!
462024-02-15T06:05:13.348ZINFOcrucible-agent (dropshot): request completed latency_us = 102 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61812 req_id = d7d66b84-d6be-4126-903d-7a64e006f8b4 response_code = 200 uri = /crucible/0/regions
472024-02-15T06:05:46.809ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:59415
482024-02-15T06:05:46.809ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59415 req_id = 5524a8a2-cd59-4976-9f61-65a7678fd0ec response_code = 200 uri = /crucible/0/regions/ce1e4a26-d199-4e3b-8aa1-836674544624
492024-02-15T06:05:46.810ZINFOcrucible-agent (datafile): path is "/data/regions/ce1e4a26-d199-4e3b-8aa1-836674544624"
502024-02-15T06:05:46.826ZINFOcrucible-agent (datafile): dataset is oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible/regions/ce1e4a26-d199-4e3b-8aa1-836674544624
512024-02-15T06:05:46.835ZINFOcrucible-agent (datafile): region ce1e4a26-d199-4e3b-8aa1-836674544624 state: Created -> Tombstoned
522024-02-15T06:05:46.835ZINFOcrucible-agent (worker): applying SMF actions before removal...
532024-02-15T06:05:46.835ZINFOcrucible-agent (dropshot): request completed latency_us = 25914 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:59415 req_id = 0f3da26b-cac2-484c-8163-29c11fcaad0b response_code = 204 uri = /crucible/0/regions/ce1e4a26-d199-4e3b-8aa1-836674544624
542024-02-15T06:05:46.835ZINFOcrucible-agent (dropshot): request completed latency_us = 22 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59415 req_id = af222fc3-7dd7-4b71-b955-689b26c7a214 response_code = 200 uri = /crucible/0/regions/ce1e4a26-d199-4e3b-8aa1-836674544624
552024-02-15T06:05:46.842ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-ce1e4a26-d199-4e3b-8aa1-836674544624 (instance states: (Some(Online), None))
562024-02-15T06:05:46.856ZINFOcrucible-agent (worker): SMF ok!
572024-02-15T06:05:46.865ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_31bd71cd-4736-4a12-a387-9b74b050396f/crucible/regions/ce1e4a26-d199-4e3b-8aa1-836674544624" region = ce1e4a26-d199-4e3b-8aa1-836674544624
582024-02-15T06:05:46.936ZINFOcrucible-agent (dropshot): request completed latency_us = 88 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59415 req_id = 9325ccb1-67a3-4a27-87ae-ecdd68c7e430 response_code = 200 uri = /crucible/0/regions/ce1e4a26-d199-4e3b-8aa1-836674544624
592024-02-15T06:05:46.951ZINFOcrucible-agent (datafile): region ce1e4a26-d199-4e3b-8aa1-836674544624 state: Tombstoned -> Destroyed
602024-02-15T06:05:47.016ZINFOcrucible-agent (dropshot): request completed latency_us = 38 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59415 req_id = a5a8268e-5ec0-47ea-8b23-d3af31c53d39 response_code = 200 uri = /crucible/0/regions/ce1e4a26-d199-4e3b-8aa1-836674544624