Max level shown:
12024-06-25T07:16:59.477ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:52944
22024-06-25T07:16:59.477ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:52944 req_id = c393d535-953d-4414-ba19-0c34843d6539 response_code = 200 uri = /crucible/0/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36
32024-06-25T07:16:59.481ZINFOcrucible-agent (datafile): path is "/data/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36"
42024-06-25T07:16:59.491ZINFOcrucible-agent (datafile): dataset is oxp_eba9a72e-79ea-4722-8854-b5f41ec1c20b/crucible/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36
52024-06-25T07:16:59.500ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_eba9a72e-79ea-4722-8854-b5f41ec1c20b/crucible/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36@568d1eaa-f10d-42de-8a04-4eeefc3e9c42
62024-06-25T07:16:59.500ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_eba9a72e-79ea-4722-8854-b5f41ec1c20b/crucible/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36", "568d1eaa-f10d-42de-8a04-4eeefc3e9c42"]
72024-06-25T07:16:59.514ZINFOcrucible-agent (dropshot): request completed latency_us = 36669 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:52944 req_id = 81fa60fe-20f0-496a-9653-4dc54dd126d6 response_code = 200 uri = /crucible/0/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36/snapshots/568d1eaa-f10d-42de-8a04-4eeefc3e9c42
82024-06-25T07:16:59.514ZINFOcrucible-agent (datafile): path is "/data/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36"
92024-06-25T07:16:59.523ZINFOcrucible-agent (datafile): dataset is oxp_eba9a72e-79ea-4722-8854-b5f41ec1c20b/crucible/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36
102024-06-25T07:16:59.531ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_eba9a72e-79ea-4722-8854-b5f41ec1c20b/crucible/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36@568d1eaa-f10d-42de-8a04-4eeefc3e9c42
112024-06-25T07:16:59.531ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_eba9a72e-79ea-4722-8854-b5f41ec1c20b/crucible/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36", "568d1eaa-f10d-42de-8a04-4eeefc3e9c42"]
122024-06-25T07:16:59.540ZINFOcrucible-agent (datafile): requesting running snapshot b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36-568d1eaa-f10d-42de-8a04-4eeefc3e9c42 state: Requested
132024-06-25T07:16:59.540ZINFOcrucible-agent (worker): applying SMF actions for region b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36 running snapshot 568d1eaa-f10d-42de-8a04-4eeefc3e9c42 (state Requested)...
142024-06-25T07:16:59.540ZINFOcrucible-agent (dropshot): request completed latency_us = 25804 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52944 req_id = fe11664c-9a78-45af-9d92-6657d1a5ac65 response_code = 200 uri = /crucible/0/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36/snapshots/568d1eaa-f10d-42de-8a04-4eeefc3e9c42/run
152024-06-25T07:16:59.545ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36-568d1eaa-f10d-42de-8a04-4eeefc3e9c42
162024-06-25T07:16:59.552ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36-568d1eaa-f10d-42de-8a04-4eeefc3e9c42
172024-06-25T07:16:59.564ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36-568d1eaa-f10d-42de-8a04-4eeefc3e9c42
182024-06-25T07:16:59.564ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36/.zfs/snapshot/568d1eaa-f10d-42de-8a04-4eeefc3e9c42
192024-06-25T07:16:59.564ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
202024-06-25T07:16:59.564ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
212024-06-25T07:16:59.565ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
222024-06-25T07:16:59.565ZINFOcrucible-agent (worker): commit
232024-06-25T07:16:59.569ZINFOcrucible-agent (worker): ok!
242024-06-25T07:16:59.573ZINFOcrucible-agent (worker): SMF ok!
252024-06-25T07:16:59.573ZINFOcrucible-agent (datafile): region b4a3dc29-7432-4e7b-9e28-3e26b1ddfd36 running snapshot 568d1eaa-f10d-42de-8a04-4eeefc3e9c42 state: Requested -> Created
262024-06-25T07:17:02.625ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:59683
272024-06-25T07:17:02.625ZINFOcrucible-agent (datafile): region 26fc5bfd-7828-40bd-946b-4978f48581f8 state: Requested
282024-06-25T07:17:02.625ZINFOcrucible-agent (dropshot): request completed latency_us = 211 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59683 req_id = 017922e8-e666-4f4e-ad9f-711bb721202e response_code = 200 uri = /crucible/0/regions
292024-06-25T07:17:02.625ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
302024-06-25T07:17:02.634ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_eba9a72e-79ea-4722-8854-b5f41ec1c20b/crucible/regions/26fc5bfd-7828-40bd-946b-4978f48581f8
312024-06-25T07:17:02.634ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_eba9a72e-79ea-4722-8854-b5f41ec1c20b/crucible/regions/26fc5bfd-7828-40bd-946b-4978f48581f8
322024-06-25T07:17:02.691ZINFOcrucible-agent (worker): creating region Region { id: RegionId("26fc5bfd-7828-40bd-946b-4978f48581f8"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/26fc5bfd-7828-40bd-946b-4978f48581f8" region = 26fc5bfd-7828-40bd-946b-4978f48581f8
332024-06-25T07:17:02.691ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "26fc5bfd-7828-40bd-946b-4978f48581f8" "--data" "/data/regions/26fc5bfd-7828-40bd-946b-4978f48581f8" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 26fc5bfd-7828-40bd-946b-4978f48581f8
342024-06-25T07:17:02.759ZINFOcrucible-agent (dropshot): request completed latency_us = 601 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59683 req_id = 976700b4-e02f-4252-8ed9-190db9e1a311 response_code = 200 uri = /crucible/0/regions
352024-06-25T07:17:02.783ZINFOcrucible-agent (worker): region files created ok region = 26fc5bfd-7828-40bd-946b-4978f48581f8
362024-06-25T07:17:02.783ZINFOcrucible-agent (datafile): region 26fc5bfd-7828-40bd-946b-4978f48581f8 state: Requested -> Created
372024-06-25T07:17:02.783ZINFOcrucible-agent (worker): applying SMF actions post create...
382024-06-25T07:17:02.784ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-26fc5bfd-7828-40bd-946b-4978f48581f8
392024-06-25T07:17:02.787ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-26fc5bfd-7828-40bd-946b-4978f48581f8
402024-06-25T07:17:02.795ZINFOcrucible-agent (worker): creating config property group
412024-06-25T07:17:02.799ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-26fc5bfd-7828-40bd-946b-4978f48581f8
422024-06-25T07:17:02.799ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/26fc5bfd-7828-40bd-946b-4978f48581f8
432024-06-25T07:17:02.799ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
442024-06-25T07:17:02.799ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
452024-06-25T07:17:02.799ZINFOcrucible-agent (worker): commit
462024-06-25T07:17:02.803ZINFOcrucible-agent (worker): ok!
472024-06-25T07:17:02.813ZINFOcrucible-agent (worker): SMF ok!
482024-06-25T07:17:03.022ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59683 req_id = ad616993-4d35-446c-8502-47478b19d4e3 response_code = 200 uri = /crucible/0/regions
492024-06-25T07:17:26.266ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59683 req_id = edb551e4-7d96-4c16-84d6-53b93a2afa2b response_code = 200 uri = /crucible/0/regions/26fc5bfd-7828-40bd-946b-4978f48581f8
502024-06-25T07:17:26.268ZINFOcrucible-agent (datafile): path is "/data/regions/26fc5bfd-7828-40bd-946b-4978f48581f8"
512024-06-25T07:17:26.278ZINFOcrucible-agent (datafile): dataset is oxp_eba9a72e-79ea-4722-8854-b5f41ec1c20b/crucible/regions/26fc5bfd-7828-40bd-946b-4978f48581f8
522024-06-25T07:17:26.287ZINFOcrucible-agent (datafile): region 26fc5bfd-7828-40bd-946b-4978f48581f8 state: Created -> Tombstoned
532024-06-25T07:17:26.287ZINFOcrucible-agent (worker): applying SMF actions before removal...
542024-06-25T07:17:26.287ZINFOcrucible-agent (dropshot): request completed latency_us = 18592 local_addr = [fd00:1122:3344:101::17]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:59683 req_id = a80f181c-6d9a-4451-b045-0b9b249c61f3 response_code = 204 uri = /crucible/0/regions/26fc5bfd-7828-40bd-946b-4978f48581f8
552024-06-25T07:17:26.289ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-26fc5bfd-7828-40bd-946b-4978f48581f8 (instance states: (Some(Online), None))
562024-06-25T07:17:26.293ZINFOcrucible-agent (dropshot): request completed latency_us = 654 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59683 req_id = 01ffaec9-f7ee-480c-8f3c-be60d64f6cb0 response_code = 200 uri = /crucible/0/regions/26fc5bfd-7828-40bd-946b-4978f48581f8
572024-06-25T07:17:26.295ZINFOcrucible-agent (worker): SMF ok!
582024-06-25T07:17:26.305ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_eba9a72e-79ea-4722-8854-b5f41ec1c20b/crucible/regions/26fc5bfd-7828-40bd-946b-4978f48581f8" region = 26fc5bfd-7828-40bd-946b-4978f48581f8
592024-06-25T07:17:26.351ZINFOcrucible-agent (datafile): region 26fc5bfd-7828-40bd-946b-4978f48581f8 state: Tombstoned -> Destroyed
602024-06-25T07:17:26.424ZINFOcrucible-agent (dropshot): request completed latency_us = 50 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59683 req_id = fbffa6b6-da60-4989-96e4-59ef0648c541 response_code = 200 uri = /crucible/0/regions/26fc5bfd-7828-40bd-946b-4978f48581f8