Max level shown:
12024-08-23T22:03:49.788ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::a]:48977
22024-08-23T22:03:49.788ZINFOcrucible-agent (datafile): region aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a state: Requested
32024-08-23T22:03:49.789ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42024-08-23T22:03:49.789ZINFOcrucible-agent (dropshot): request completed latency_us = 743 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48977 req_id = e3ece014-e23c-479e-b762-29c2c4d1cbde response_code = 200 uri = /crucible/0/regions
52024-08-23T22:03:49.800ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a
62024-08-23T22:03:49.800ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a
72024-08-23T22:03:49.872ZINFOcrucible-agent (worker): creating region Region { id: RegionId("aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a" region = aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a
82024-08-23T22:03:49.872ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a" "--data" "/data/regions/aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a
92024-08-23T22:03:49.994ZINFOcrucible-agent (worker): region files created ok region = aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a
102024-08-23T22:03:49.994ZINFOcrucible-agent (datafile): region aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a state: Requested -> Created
112024-08-23T22:03:49.994ZINFOcrucible-agent (worker): applying SMF actions post create...
122024-08-23T22:03:50.001ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a
132024-08-23T22:03:50.016ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a
142024-08-23T22:03:50.039ZINFOcrucible-agent (worker): creating config property group
152024-08-23T22:03:50.049ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a
162024-08-23T22:03:50.050ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/aa5fa2b5-77b8-4f08-934d-4bc3563f2e6a
172024-08-23T22:03:50.050ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182024-08-23T22:03:50.050ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
192024-08-23T22:03:50.050ZINFOcrucible-agent (worker): commit
202024-08-23T22:03:50.059ZINFOcrucible-agent (worker): ok!
212024-08-23T22:03:50.066ZINFOcrucible-agent (worker): SMF ok!
222024-08-23T22:03:50.086ZINFOcrucible-agent (dropshot): request completed latency_us = 128 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48977 req_id = b8b8385a-a446-4f1e-abc7-fd77e6019f30 response_code = 200 uri = /crucible/0/regions
232024-08-23T22:03:50.612ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48977 req_id = 81b08e67-ad3c-4fe4-8aa6-e7dd5c046bd0 response_code = 200 uri = /crucible/0/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9
242024-08-23T22:03:50.618ZINFOcrucible-agent (datafile): path is "/data/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9"
252024-08-23T22:03:50.628ZINFOcrucible-agent (datafile): dataset is oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9
262024-08-23T22:03:50.637ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9@1a76e494-6783-46a7-a79f-50626afc5c9d
272024-08-23T22:03:50.637ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9", "1a76e494-6783-46a7-a79f-50626afc5c9d"]
282024-08-23T22:03:50.648ZINFOcrucible-agent (dropshot): request completed latency_us = 33009 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48977 req_id = d3cf4966-33be-4b24-98fb-73041d3497eb response_code = 200 uri = /crucible/0/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9/snapshots/1a76e494-6783-46a7-a79f-50626afc5c9d
292024-08-23T22:03:50.651ZINFOcrucible-agent (datafile): path is "/data/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9"
302024-08-23T22:03:50.658ZINFOcrucible-agent (datafile): dataset is oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9
312024-08-23T22:03:50.667ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9@1a76e494-6783-46a7-a79f-50626afc5c9d
322024-08-23T22:03:50.667ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9", "1a76e494-6783-46a7-a79f-50626afc5c9d"]
332024-08-23T22:03:50.675ZINFOcrucible-agent (datafile): requesting running snapshot 286c6c79-8c08-440f-93a7-8a8e700dd4e9-1a76e494-6783-46a7-a79f-50626afc5c9d state: Requested
342024-08-23T22:03:50.675ZINFOcrucible-agent (worker): applying SMF actions for region 286c6c79-8c08-440f-93a7-8a8e700dd4e9 running snapshot 1a76e494-6783-46a7-a79f-50626afc5c9d (state Requested)...
352024-08-23T22:03:50.675ZINFOcrucible-agent (dropshot): request completed latency_us = 24007 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48977 req_id = d9220c0f-0106-4085-9d09-5e381379675e response_code = 200 uri = /crucible/0/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9/snapshots/1a76e494-6783-46a7-a79f-50626afc5c9d/run
362024-08-23T22:03:50.678ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-286c6c79-8c08-440f-93a7-8a8e700dd4e9-1a76e494-6783-46a7-a79f-50626afc5c9d
372024-08-23T22:03:50.684ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-286c6c79-8c08-440f-93a7-8a8e700dd4e9-1a76e494-6783-46a7-a79f-50626afc5c9d
382024-08-23T22:03:50.702ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-286c6c79-8c08-440f-93a7-8a8e700dd4e9-1a76e494-6783-46a7-a79f-50626afc5c9d
392024-08-23T22:03:50.703ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9/.zfs/snapshot/1a76e494-6783-46a7-a79f-50626afc5c9d
402024-08-23T22:03:50.703ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412024-08-23T22:03:50.703ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422024-08-23T22:03:50.703ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
432024-08-23T22:03:50.703ZINFOcrucible-agent (worker): commit
442024-08-23T22:03:50.711ZINFOcrucible-agent (worker): ok!
452024-08-23T22:03:50.719ZINFOcrucible-agent (worker): SMF ok!
462024-08-23T22:03:50.719ZINFOcrucible-agent (datafile): region 286c6c79-8c08-440f-93a7-8a8e700dd4e9 running snapshot 1a76e494-6783-46a7-a79f-50626afc5c9d state: Requested -> Created
472024-08-23T22:03:50.944ZINFOcrucible-agent (datafile): path is "/data/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9"
482024-08-23T22:03:50.954ZINFOcrucible-agent (datafile): dataset is oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9
492024-08-23T22:03:50.963ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9@1a76e494-6783-46a7-a79f-50626afc5c9d
502024-08-23T22:03:50.963ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ac03450-9a27-4ec4-9669-9eb798da4419/crucible/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9", "1a76e494-6783-46a7-a79f-50626afc5c9d"]
512024-08-23T22:03:50.971ZINFOcrucible-agent (dropshot): request completed latency_us = 26625 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48977 req_id = 903a9a7e-ef92-4847-a007-832dd5bd7334 response_code = 200 uri = /crucible/0/regions/286c6c79-8c08-440f-93a7-8a8e700dd4e9/snapshots/1a76e494-6783-46a7-a79f-50626afc5c9d/run