Max level shown:
12024-08-09T21:18:55.640ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:35127
22024-08-09T21:18:55.640ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35127 req_id = 7c397f90-ccab-4bbb-a838-c7c7497b87df response_code = 200 uri = /crucible/0/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c
32024-08-09T21:18:55.646ZINFOcrucible-agent (datafile): path is "/data/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c"
42024-08-09T21:18:55.655ZINFOcrucible-agent (datafile): dataset is oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c
52024-08-09T21:18:55.663ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c@1f5b562c-6f27-4b6c-b741-f40becb84c6b
62024-08-09T21:18:55.663ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c", "1f5b562c-6f27-4b6c-b741-f40becb84c6b"]
72024-08-09T21:18:55.676ZINFOcrucible-agent (dropshot): request completed latency_us = 32909 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35127 req_id = 26be72f9-289b-4bbc-86f4-a88e7cb72fab response_code = 200 uri = /crucible/0/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c/snapshots/1f5b562c-6f27-4b6c-b741-f40becb84c6b
82024-08-09T21:18:55.679ZINFOcrucible-agent (datafile): path is "/data/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c"
92024-08-09T21:18:55.688ZINFOcrucible-agent (datafile): dataset is oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c
102024-08-09T21:18:55.696ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c@1f5b562c-6f27-4b6c-b741-f40becb84c6b
112024-08-09T21:18:55.696ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c", "1f5b562c-6f27-4b6c-b741-f40becb84c6b"]
122024-08-09T21:18:55.704ZINFOcrucible-agent (datafile): requesting running snapshot 4c8c9fcf-617c-4e28-b9ca-384c3eec448c-1f5b562c-6f27-4b6c-b741-f40becb84c6b state: Requested
132024-08-09T21:18:55.705ZINFOcrucible-agent (worker): applying SMF actions for region 4c8c9fcf-617c-4e28-b9ca-384c3eec448c running snapshot 1f5b562c-6f27-4b6c-b741-f40becb84c6b (state Requested)...
142024-08-09T21:18:55.705ZINFOcrucible-agent (dropshot): request completed latency_us = 26106 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35127 req_id = 0d3a3ed1-a7c5-4d8b-9188-9508ac0e1203 response_code = 200 uri = /crucible/0/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c/snapshots/1f5b562c-6f27-4b6c-b741-f40becb84c6b/run
152024-08-09T21:18:55.712ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-4c8c9fcf-617c-4e28-b9ca-384c3eec448c-1f5b562c-6f27-4b6c-b741-f40becb84c6b
162024-08-09T21:18:55.723ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-4c8c9fcf-617c-4e28-b9ca-384c3eec448c-1f5b562c-6f27-4b6c-b741-f40becb84c6b
172024-08-09T21:18:55.745ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-4c8c9fcf-617c-4e28-b9ca-384c3eec448c-1f5b562c-6f27-4b6c-b741-f40becb84c6b
182024-08-09T21:18:55.745ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c/.zfs/snapshot/1f5b562c-6f27-4b6c-b741-f40becb84c6b
192024-08-09T21:18:55.745ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
202024-08-09T21:18:55.745ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
212024-08-09T21:18:55.745ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
222024-08-09T21:18:55.745ZINFOcrucible-agent (worker): commit
232024-08-09T21:18:55.752ZINFOcrucible-agent (worker): ok!
242024-08-09T21:18:55.760ZINFOcrucible-agent (worker): SMF ok!
252024-08-09T21:18:55.760ZINFOcrucible-agent (datafile): region 4c8c9fcf-617c-4e28-b9ca-384c3eec448c running snapshot 1f5b562c-6f27-4b6c-b741-f40becb84c6b state: Requested -> Created
262024-08-09T21:18:56.053ZINFOcrucible-agent (datafile): path is "/data/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c"
272024-08-09T21:18:56.063ZINFOcrucible-agent (datafile): dataset is oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c
282024-08-09T21:18:56.072ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c@1f5b562c-6f27-4b6c-b741-f40becb84c6b
292024-08-09T21:18:56.072ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c", "1f5b562c-6f27-4b6c-b741-f40becb84c6b"]
302024-08-09T21:18:56.080ZINFOcrucible-agent (dropshot): request completed latency_us = 27064 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35127 req_id = 8ce7c9f1-4592-49d6-b3d1-277b2df1bf0a response_code = 200 uri = /crucible/0/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c/snapshots/1f5b562c-6f27-4b6c-b741-f40becb84c6b/run
312024-08-09T21:18:59.002ZINFOcrucible-agent (datafile): region 19c5874e-967f-4ef9-8099-3765d0604df5 state: Requested
322024-08-09T21:18:59.002ZINFOcrucible-agent (dropshot): request completed latency_us = 245 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35127 req_id = a926b844-682e-4cd5-90d6-3f67643c519b response_code = 200 uri = /crucible/0/regions
332024-08-09T21:18:59.002ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
342024-08-09T21:18:59.012ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/19c5874e-967f-4ef9-8099-3765d0604df5
352024-08-09T21:18:59.012ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/19c5874e-967f-4ef9-8099-3765d0604df5
362024-08-09T21:18:59.080ZINFOcrucible-agent (worker): creating region Region { id: RegionId("19c5874e-967f-4ef9-8099-3765d0604df5"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/19c5874e-967f-4ef9-8099-3765d0604df5" region = 19c5874e-967f-4ef9-8099-3765d0604df5
372024-08-09T21:18:59.080ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "19c5874e-967f-4ef9-8099-3765d0604df5" "--data" "/data/regions/19c5874e-967f-4ef9-8099-3765d0604df5" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 19c5874e-967f-4ef9-8099-3765d0604df5
382024-08-09T21:18:59.191ZINFOcrucible-agent (worker): region files created ok region = 19c5874e-967f-4ef9-8099-3765d0604df5
392024-08-09T21:18:59.191ZINFOcrucible-agent (datafile): region 19c5874e-967f-4ef9-8099-3765d0604df5 state: Requested -> Created
402024-08-09T21:18:59.191ZINFOcrucible-agent (worker): applying SMF actions post create...
412024-08-09T21:18:59.192ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-19c5874e-967f-4ef9-8099-3765d0604df5
422024-08-09T21:18:59.195ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35127 req_id = 8999db62-4b8e-4d50-a2b4-12486519196e response_code = 200 uri = /crucible/0/regions
432024-08-09T21:18:59.205ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-19c5874e-967f-4ef9-8099-3765d0604df5
442024-08-09T21:18:59.236ZINFOcrucible-agent (worker): creating config property group
452024-08-09T21:18:59.250ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-19c5874e-967f-4ef9-8099-3765d0604df5
462024-08-09T21:18:59.250ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/19c5874e-967f-4ef9-8099-3765d0604df5
472024-08-09T21:18:59.250ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
482024-08-09T21:18:59.250ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
492024-08-09T21:18:59.250ZINFOcrucible-agent (worker): commit
502024-08-09T21:18:59.258ZINFOcrucible-agent (worker): ok!
512024-08-09T21:18:59.272ZINFOcrucible-agent (worker): SMF ok!
522024-08-09T21:19:24.273ZINFOcrucible-agent (dropshot): request completed latency_us = 639 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35127 req_id = e152646e-d9d3-41cf-9acc-7970b742c361 response_code = 200 uri = /crucible/0/regions/19c5874e-967f-4ef9-8099-3765d0604df5
532024-08-09T21:19:24.276ZINFOcrucible-agent (datafile): path is "/data/regions/19c5874e-967f-4ef9-8099-3765d0604df5"
542024-08-09T21:19:24.287ZINFOcrucible-agent (datafile): dataset is oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/19c5874e-967f-4ef9-8099-3765d0604df5
552024-08-09T21:19:24.296ZINFOcrucible-agent (datafile): region 19c5874e-967f-4ef9-8099-3765d0604df5 state: Created -> Tombstoned
562024-08-09T21:19:24.297ZINFOcrucible-agent (worker): applying SMF actions before removal...
572024-08-09T21:19:24.297ZINFOcrucible-agent (dropshot): request completed latency_us = 21080 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:35127 req_id = c8372163-f905-4a03-85de-818d1916e2de response_code = 204 uri = /crucible/0/regions/19c5874e-967f-4ef9-8099-3765d0604df5
582024-08-09T21:19:24.298ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-19c5874e-967f-4ef9-8099-3765d0604df5 (instance states: (Some(Online), None))
592024-08-09T21:19:24.300ZINFOcrucible-agent (dropshot): request completed latency_us = 22 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35127 req_id = a9b402c7-b92a-46d4-a630-e5d5878df689 response_code = 200 uri = /crucible/0/regions/19c5874e-967f-4ef9-8099-3765d0604df5
602024-08-09T21:19:24.304ZINFOcrucible-agent (worker): SMF ok!
612024-08-09T21:19:24.314ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/19c5874e-967f-4ef9-8099-3765d0604df5" region = 19c5874e-967f-4ef9-8099-3765d0604df5
622024-08-09T21:19:24.361ZINFOcrucible-agent (datafile): region 19c5874e-967f-4ef9-8099-3765d0604df5 state: Tombstoned -> Destroyed
632024-08-09T21:19:24.383ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:35127 req_id = 36e541c9-9413-4458-99a8-ebb4da71b62f response_code = 200 uri = /crucible/0/regions/19c5874e-967f-4ef9-8099-3765d0604df5