Max level shown:
12024-08-27T23:46:16.264ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::a]:35824
22024-08-27T23:46:16.290ZINFOcrucible-agent (datafile): region 9db73dac-9bb3-4084-9075-62fc0daa27bd state: Requested
32024-08-27T23:46:16.297ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42024-08-27T23:46:16.297ZINFOcrucible-agent (dropshot): request completed latency_us = 16417 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35824 req_id = 75cf6450-bff7-4061-96c8-3f85ea6eb9e6 response_code = 200 uri = /crucible/0/regions
52024-08-27T23:46:16.309ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd
62024-08-27T23:46:16.309ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd
72024-08-27T23:46:16.357ZINFOcrucible-agent (worker): creating region Region { id: RegionId("9db73dac-9bb3-4084-9075-62fc0daa27bd"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd" region = 9db73dac-9bb3-4084-9075-62fc0daa27bd
82024-08-27T23:46:16.359ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "9db73dac-9bb3-4084-9075-62fc0daa27bd" "--data" "/data/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 9db73dac-9bb3-4084-9075-62fc0daa27bd
92024-08-27T23:46:16.614ZINFOcrucible-agent (worker): region files created ok region = 9db73dac-9bb3-4084-9075-62fc0daa27bd
102024-08-27T23:46:16.614ZINFOcrucible-agent (datafile): region 9db73dac-9bb3-4084-9075-62fc0daa27bd state: Requested -> Created
112024-08-27T23:46:16.614ZINFOcrucible-agent (worker): applying SMF actions post create...
122024-08-27T23:46:16.614ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-9db73dac-9bb3-4084-9075-62fc0daa27bd
132024-08-27T23:46:16.620ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-9db73dac-9bb3-4084-9075-62fc0daa27bd
142024-08-27T23:46:16.641ZINFOcrucible-agent (worker): creating config property group
152024-08-27T23:46:16.645ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-9db73dac-9bb3-4084-9075-62fc0daa27bd
162024-08-27T23:46:16.654ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd
172024-08-27T23:46:16.656ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
182024-08-27T23:46:16.656ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
192024-08-27T23:46:16.656ZINFOcrucible-agent (worker): commit
202024-08-27T23:46:16.661ZINFOcrucible-agent (worker): ok!
212024-08-27T23:46:16.665ZINFOcrucible-agent (worker): SMF ok!
222024-08-27T23:46:16.679ZINFOcrucible-agent (dropshot): request completed latency_us = 1011 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:35824 req_id = 96c0671f-e1ba-4ecc-a9f2-da76f7f7413b response_code = 200 uri = /crucible/0/regions
232024-08-27T23:51:25.333ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::a]:44054
242024-08-27T23:51:25.333ZINFOcrucible-agent (dropshot): request completed latency_us = 58 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:44054 req_id = e34ec30f-3408-4fd7-aadf-b4a73e150fc7 response_code = 200 uri = /crucible/0/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd
252024-08-27T23:51:25.339ZINFOcrucible-agent (datafile): path is "/data/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd"
262024-08-27T23:51:25.348ZINFOcrucible-agent (datafile): dataset is oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd
272024-08-27T23:51:25.356ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd@c923c5bb-f150-4ef2-823e-54e88c80e377
282024-08-27T23:51:25.356ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd", "c923c5bb-f150-4ef2-823e-54e88c80e377"]
292024-08-27T23:51:25.366ZINFOcrucible-agent (dropshot): request completed latency_us = 29365 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:44054 req_id = ec00b302-cdc1-429b-b064-cc1414fdfa71 response_code = 200 uri = /crucible/0/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd/snapshots/c923c5bb-f150-4ef2-823e-54e88c80e377
302024-08-27T23:51:25.369ZINFOcrucible-agent (datafile): path is "/data/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd"
312024-08-27T23:51:25.378ZINFOcrucible-agent (datafile): dataset is oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd
322024-08-27T23:51:25.385ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd@c923c5bb-f150-4ef2-823e-54e88c80e377
332024-08-27T23:51:25.385ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd", "c923c5bb-f150-4ef2-823e-54e88c80e377"]
342024-08-27T23:51:25.392ZINFOcrucible-agent (datafile): requesting running snapshot 9db73dac-9bb3-4084-9075-62fc0daa27bd-c923c5bb-f150-4ef2-823e-54e88c80e377 state: Requested
352024-08-27T23:51:25.393ZINFOcrucible-agent (worker): applying SMF actions for region 9db73dac-9bb3-4084-9075-62fc0daa27bd running snapshot c923c5bb-f150-4ef2-823e-54e88c80e377 (state Requested)...
362024-08-27T23:51:25.393ZINFOcrucible-agent (dropshot): request completed latency_us = 23908 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:44054 req_id = 55e26128-b91d-45fa-a24e-2b901c3d0367 response_code = 200 uri = /crucible/0/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd/snapshots/c923c5bb-f150-4ef2-823e-54e88c80e377/run
372024-08-27T23:51:25.404ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd
382024-08-27T23:51:25.404ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
392024-08-27T23:51:25.404ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::1a
402024-08-27T23:51:25.404ZINFOcrucible-agent (worker): reconfig required
412024-08-27T23:51:25.404ZINFOcrucible-agent (worker): using existing config property group
422024-08-27T23:51:25.404ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-9db73dac-9bb3-4084-9075-62fc0daa27bd
432024-08-27T23:51:25.404ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd
442024-08-27T23:51:25.404ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
452024-08-27T23:51:25.404ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
462024-08-27T23:51:25.404ZINFOcrucible-agent (worker): commit
472024-08-27T23:51:25.412ZINFOcrucible-agent (worker): ok!
482024-08-27T23:51:25.413ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-9db73dac-9bb3-4084-9075-62fc0daa27bd-c923c5bb-f150-4ef2-823e-54e88c80e377
492024-08-27T23:51:25.416ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-9db73dac-9bb3-4084-9075-62fc0daa27bd-c923c5bb-f150-4ef2-823e-54e88c80e377
502024-08-27T23:51:25.427ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-9db73dac-9bb3-4084-9075-62fc0daa27bd-c923c5bb-f150-4ef2-823e-54e88c80e377
512024-08-27T23:51:25.427ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd/.zfs/snapshot/c923c5bb-f150-4ef2-823e-54e88c80e377
522024-08-27T23:51:25.427ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
532024-08-27T23:51:25.427ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
542024-08-27T23:51:25.427ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
552024-08-27T23:51:25.427ZINFOcrucible-agent (worker): commit
562024-08-27T23:51:25.431ZINFOcrucible-agent (worker): ok!
572024-08-27T23:51:25.435ZINFOcrucible-agent (worker): SMF ok!
582024-08-27T23:51:25.435ZINFOcrucible-agent (datafile): region 9db73dac-9bb3-4084-9075-62fc0daa27bd running snapshot c923c5bb-f150-4ef2-823e-54e88c80e377 state: Requested -> Created
592024-08-27T23:51:25.564ZINFOcrucible-agent (datafile): path is "/data/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd"
602024-08-27T23:51:25.574ZINFOcrucible-agent (datafile): dataset is oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd
612024-08-27T23:51:25.581ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd@c923c5bb-f150-4ef2-823e-54e88c80e377
622024-08-27T23:51:25.581ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6d2d4b30-dc11-496b-9bd8-e427a70cc372/crucible/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd", "c923c5bb-f150-4ef2-823e-54e88c80e377"]
632024-08-27T23:51:25.589ZINFOcrucible-agent (dropshot): request completed latency_us = 24721 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:44054 req_id = aabb22a5-32b6-4167-b056-18aa92d610b8 response_code = 200 uri = /crucible/0/regions/9db73dac-9bb3-4084-9075-62fc0daa27bd/snapshots/c923c5bb-f150-4ef2-823e-54e88c80e377/run