Max level shown:
12024-11-07T02:54:40.222ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:55689
22024-11-07T02:54:40.266ZINFOcrucible-agent (datafile): region 8f5383b1-019a-4a9b-9bdf-14da83009c49 state: Requested
32024-11-07T02:54:40.274ZINFOcrucible-agent (dropshot): request completed latency_us = 22842 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55689 req_id = aa87e24f-0d94-424b-8fe2-bb55a2450938 response_code = 200 uri = /crucible/0/regions
42024-11-07T02:54:40.274ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52024-11-07T02:54:40.287ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_7779de2f-377d-433e-9f3f-656a23324b51/crucible/regions/8f5383b1-019a-4a9b-9bdf-14da83009c49
62024-11-07T02:54:40.287ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_7779de2f-377d-433e-9f3f-656a23324b51/crucible/regions/8f5383b1-019a-4a9b-9bdf-14da83009c49
72024-11-07T02:54:40.343ZINFOcrucible-agent (worker): creating region Region { id: RegionId("8f5383b1-019a-4a9b-9bdf-14da83009c49"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/8f5383b1-019a-4a9b-9bdf-14da83009c49" region = 8f5383b1-019a-4a9b-9bdf-14da83009c49
82024-11-07T02:54:40.347ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "8f5383b1-019a-4a9b-9bdf-14da83009c49" "--data" "/data/regions/8f5383b1-019a-4a9b-9bdf-14da83009c49" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 8f5383b1-019a-4a9b-9bdf-14da83009c49
92024-11-07T02:54:40.631ZINFOcrucible-agent (dropshot): request completed latency_us = 102 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55689 req_id = c609043f-2f39-4f02-ab7f-becbad376509 response_code = 200 uri = /crucible/0/regions
102024-11-07T02:54:40.707ZINFOcrucible-agent (worker): region files created ok region = 8f5383b1-019a-4a9b-9bdf-14da83009c49
112024-11-07T02:54:40.707ZINFOcrucible-agent (datafile): region 8f5383b1-019a-4a9b-9bdf-14da83009c49 state: Requested -> Created
122024-11-07T02:54:40.707ZINFOcrucible-agent (worker): applying SMF actions post create...
132024-11-07T02:54:40.707ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-8f5383b1-019a-4a9b-9bdf-14da83009c49
142024-11-07T02:54:40.715ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-8f5383b1-019a-4a9b-9bdf-14da83009c49
152024-11-07T02:54:40.741ZINFOcrucible-agent (worker): creating config property group
162024-11-07T02:54:40.745ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8f5383b1-019a-4a9b-9bdf-14da83009c49
172024-11-07T02:54:40.745ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8f5383b1-019a-4a9b-9bdf-14da83009c49
182024-11-07T02:54:40.748ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192024-11-07T02:54:40.748ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
202024-11-07T02:54:40.748ZINFOcrucible-agent (worker): commit
212024-11-07T02:54:40.763ZINFOcrucible-agent (worker): ok!
222024-11-07T02:54:40.767ZINFOcrucible-agent (worker): SMF ok!
232024-11-07T02:54:41.000ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55689 req_id = 5b6841eb-2864-4dd0-8d81-2ea43dbfc686 response_code = 200 uri = /crucible/0/regions
242024-11-07T02:54:45.010ZINFOcrucible-agent (datafile): region c5cab0b4-69fe-49a0-8995-6ce31d434651 state: Requested
252024-11-07T02:54:45.011ZINFOcrucible-agent (dropshot): request completed latency_us = 196 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55689 req_id = d8cf04a5-22b8-42fb-a23f-11bcfece053f response_code = 200 uri = /crucible/0/regions
262024-11-07T02:54:45.011ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
272024-11-07T02:54:45.021ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_7779de2f-377d-433e-9f3f-656a23324b51/crucible/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651
282024-11-07T02:54:45.021ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_7779de2f-377d-433e-9f3f-656a23324b51/crucible/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651
292024-11-07T02:54:45.091ZINFOcrucible-agent (worker): creating region Region { id: RegionId("c5cab0b4-69fe-49a0-8995-6ce31d434651"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651" region = c5cab0b4-69fe-49a0-8995-6ce31d434651
302024-11-07T02:54:45.092ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "c5cab0b4-69fe-49a0-8995-6ce31d434651" "--data" "/data/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = c5cab0b4-69fe-49a0-8995-6ce31d434651
312024-11-07T02:54:45.162ZINFOcrucible-agent (worker): region files created ok region = c5cab0b4-69fe-49a0-8995-6ce31d434651
322024-11-07T02:54:45.162ZINFOcrucible-agent (datafile): region c5cab0b4-69fe-49a0-8995-6ce31d434651 state: Requested -> Created
332024-11-07T02:54:45.162ZINFOcrucible-agent (worker): applying SMF actions post create...
342024-11-07T02:54:45.163ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/8f5383b1-019a-4a9b-9bdf-14da83009c49
352024-11-07T02:54:45.164ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
362024-11-07T02:54:45.164ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::16
372024-11-07T02:54:45.164ZINFOcrucible-agent (worker): reconfig required
382024-11-07T02:54:45.164ZINFOcrucible-agent (worker): using existing config property group
392024-11-07T02:54:45.164ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8f5383b1-019a-4a9b-9bdf-14da83009c49
402024-11-07T02:54:45.164ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8f5383b1-019a-4a9b-9bdf-14da83009c49
412024-11-07T02:54:45.164ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
422024-11-07T02:54:45.164ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
432024-11-07T02:54:45.164ZINFOcrucible-agent (worker): commit
442024-11-07T02:54:45.168ZINFOcrucible-agent (worker): ok!
452024-11-07T02:54:45.168ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-c5cab0b4-69fe-49a0-8995-6ce31d434651
462024-11-07T02:54:45.171ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-c5cab0b4-69fe-49a0-8995-6ce31d434651
472024-11-07T02:54:45.178ZINFOcrucible-agent (worker): creating config property group
482024-11-07T02:54:45.184ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-c5cab0b4-69fe-49a0-8995-6ce31d434651
492024-11-07T02:54:45.185ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651
502024-11-07T02:54:45.185ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
512024-11-07T02:54:45.185ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
522024-11-07T02:54:45.185ZINFOcrucible-agent (worker): commit
532024-11-07T02:54:45.188ZINFOcrucible-agent (worker): ok!
542024-11-07T02:54:45.193ZINFOcrucible-agent (worker): SMF ok!
552024-11-07T02:54:45.215ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55689 req_id = 6abe50bf-8264-48b2-b228-af7d1e5e62c2 response_code = 200 uri = /crucible/0/regions
562024-11-07T02:55:09.902ZINFOcrucible-agent (dropshot): request completed latency_us = 2044 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:55689 req_id = 7b8fa356-f9ca-4eec-9dd1-be1e696675e9 response_code = 200 uri = /crucible/0/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651
572024-11-07T02:55:09.905ZINFOcrucible-agent (datafile): path is "/data/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651"
582024-11-07T02:55:09.916ZINFOcrucible-agent (datafile): dataset is oxp_7779de2f-377d-433e-9f3f-656a23324b51/crucible/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651
592024-11-07T02:55:09.925ZINFOcrucible-agent (datafile): region c5cab0b4-69fe-49a0-8995-6ce31d434651 state: Created -> Tombstoned
602024-11-07T02:55:09.926ZINFOcrucible-agent (worker): applying SMF actions before removal...
612024-11-07T02:55:09.927ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-c5cab0b4-69fe-49a0-8995-6ce31d434651 (instance states: (Some(Online), None))
622024-11-07T02:55:09.928ZINFOcrucible-agent (dropshot): request completed latency_us = 23590 local_addr = [fd00:1122:3344:101::16]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:55689 req_id = 4103a602-9449-4f75-92e9-60b7018a2a8a response_code = 204 uri = /crucible/0/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651
632024-11-07T02:55:09.932ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/8f5383b1-019a-4a9b-9bdf-14da83009c49
642024-11-07T02:55:09.932ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
652024-11-07T02:55:09.932ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::16
662024-11-07T02:55:09.932ZINFOcrucible-agent (worker): reconfig required
672024-11-07T02:55:09.932ZINFOcrucible-agent (worker): using existing config property group
682024-11-07T02:55:09.932ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8f5383b1-019a-4a9b-9bdf-14da83009c49
692024-11-07T02:55:09.933ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8f5383b1-019a-4a9b-9bdf-14da83009c49
702024-11-07T02:55:09.933ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
712024-11-07T02:55:09.933ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
722024-11-07T02:55:09.933ZINFOcrucible-agent (worker): commit
732024-11-07T02:55:09.933ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:55689 req_id = 4539480c-86be-4c28-8b30-2286d2c519b7 response_code = 200 uri = /crucible/0/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651
742024-11-07T02:55:09.937ZINFOcrucible-agent (worker): ok!
752024-11-07T02:55:09.938ZINFOcrucible-agent (worker): SMF ok!
762024-11-07T02:55:09.949ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_7779de2f-377d-433e-9f3f-656a23324b51/crucible/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651" region = c5cab0b4-69fe-49a0-8995-6ce31d434651
772024-11-07T02:55:10.008ZINFOcrucible-agent (datafile): region c5cab0b4-69fe-49a0-8995-6ce31d434651 state: Tombstoned -> Destroyed
782024-11-07T02:55:10.044ZINFOcrucible-agent (dropshot): request completed latency_us = 507 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:55689 req_id = 7558af02-41e9-4870-a96b-22ad4df937ce response_code = 200 uri = /crucible/0/regions/c5cab0b4-69fe-49a0-8995-6ce31d434651