Max level shown:
12024-11-01T07:02:49.084ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:33566
22024-11-01T07:02:49.127ZINFOcrucible-agent (datafile): region 776d6904-6e98-4655-a094-154844de0bfe state: Requested
32024-11-01T07:02:49.131ZINFOcrucible-agent (dropshot): request completed latency_us = 18344 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33566 req_id = 89195e26-03e4-4ec2-ad64-47f0e40abb96 response_code = 200 uri = /crucible/0/regions
42024-11-01T07:02:49.134ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52024-11-01T07:02:49.144ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_415a68a1-91bd-446b-a587-2b70d19e26e9/crucible/regions/776d6904-6e98-4655-a094-154844de0bfe
62024-11-01T07:02:49.144ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_415a68a1-91bd-446b-a587-2b70d19e26e9/crucible/regions/776d6904-6e98-4655-a094-154844de0bfe
72024-11-01T07:02:49.189ZINFOcrucible-agent (worker): creating region Region { id: RegionId("776d6904-6e98-4655-a094-154844de0bfe"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/776d6904-6e98-4655-a094-154844de0bfe" region = 776d6904-6e98-4655-a094-154844de0bfe
82024-11-01T07:02:49.193ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "776d6904-6e98-4655-a094-154844de0bfe" "--data" "/data/regions/776d6904-6e98-4655-a094-154844de0bfe" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 776d6904-6e98-4655-a094-154844de0bfe
92024-11-01T07:02:49.289ZINFOcrucible-agent (dropshot): request completed latency_us = 124 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33566 req_id = 683b166e-1e55-4d8f-8b88-c6fde64cc155 response_code = 200 uri = /crucible/0/regions
102024-11-01T07:02:49.555ZINFOcrucible-agent (worker): region files created ok region = 776d6904-6e98-4655-a094-154844de0bfe
112024-11-01T07:02:49.555ZINFOcrucible-agent (datafile): region 776d6904-6e98-4655-a094-154844de0bfe state: Requested -> Created
122024-11-01T07:02:49.555ZINFOcrucible-agent (worker): applying SMF actions post create...
132024-11-01T07:02:49.556ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-776d6904-6e98-4655-a094-154844de0bfe
142024-11-01T07:02:49.563ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-776d6904-6e98-4655-a094-154844de0bfe
152024-11-01T07:02:49.585ZINFOcrucible-agent (worker): creating config property group
162024-11-01T07:02:49.588ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-776d6904-6e98-4655-a094-154844de0bfe
172024-11-01T07:02:49.589ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/776d6904-6e98-4655-a094-154844de0bfe
182024-11-01T07:02:49.592ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192024-11-01T07:02:49.592ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
202024-11-01T07:02:49.592ZINFOcrucible-agent (worker): commit
212024-11-01T07:02:49.604ZINFOcrucible-agent (worker): ok!
222024-11-01T07:02:49.609ZINFOcrucible-agent (worker): SMF ok!
232024-11-01T07:02:49.777ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33566 req_id = 4b179918-1c90-476b-ada0-942877c2742b response_code = 200 uri = /crucible/0/regions
242024-11-01T07:02:54.860ZINFOcrucible-agent (datafile): region 1da17d77-3c60-4828-8cff-29595c558079 state: Requested
252024-11-01T07:02:54.861ZINFOcrucible-agent (dropshot): request completed latency_us = 228 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33566 req_id = 28b96805-dba0-4271-86ee-ebbd873a1ec8 response_code = 200 uri = /crucible/0/regions
262024-11-01T07:02:54.861ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
272024-11-01T07:02:54.871ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_415a68a1-91bd-446b-a587-2b70d19e26e9/crucible/regions/1da17d77-3c60-4828-8cff-29595c558079
282024-11-01T07:02:54.871ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_415a68a1-91bd-446b-a587-2b70d19e26e9/crucible/regions/1da17d77-3c60-4828-8cff-29595c558079
292024-11-01T07:02:54.928ZINFOcrucible-agent (worker): creating region Region { id: RegionId("1da17d77-3c60-4828-8cff-29595c558079"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/1da17d77-3c60-4828-8cff-29595c558079" region = 1da17d77-3c60-4828-8cff-29595c558079
302024-11-01T07:02:54.928ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "1da17d77-3c60-4828-8cff-29595c558079" "--data" "/data/regions/1da17d77-3c60-4828-8cff-29595c558079" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 1da17d77-3c60-4828-8cff-29595c558079
312024-11-01T07:02:55.047ZINFOcrucible-agent (worker): region files created ok region = 1da17d77-3c60-4828-8cff-29595c558079
322024-11-01T07:02:55.047ZINFOcrucible-agent (datafile): region 1da17d77-3c60-4828-8cff-29595c558079 state: Requested -> Created
332024-11-01T07:02:55.047ZINFOcrucible-agent (worker): applying SMF actions post create...
342024-11-01T07:02:55.048ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-1da17d77-3c60-4828-8cff-29595c558079
352024-11-01T07:02:55.051ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-1da17d77-3c60-4828-8cff-29595c558079
362024-11-01T07:02:55.059ZINFOcrucible-agent (worker): creating config property group
372024-11-01T07:02:55.067ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-1da17d77-3c60-4828-8cff-29595c558079
382024-11-01T07:02:55.067ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/1da17d77-3c60-4828-8cff-29595c558079
392024-11-01T07:02:55.067ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
402024-11-01T07:02:55.067ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
412024-11-01T07:02:55.067ZINFOcrucible-agent (worker): commit
422024-11-01T07:02:55.071ZINFOcrucible-agent (worker): ok!
432024-11-01T07:02:55.078ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/776d6904-6e98-4655-a094-154844de0bfe
442024-11-01T07:02:55.078ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
452024-11-01T07:02:55.078ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::19
462024-11-01T07:02:55.078ZINFOcrucible-agent (worker): reconfig required
472024-11-01T07:02:55.078ZINFOcrucible-agent (worker): using existing config property group
482024-11-01T07:02:55.078ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-776d6904-6e98-4655-a094-154844de0bfe
492024-11-01T07:02:55.078ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/776d6904-6e98-4655-a094-154844de0bfe
502024-11-01T07:02:55.079ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
512024-11-01T07:02:55.079ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
522024-11-01T07:02:55.079ZINFOcrucible-agent (worker): commit
532024-11-01T07:02:55.088ZINFOcrucible-agent (worker): ok!
542024-11-01T07:02:55.089ZINFOcrucible-agent (worker): SMF ok!
552024-11-01T07:02:55.208ZINFOcrucible-agent (dropshot): request completed latency_us = 98 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33566 req_id = b55ee5ca-76ae-45db-982b-e38af9cfcc1d response_code = 200 uri = /crucible/0/regions
562024-11-01T07:03:19.305ZINFOcrucible-agent (dropshot): request completed latency_us = 212 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33566 req_id = e54daf7f-0c11-47ab-b9b7-71a0f2004fbd response_code = 200 uri = /crucible/0/regions/1da17d77-3c60-4828-8cff-29595c558079
572024-11-01T07:03:19.309ZINFOcrucible-agent (datafile): path is "/data/regions/1da17d77-3c60-4828-8cff-29595c558079"
582024-11-01T07:03:19.321ZINFOcrucible-agent (datafile): dataset is oxp_415a68a1-91bd-446b-a587-2b70d19e26e9/crucible/regions/1da17d77-3c60-4828-8cff-29595c558079
592024-11-01T07:03:19.332ZINFOcrucible-agent (datafile): region 1da17d77-3c60-4828-8cff-29595c558079 state: Created -> Tombstoned
602024-11-01T07:03:19.332ZINFOcrucible-agent (worker): applying SMF actions before removal...
612024-11-01T07:03:19.333ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-1da17d77-3c60-4828-8cff-29595c558079 (instance states: (Some(Online), None))
622024-11-01T07:03:19.334ZINFOcrucible-agent (dropshot): request completed latency_us = 25573 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:33566 req_id = da81d8cb-8b2f-4887-bf88-31900cac3733 response_code = 204 uri = /crucible/0/regions/1da17d77-3c60-4828-8cff-29595c558079
632024-11-01T07:03:19.337ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/776d6904-6e98-4655-a094-154844de0bfe
642024-11-01T07:03:19.337ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
652024-11-01T07:03:19.338ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::19
662024-11-01T07:03:19.338ZINFOcrucible-agent (worker): reconfig required
672024-11-01T07:03:19.338ZINFOcrucible-agent (worker): using existing config property group
682024-11-01T07:03:19.338ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-776d6904-6e98-4655-a094-154844de0bfe
692024-11-01T07:03:19.338ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/776d6904-6e98-4655-a094-154844de0bfe
702024-11-01T07:03:19.338ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
712024-11-01T07:03:19.338ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
722024-11-01T07:03:19.338ZINFOcrucible-agent (worker): commit
732024-11-01T07:03:19.339ZINFOcrucible-agent (dropshot): request completed latency_us = 39 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33566 req_id = 7c4fa183-f0c5-425a-b13c-a82271580ee5 response_code = 200 uri = /crucible/0/regions/1da17d77-3c60-4828-8cff-29595c558079
742024-11-01T07:03:19.342ZINFOcrucible-agent (worker): ok!
752024-11-01T07:03:19.343ZINFOcrucible-agent (worker): SMF ok!
762024-11-01T07:03:19.354ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_415a68a1-91bd-446b-a587-2b70d19e26e9/crucible/regions/1da17d77-3c60-4828-8cff-29595c558079" region = 1da17d77-3c60-4828-8cff-29595c558079
772024-11-01T07:03:19.426ZINFOcrucible-agent (datafile): region 1da17d77-3c60-4828-8cff-29595c558079 state: Tombstoned -> Destroyed
782024-11-01T07:03:19.493ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33566 req_id = 8d8a94bc-9602-4277-82d3-815333b7df90 response_code = 200 uri = /crucible/0/regions/1da17d77-3c60-4828-8cff-29595c558079