Max level shown:
12024-08-28T01:02:23.381ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:48894
22024-08-28T01:02:23.413ZINFOcrucible-agent (datafile): region 0be9131b-3985-45f3-bd91-0d31fd4f26b8 state: Requested
32024-08-28T01:02:23.421ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42024-08-28T01:02:23.425ZINFOcrucible-agent (dropshot): request completed latency_us = 24057 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48894 req_id = ffbf7e00-3ebe-45c4-9d44-c14cf3f99b07 response_code = 200 uri = /crucible/0/regions
52024-08-28T01:02:23.432ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_2deff32e-1106-452e-a0e4-7f5633a4a1c4/crucible/regions/0be9131b-3985-45f3-bd91-0d31fd4f26b8
62024-08-28T01:02:23.432ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_2deff32e-1106-452e-a0e4-7f5633a4a1c4/crucible/regions/0be9131b-3985-45f3-bd91-0d31fd4f26b8
72024-08-28T01:02:23.499ZINFOcrucible-agent (worker): creating region Region { id: RegionId("0be9131b-3985-45f3-bd91-0d31fd4f26b8"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/0be9131b-3985-45f3-bd91-0d31fd4f26b8" region = 0be9131b-3985-45f3-bd91-0d31fd4f26b8
82024-08-28T01:02:23.501ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "0be9131b-3985-45f3-bd91-0d31fd4f26b8" "--data" "/data/regions/0be9131b-3985-45f3-bd91-0d31fd4f26b8" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 0be9131b-3985-45f3-bd91-0d31fd4f26b8
92024-08-28T01:02:23.787ZINFOcrucible-agent (dropshot): request completed latency_us = 87 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48894 req_id = dbeba89c-f7b0-4f29-925e-724d8bb02a4a response_code = 200 uri = /crucible/0/regions
102024-08-28T01:02:23.868ZINFOcrucible-agent (worker): region files created ok region = 0be9131b-3985-45f3-bd91-0d31fd4f26b8
112024-08-28T01:02:23.868ZINFOcrucible-agent (datafile): region 0be9131b-3985-45f3-bd91-0d31fd4f26b8 state: Requested -> Created
122024-08-28T01:02:23.869ZINFOcrucible-agent (worker): applying SMF actions post create...
132024-08-28T01:02:23.869ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-0be9131b-3985-45f3-bd91-0d31fd4f26b8
142024-08-28T01:02:23.877ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-0be9131b-3985-45f3-bd91-0d31fd4f26b8
152024-08-28T01:02:23.901ZINFOcrucible-agent (worker): creating config property group
162024-08-28T01:02:23.906ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-0be9131b-3985-45f3-bd91-0d31fd4f26b8
172024-08-28T01:02:23.916ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/0be9131b-3985-45f3-bd91-0d31fd4f26b8
182024-08-28T01:02:23.918ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192024-08-28T01:02:23.918ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
202024-08-28T01:02:23.918ZINFOcrucible-agent (worker): commit
212024-08-28T01:02:23.925ZINFOcrucible-agent (worker): ok!
222024-08-28T01:02:23.933ZINFOcrucible-agent (worker): SMF ok!
232024-08-28T01:02:24.044ZINFOcrucible-agent (dropshot): request completed latency_us = 81 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48894 req_id = b75ce728-802e-4f50-8216-d6dee3fd8668 response_code = 200 uri = /crucible/0/regions
242024-08-28T01:02:29.495ZINFOcrucible-agent (datafile): region ab03482e-9f77-416a-99c0-aee6d96ac29e state: Requested
252024-08-28T01:02:29.495ZINFOcrucible-agent (dropshot): request completed latency_us = 823 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48894 req_id = d2ae3189-81db-4aab-9234-f839e6109cad response_code = 200 uri = /crucible/0/regions
262024-08-28T01:02:29.495ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
272024-08-28T01:02:29.505ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_2deff32e-1106-452e-a0e4-7f5633a4a1c4/crucible/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e
282024-08-28T01:02:29.505ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_2deff32e-1106-452e-a0e4-7f5633a4a1c4/crucible/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e
292024-08-28T01:02:29.579ZINFOcrucible-agent (worker): creating region Region { id: RegionId("ab03482e-9f77-416a-99c0-aee6d96ac29e"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e" region = ab03482e-9f77-416a-99c0-aee6d96ac29e
302024-08-28T01:02:29.579ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "ab03482e-9f77-416a-99c0-aee6d96ac29e" "--data" "/data/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = ab03482e-9f77-416a-99c0-aee6d96ac29e
312024-08-28T01:02:29.687ZINFOcrucible-agent (worker): region files created ok region = ab03482e-9f77-416a-99c0-aee6d96ac29e
322024-08-28T01:02:29.687ZINFOcrucible-agent (datafile): region ab03482e-9f77-416a-99c0-aee6d96ac29e state: Requested -> Created
332024-08-28T01:02:29.687ZINFOcrucible-agent (worker): applying SMF actions post create...
342024-08-28T01:02:29.688ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/0be9131b-3985-45f3-bd91-0d31fd4f26b8
352024-08-28T01:02:29.688ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
362024-08-28T01:02:29.688ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::12
372024-08-28T01:02:29.688ZINFOcrucible-agent (worker): reconfig required
382024-08-28T01:02:29.688ZINFOcrucible-agent (worker): using existing config property group
392024-08-28T01:02:29.688ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-0be9131b-3985-45f3-bd91-0d31fd4f26b8
402024-08-28T01:02:29.688ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/0be9131b-3985-45f3-bd91-0d31fd4f26b8
412024-08-28T01:02:29.688ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
422024-08-28T01:02:29.688ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432024-08-28T01:02:29.688ZINFOcrucible-agent (worker): commit
442024-08-28T01:02:29.696ZINFOcrucible-agent (worker): ok!
452024-08-28T01:02:29.697ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-ab03482e-9f77-416a-99c0-aee6d96ac29e
462024-08-28T01:02:29.704ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-ab03482e-9f77-416a-99c0-aee6d96ac29e
472024-08-28T01:02:29.719ZINFOcrucible-agent (worker): creating config property group
482024-08-28T01:02:29.729ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-ab03482e-9f77-416a-99c0-aee6d96ac29e
492024-08-28T01:02:29.729ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e
502024-08-28T01:02:29.729ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
512024-08-28T01:02:29.729ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
522024-08-28T01:02:29.729ZINFOcrucible-agent (worker): commit
532024-08-28T01:02:29.734ZINFOcrucible-agent (worker): ok!
542024-08-28T01:02:29.744ZINFOcrucible-agent (worker): SMF ok!
552024-08-28T01:02:29.748ZINFOcrucible-agent (dropshot): request completed latency_us = 85 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48894 req_id = 086f80ec-9ca4-4856-b656-88e5e18ee553 response_code = 200 uri = /crucible/0/regions
562024-08-28T01:02:54.786ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48894 req_id = a2f69c4a-af15-4267-8ab4-e7f5422543e7 response_code = 200 uri = /crucible/0/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e
572024-08-28T01:02:54.788ZINFOcrucible-agent (datafile): path is "/data/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e"
582024-08-28T01:02:54.799ZINFOcrucible-agent (datafile): dataset is oxp_2deff32e-1106-452e-a0e4-7f5633a4a1c4/crucible/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e
592024-08-28T01:02:54.807ZINFOcrucible-agent (datafile): region ab03482e-9f77-416a-99c0-aee6d96ac29e state: Created -> Tombstoned
602024-08-28T01:02:54.807ZINFOcrucible-agent (worker): applying SMF actions before removal...
612024-08-28T01:02:54.807ZINFOcrucible-agent (dropshot): request completed latency_us = 18589 local_addr = [fd00:1122:3344:101::12]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:48894 req_id = c767fe77-f41a-457b-bcb2-450f914996fd response_code = 204 uri = /crucible/0/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e
622024-08-28T01:02:54.808ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-ab03482e-9f77-416a-99c0-aee6d96ac29e (instance states: (Some(Online), None))
632024-08-28T01:02:54.811ZINFOcrucible-agent (dropshot): request completed latency_us = 26 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48894 req_id = f353c5ae-438a-4442-9f38-ad891f1ea04f response_code = 200 uri = /crucible/0/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e
642024-08-28T01:02:54.812ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/0be9131b-3985-45f3-bd91-0d31fd4f26b8
652024-08-28T01:02:54.812ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
662024-08-28T01:02:54.812ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::12
672024-08-28T01:02:54.812ZINFOcrucible-agent (worker): reconfig required
682024-08-28T01:02:54.812ZINFOcrucible-agent (worker): using existing config property group
692024-08-28T01:02:54.812ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-0be9131b-3985-45f3-bd91-0d31fd4f26b8
702024-08-28T01:02:54.813ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/0be9131b-3985-45f3-bd91-0d31fd4f26b8
712024-08-28T01:02:54.813ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
722024-08-28T01:02:54.813ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
732024-08-28T01:02:54.813ZINFOcrucible-agent (worker): commit
742024-08-28T01:02:54.817ZINFOcrucible-agent (worker): ok!
752024-08-28T01:02:54.817ZINFOcrucible-agent (worker): SMF ok!
762024-08-28T01:02:54.828ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_2deff32e-1106-452e-a0e4-7f5633a4a1c4/crucible/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e" region = ab03482e-9f77-416a-99c0-aee6d96ac29e
772024-08-28T01:02:54.877ZINFOcrucible-agent (datafile): region ab03482e-9f77-416a-99c0-aee6d96ac29e state: Tombstoned -> Destroyed
782024-08-28T01:02:54.904ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48894 req_id = 4ae19f2d-d80c-4510-ab86-a7968b917f1b response_code = 200 uri = /crucible/0/regions/ab03482e-9f77-416a-99c0-aee6d96ac29e