Max level shown:
12024-09-03T23:01:42.861ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:55588
22024-09-03T23:01:42.861ZINFOcrucible-agent (datafile): region 3d1d8368-68be-4543-bd63-f946cd5bf0a2 state: Requested
32024-09-03T23:01:42.861ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42024-09-03T23:01:42.861ZINFOcrucible-agent (dropshot): request completed latency_us = 610 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55588 req_id = 1830b20f-e260-42fa-ba43-db027922f4e1 response_code = 200 uri = /crucible/0/regions
52024-09-03T23:01:42.873ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/3d1d8368-68be-4543-bd63-f946cd5bf0a2
62024-09-03T23:01:42.873ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/3d1d8368-68be-4543-bd63-f946cd5bf0a2
72024-09-03T23:01:42.938ZINFOcrucible-agent (worker): creating region Region { id: RegionId("3d1d8368-68be-4543-bd63-f946cd5bf0a2"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/3d1d8368-68be-4543-bd63-f946cd5bf0a2" region = 3d1d8368-68be-4543-bd63-f946cd5bf0a2
82024-09-03T23:01:42.938ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "3d1d8368-68be-4543-bd63-f946cd5bf0a2" "--data" "/data/regions/3d1d8368-68be-4543-bd63-f946cd5bf0a2" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 3d1d8368-68be-4543-bd63-f946cd5bf0a2
92024-09-03T23:01:43.075ZINFOcrucible-agent (worker): region files created ok region = 3d1d8368-68be-4543-bd63-f946cd5bf0a2
102024-09-03T23:01:43.075ZINFOcrucible-agent (datafile): region 3d1d8368-68be-4543-bd63-f946cd5bf0a2 state: Requested -> Created
112024-09-03T23:01:43.075ZINFOcrucible-agent (worker): applying SMF actions post create...
122024-09-03T23:01:43.076ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-3d1d8368-68be-4543-bd63-f946cd5bf0a2
132024-09-03T23:01:43.085ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-3d1d8368-68be-4543-bd63-f946cd5bf0a2
142024-09-03T23:01:43.113ZINFOcrucible-agent (worker): creating config property group
152024-09-03T23:01:43.135ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-3d1d8368-68be-4543-bd63-f946cd5bf0a2
162024-09-03T23:01:43.136ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3d1d8368-68be-4543-bd63-f946cd5bf0a2
172024-09-03T23:01:43.136ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182024-09-03T23:01:43.136ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192024-09-03T23:01:43.136ZINFOcrucible-agent (worker): commit
202024-09-03T23:01:43.136ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55588 req_id = 696b280d-033e-4126-9b7a-864b813f7487 response_code = 200 uri = /crucible/0/regions
212024-09-03T23:01:43.151ZINFOcrucible-agent (worker): ok!
222024-09-03T23:01:43.163ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6
232024-09-03T23:01:43.163ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
242024-09-03T23:01:43.163ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::17
252024-09-03T23:01:43.163ZINFOcrucible-agent (worker): reconfig required
262024-09-03T23:01:43.163ZINFOcrucible-agent (worker): using existing config property group
272024-09-03T23:01:43.163ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4dbc6cbe-02cf-4469-8612-b6869a3b80d6
282024-09-03T23:01:43.163ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6
292024-09-03T23:01:43.166ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302024-09-03T23:01:43.166ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
312024-09-03T23:01:43.166ZINFOcrucible-agent (worker): commit
322024-09-03T23:01:43.185ZINFOcrucible-agent (worker): ok!
332024-09-03T23:01:43.185ZINFOcrucible-agent (worker): SMF ok!
342024-09-03T23:01:44.892ZINFOcrucible-agent (dropshot): request completed latency_us = 3528 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:55588 req_id = ad748275-ffde-46ed-a0ac-a778c6684c43 response_code = 200 uri = /crucible/0/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6
352024-09-03T23:01:44.896ZINFOcrucible-agent (datafile): path is "/data/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6"
362024-09-03T23:01:44.904ZINFOcrucible-agent (datafile): dataset is oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6
372024-09-03T23:01:44.913ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6@1f23f307-a46b-490d-903b-123225c5acfd
382024-09-03T23:01:44.913ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6", "1f23f307-a46b-490d-903b-123225c5acfd"]
392024-09-03T23:01:44.924ZINFOcrucible-agent (dropshot): request completed latency_us = 28379 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:55588 req_id = c938ecae-b326-4a44-bad5-ba0ea8b179ac response_code = 200 uri = /crucible/0/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6/snapshots/1f23f307-a46b-490d-903b-123225c5acfd
402024-09-03T23:01:44.928ZINFOcrucible-agent (datafile): path is "/data/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6"
412024-09-03T23:01:44.937ZINFOcrucible-agent (datafile): dataset is oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6
422024-09-03T23:01:44.946ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6@1f23f307-a46b-490d-903b-123225c5acfd
432024-09-03T23:01:44.946ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6", "1f23f307-a46b-490d-903b-123225c5acfd"]
442024-09-03T23:01:44.954ZINFOcrucible-agent (datafile): requesting running snapshot 4dbc6cbe-02cf-4469-8612-b6869a3b80d6-1f23f307-a46b-490d-903b-123225c5acfd state: Requested
452024-09-03T23:01:44.955ZINFOcrucible-agent (worker): applying SMF actions for region 4dbc6cbe-02cf-4469-8612-b6869a3b80d6 running snapshot 1f23f307-a46b-490d-903b-123225c5acfd (state Requested)...
462024-09-03T23:01:44.955ZINFOcrucible-agent (dropshot): request completed latency_us = 26552 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55588 req_id = 4a3fef1f-c7d0-4e56-a277-c9b5a3391637 response_code = 200 uri = /crucible/0/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6/snapshots/1f23f307-a46b-490d-903b-123225c5acfd/run
472024-09-03T23:01:44.956ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6
482024-09-03T23:01:44.956ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
492024-09-03T23:01:44.957ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::17
502024-09-03T23:01:44.957ZINFOcrucible-agent (worker): reconfig required
512024-09-03T23:01:44.957ZINFOcrucible-agent (worker): using existing config property group
522024-09-03T23:01:44.957ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4dbc6cbe-02cf-4469-8612-b6869a3b80d6
532024-09-03T23:01:44.957ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6
542024-09-03T23:01:44.957ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
552024-09-03T23:01:44.957ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
562024-09-03T23:01:44.957ZINFOcrucible-agent (worker): commit
572024-09-03T23:01:44.964ZINFOcrucible-agent (worker): ok!
582024-09-03T23:01:44.965ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-4dbc6cbe-02cf-4469-8612-b6869a3b80d6-1f23f307-a46b-490d-903b-123225c5acfd
592024-09-03T23:01:44.976ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-4dbc6cbe-02cf-4469-8612-b6869a3b80d6-1f23f307-a46b-490d-903b-123225c5acfd
602024-09-03T23:01:44.996ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-4dbc6cbe-02cf-4469-8612-b6869a3b80d6-1f23f307-a46b-490d-903b-123225c5acfd
612024-09-03T23:01:44.996ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6/.zfs/snapshot/1f23f307-a46b-490d-903b-123225c5acfd
622024-09-03T23:01:44.996ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
632024-09-03T23:01:44.996ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
642024-09-03T23:01:44.996ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
652024-09-03T23:01:44.996ZINFOcrucible-agent (worker): commit
662024-09-03T23:01:45.003ZINFOcrucible-agent (worker): ok!
672024-09-03T23:01:45.011ZINFOcrucible-agent (worker): SMF ok!
682024-09-03T23:01:45.011ZINFOcrucible-agent (datafile): region 4dbc6cbe-02cf-4469-8612-b6869a3b80d6 running snapshot 1f23f307-a46b-490d-903b-123225c5acfd state: Requested -> Created
692024-09-03T23:01:45.328ZINFOcrucible-agent (datafile): path is "/data/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6"
702024-09-03T23:01:45.338ZINFOcrucible-agent (datafile): dataset is oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6
712024-09-03T23:01:45.346ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6@1f23f307-a46b-490d-903b-123225c5acfd
722024-09-03T23:01:45.346ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_eebed2ec-c17c-4e80-b666-9f528996334b/crucible/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6", "1f23f307-a46b-490d-903b-123225c5acfd"]
732024-09-03T23:01:45.355ZINFOcrucible-agent (dropshot): request completed latency_us = 26564 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:55588 req_id = 561cf7c3-cfa3-4a0f-b7cb-28e85dfdce26 response_code = 200 uri = /crucible/0/regions/4dbc6cbe-02cf-4469-8612-b6869a3b80d6/snapshots/1f23f307-a46b-490d-903b-123225c5acfd/run