Max level shown:
1 [ May 8 03:36:46 Enabled. ]
2 [ May 8 03:36:46 Rereading configuration. ]
3 [ May 8 03:36:48 Rereading configuration. ]
4 [ May 8 03:36:50 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_58eb44a6-f8c0-46ec-bcea-52bf3efa8996/crucible -l [fd00:1122:3344:101::12]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-05-08T03:36:50.665ZINFOcrucible-agent: dataset: "oxp_58eb44a6-f8c0-46ec-bcea-52bf3efa8996/crucible"
72024-05-08T03:36:50.669ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::12]:32345
82024-05-08T03:36:50.669ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-05-08T03:36:50.691ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-05-08T03:36:50.800ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::12]:32345
112024-05-08T03:38:02.569ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:43234
122024-05-08T03:38:02.633ZINFOcrucible-agent (datafile): region 39b609bd-26cf-4824-b906-d848088c075e state: Requested
132024-05-08T03:38:02.636ZINFOcrucible-agent (dropshot): request completed latency_us = 11366 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43234 req_id = 070268cd-0e0a-495a-873e-12f7d5ad26e2 response_code = 200 uri = /crucible/0/regions
142024-05-08T03:38:02.636ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
152024-05-08T03:38:02.647ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_58eb44a6-f8c0-46ec-bcea-52bf3efa8996/crucible/regions/39b609bd-26cf-4824-b906-d848088c075e
162024-05-08T03:38:02.647ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_58eb44a6-f8c0-46ec-bcea-52bf3efa8996/crucible/regions/39b609bd-26cf-4824-b906-d848088c075e
172024-05-08T03:38:02.723ZINFOcrucible-agent (worker): creating region Region { id: RegionId("39b609bd-26cf-4824-b906-d848088c075e"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/39b609bd-26cf-4824-b906-d848088c075e" region = 39b609bd-26cf-4824-b906-d848088c075e
182024-05-08T03:38:02.723ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "39b609bd-26cf-4824-b906-d848088c075e" "--data" "/data/regions/39b609bd-26cf-4824-b906-d848088c075e" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 39b609bd-26cf-4824-b906-d848088c075e
192024-05-08T03:38:02.806ZINFOcrucible-agent (dropshot): request completed latency_us = 1026 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43234 req_id = 978af04f-dec2-4068-a82f-5ff21597deaa response_code = 200 uri = /crucible/0/regions
202024-05-08T03:38:02.864ZINFOcrucible-agent (worker): region files created ok region = 39b609bd-26cf-4824-b906-d848088c075e
212024-05-08T03:38:02.864ZINFOcrucible-agent (datafile): region 39b609bd-26cf-4824-b906-d848088c075e state: Requested -> Created
222024-05-08T03:38:02.864ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-05-08T03:38:02.865ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-39b609bd-26cf-4824-b906-d848088c075e
242024-05-08T03:38:02.868ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-39b609bd-26cf-4824-b906-d848088c075e
252024-05-08T03:38:02.879ZINFOcrucible-agent (worker): creating config property group
262024-05-08T03:38:02.882ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-39b609bd-26cf-4824-b906-d848088c075e
272024-05-08T03:38:02.883ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/39b609bd-26cf-4824-b906-d848088c075e
282024-05-08T03:38:02.885ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-05-08T03:38:02.885ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
302024-05-08T03:38:02.885ZINFOcrucible-agent (worker): commit
312024-05-08T03:38:02.893ZINFOcrucible-agent (worker): ok!
322024-05-08T03:38:02.898ZINFOcrucible-agent (worker): SMF ok!
332024-05-08T03:38:03.469ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43234 req_id = 9650d069-c30f-4057-9cf6-989a78e5bd11 response_code = 200 uri = /crucible/0/regions
342024-05-08T03:43:25.305ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:46160
352024-05-08T03:43:25.305ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:46160 req_id = eaf40dd6-f1be-4d8c-87a3-eacff1dea2f0 response_code = 200 uri = /crucible/0/regions/39b609bd-26cf-4824-b906-d848088c075e
362024-05-08T03:43:25.305ZINFOcrucible-agent (datafile): path is "/data/regions/39b609bd-26cf-4824-b906-d848088c075e"
372024-05-08T03:43:25.314ZINFOcrucible-agent (datafile): dataset is oxp_58eb44a6-f8c0-46ec-bcea-52bf3efa8996/crucible/regions/39b609bd-26cf-4824-b906-d848088c075e
382024-05-08T03:43:25.323ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_58eb44a6-f8c0-46ec-bcea-52bf3efa8996/crucible/regions/39b609bd-26cf-4824-b906-d848088c075e@7f78880d-9da4-401f-9e4e-afcc4d6d1405
392024-05-08T03:43:25.323ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_58eb44a6-f8c0-46ec-bcea-52bf3efa8996/crucible/regions/39b609bd-26cf-4824-b906-d848088c075e", "7f78880d-9da4-401f-9e4e-afcc4d6d1405"]
402024-05-08T03:43:25.339ZINFOcrucible-agent (dropshot): request completed latency_us = 34557 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:46160 req_id = 4eaea094-3b64-4bb4-8a87-8648f342176c response_code = 200 uri = /crucible/0/regions/39b609bd-26cf-4824-b906-d848088c075e/snapshots/7f78880d-9da4-401f-9e4e-afcc4d6d1405
412024-05-08T03:43:25.340ZINFOcrucible-agent (datafile): path is "/data/regions/39b609bd-26cf-4824-b906-d848088c075e"
422024-05-08T03:43:25.348ZINFOcrucible-agent (datafile): dataset is oxp_58eb44a6-f8c0-46ec-bcea-52bf3efa8996/crucible/regions/39b609bd-26cf-4824-b906-d848088c075e
432024-05-08T03:43:25.355ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_58eb44a6-f8c0-46ec-bcea-52bf3efa8996/crucible/regions/39b609bd-26cf-4824-b906-d848088c075e@7f78880d-9da4-401f-9e4e-afcc4d6d1405
442024-05-08T03:43:25.355ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_58eb44a6-f8c0-46ec-bcea-52bf3efa8996/crucible/regions/39b609bd-26cf-4824-b906-d848088c075e", "7f78880d-9da4-401f-9e4e-afcc4d6d1405"]
452024-05-08T03:43:25.364ZINFOcrucible-agent (datafile): requesting running snapshot 39b609bd-26cf-4824-b906-d848088c075e-7f78880d-9da4-401f-9e4e-afcc4d6d1405 state: Requested
462024-05-08T03:43:25.364ZINFOcrucible-agent (worker): applying SMF actions for region 39b609bd-26cf-4824-b906-d848088c075e running snapshot 7f78880d-9da4-401f-9e4e-afcc4d6d1405 (state Requested)...
472024-05-08T03:43:25.364ZINFOcrucible-agent (dropshot): request completed latency_us = 24617 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46160 req_id = d90142e0-393e-4f79-ac6c-7392550b9e63 response_code = 200 uri = /crucible/0/regions/39b609bd-26cf-4824-b906-d848088c075e/snapshots/7f78880d-9da4-401f-9e4e-afcc4d6d1405/run
482024-05-08T03:43:25.375ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/39b609bd-26cf-4824-b906-d848088c075e
492024-05-08T03:43:25.375ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
502024-05-08T03:43:25.375ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::12
512024-05-08T03:43:25.375ZINFOcrucible-agent (worker): reconfig required
522024-05-08T03:43:25.375ZINFOcrucible-agent (worker): using existing config property group
532024-05-08T03:43:25.375ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-39b609bd-26cf-4824-b906-d848088c075e
542024-05-08T03:43:25.375ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/39b609bd-26cf-4824-b906-d848088c075e
552024-05-08T03:43:25.375ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
562024-05-08T03:43:25.375ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
572024-05-08T03:43:25.376ZINFOcrucible-agent (worker): commit
582024-05-08T03:43:25.381ZINFOcrucible-agent (worker): ok!
592024-05-08T03:43:25.381ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-39b609bd-26cf-4824-b906-d848088c075e-7f78880d-9da4-401f-9e4e-afcc4d6d1405
602024-05-08T03:43:25.387ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-39b609bd-26cf-4824-b906-d848088c075e-7f78880d-9da4-401f-9e4e-afcc4d6d1405
612024-05-08T03:43:25.397ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-39b609bd-26cf-4824-b906-d848088c075e-7f78880d-9da4-401f-9e4e-afcc4d6d1405
622024-05-08T03:43:25.398ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/39b609bd-26cf-4824-b906-d848088c075e/.zfs/snapshot/7f78880d-9da4-401f-9e4e-afcc4d6d1405
632024-05-08T03:43:25.398ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
642024-05-08T03:43:25.398ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
652024-05-08T03:43:25.398ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
662024-05-08T03:43:25.398ZINFOcrucible-agent (worker): commit
672024-05-08T03:43:25.401ZINFOcrucible-agent (worker): ok!
682024-05-08T03:43:25.405ZINFOcrucible-agent (worker): SMF ok!
692024-05-08T03:43:25.405ZINFOcrucible-agent (datafile): region 39b609bd-26cf-4824-b906-d848088c075e running snapshot 7f78880d-9da4-401f-9e4e-afcc4d6d1405 state: Requested -> Created