Max level shown:
1 [ Feb 22 18:31:16 Enabled. ]
2 [ Feb 22 18:31:16 Rereading configuration. ]
3 [ Feb 22 18:31:18 Rereading configuration. ]
4 [ Feb 22 18:31:27 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible -l [fd00:1122:3344:101::16]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-02-22T18:31:27.101ZINFOcrucible-agent: dataset: "oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible"
72024-02-22T18:31:27.105ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::16]:32345
82024-02-22T18:31:27.105ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-02-22T18:31:27.319ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::16]:32345
102024-02-22T18:32:30.014ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:41583
112024-02-22T18:32:30.029ZINFOcrucible-agent (datafile): region 9f852063-217e-476c-be0f-1cdbe52b6721 state: Requested
122024-02-22T18:32:30.030ZINFOcrucible-agent (dropshot): request completed latency_us = 5001 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:41583 req_id = 31624d3a-721a-469e-8c29-dabbba57275f response_code = 200 uri = /crucible/0/regions
132024-02-22T18:32:30.030ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
142024-02-22T18:32:30.039ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/9f852063-217e-476c-be0f-1cdbe52b6721
152024-02-22T18:32:30.039ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/9f852063-217e-476c-be0f-1cdbe52b6721
162024-02-22T18:32:30.097ZINFOcrucible-agent (worker): creating region Region { id: RegionId("9f852063-217e-476c-be0f-1cdbe52b6721"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None } at "/data/regions/9f852063-217e-476c-be0f-1cdbe52b6721" region = 9f852063-217e-476c-be0f-1cdbe52b6721
172024-02-22T18:32:30.163ZINFOcrucible-agent (dropshot): request completed latency_us = 675 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:41583 req_id = f2325b57-fe89-42f0-976f-4f70c35c8b43 response_code = 200 uri = /crucible/0/regions
182024-02-22T18:32:30.231ZINFOcrucible-agent (worker): region files created ok region = 9f852063-217e-476c-be0f-1cdbe52b6721
192024-02-22T18:32:30.231ZINFOcrucible-agent (datafile): region 9f852063-217e-476c-be0f-1cdbe52b6721 state: Requested -> Created
202024-02-22T18:32:30.232ZINFOcrucible-agent (worker): applying SMF actions post create...
212024-02-22T18:32:30.232ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-9f852063-217e-476c-be0f-1cdbe52b6721
222024-02-22T18:32:30.236ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-9f852063-217e-476c-be0f-1cdbe52b6721
232024-02-22T18:32:30.251ZINFOcrucible-agent (worker): creating config property group
242024-02-22T18:32:30.254ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-9f852063-217e-476c-be0f-1cdbe52b6721
252024-02-22T18:32:30.256ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9f852063-217e-476c-be0f-1cdbe52b6721
262024-02-22T18:32:30.258ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
272024-02-22T18:32:30.258ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
282024-02-22T18:32:30.258ZINFOcrucible-agent (worker): commit
292024-02-22T18:32:30.264ZINFOcrucible-agent (worker): ok!
302024-02-22T18:32:30.268ZINFOcrucible-agent (worker): SMF ok!
312024-02-22T18:32:30.515ZINFOcrucible-agent (dropshot): request completed latency_us = 103 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:41583 req_id = 597286ce-9cd1-454b-9b66-285cf05c0b65 response_code = 200 uri = /crucible/0/regions
322024-02-22T18:37:52.120ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:51629
332024-02-22T18:37:52.123ZINFOcrucible-agent (dropshot): request completed latency_us = 2804 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:51629 req_id = d29caa86-94a1-451a-b2dd-1cafd7538c9f response_code = 200 uri = /crucible/0/regions/9f852063-217e-476c-be0f-1cdbe52b6721
342024-02-22T18:37:52.123ZINFOcrucible-agent (datafile): path is "/data/regions/9f852063-217e-476c-be0f-1cdbe52b6721"
352024-02-22T18:37:52.134ZINFOcrucible-agent (datafile): dataset is oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/9f852063-217e-476c-be0f-1cdbe52b6721
362024-02-22T18:37:52.144ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/9f852063-217e-476c-be0f-1cdbe52b6721@5d9df203-ce89-4f7f-9ce1-08b8316a4558
372024-02-22T18:37:52.144ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/9f852063-217e-476c-be0f-1cdbe52b6721", "5d9df203-ce89-4f7f-9ce1-08b8316a4558"]
382024-02-22T18:37:52.156ZINFOcrucible-agent (dropshot): request completed latency_us = 32542 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:51629 req_id = 4e71034e-9e3d-4afb-9b51-20118d328d60 response_code = 200 uri = /crucible/0/regions/9f852063-217e-476c-be0f-1cdbe52b6721/snapshots/5d9df203-ce89-4f7f-9ce1-08b8316a4558
392024-02-22T18:37:52.156ZINFOcrucible-agent (datafile): path is "/data/regions/9f852063-217e-476c-be0f-1cdbe52b6721"
402024-02-22T18:37:52.165ZINFOcrucible-agent (datafile): dataset is oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/9f852063-217e-476c-be0f-1cdbe52b6721
412024-02-22T18:37:52.173ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/9f852063-217e-476c-be0f-1cdbe52b6721@5d9df203-ce89-4f7f-9ce1-08b8316a4558
422024-02-22T18:37:52.173ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/9f852063-217e-476c-be0f-1cdbe52b6721", "5d9df203-ce89-4f7f-9ce1-08b8316a4558"]
432024-02-22T18:37:52.182ZINFOcrucible-agent (datafile): requesting running snapshot 9f852063-217e-476c-be0f-1cdbe52b6721-5d9df203-ce89-4f7f-9ce1-08b8316a4558 state: Requested
442024-02-22T18:37:52.183ZINFOcrucible-agent (worker): applying SMF actions for region 9f852063-217e-476c-be0f-1cdbe52b6721 running snapshot 5d9df203-ce89-4f7f-9ce1-08b8316a4558 (state Requested)...
452024-02-22T18:37:52.183ZINFOcrucible-agent (dropshot): request completed latency_us = 26801 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51629 req_id = dc1ef83d-ca8b-4c5d-bf1b-bb2ce9ffb462 response_code = 200 uri = /crucible/0/regions/9f852063-217e-476c-be0f-1cdbe52b6721/snapshots/5d9df203-ce89-4f7f-9ce1-08b8316a4558/run
462024-02-22T18:37:52.187ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/9f852063-217e-476c-be0f-1cdbe52b6721
472024-02-22T18:37:52.187ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
482024-02-22T18:37:52.187ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::16
492024-02-22T18:37:52.187ZINFOcrucible-agent (worker): reconfig required
502024-02-22T18:37:52.187ZINFOcrucible-agent (worker): using existing config property group
512024-02-22T18:37:52.187ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-9f852063-217e-476c-be0f-1cdbe52b6721
522024-02-22T18:37:52.188ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9f852063-217e-476c-be0f-1cdbe52b6721
532024-02-22T18:37:52.188ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
542024-02-22T18:37:52.188ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
552024-02-22T18:37:52.188ZINFOcrucible-agent (worker): commit
562024-02-22T18:37:52.197ZINFOcrucible-agent (worker): ok!
572024-02-22T18:37:52.198ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-9f852063-217e-476c-be0f-1cdbe52b6721-5d9df203-ce89-4f7f-9ce1-08b8316a4558
582024-02-22T18:37:52.201ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-9f852063-217e-476c-be0f-1cdbe52b6721-5d9df203-ce89-4f7f-9ce1-08b8316a4558
592024-02-22T18:37:52.213ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-9f852063-217e-476c-be0f-1cdbe52b6721-5d9df203-ce89-4f7f-9ce1-08b8316a4558
602024-02-22T18:37:52.213ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9f852063-217e-476c-be0f-1cdbe52b6721/.zfs/snapshot/5d9df203-ce89-4f7f-9ce1-08b8316a4558
612024-02-22T18:37:52.213ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
622024-02-22T18:37:52.213ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
632024-02-22T18:37:52.213ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
642024-02-22T18:37:52.213ZINFOcrucible-agent (worker): commit
652024-02-22T18:37:52.219ZINFOcrucible-agent (worker): ok!
662024-02-22T18:37:52.225ZINFOcrucible-agent (worker): SMF ok!
672024-02-22T18:37:52.225ZINFOcrucible-agent (datafile): region 9f852063-217e-476c-be0f-1cdbe52b6721 running snapshot 5d9df203-ce89-4f7f-9ce1-08b8316a4558 state: Requested -> Created