Max level shown:
1 [ Dec 21 00:16:48 Enabled. ]
2 [ Dec 21 00:16:48 Rereading configuration. ]
3 [ Dec 21 00:16:50 Rereading configuration. ]
4 [ Dec 21 00:16:54 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_db606984-291b-40d5-964e-14a228d68e45/crucible -l [fd00:1122:3344:101::15]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-12-21T00:16:54.723ZINFOcrucible-agent: dataset: "oxp_db606984-291b-40d5-964e-14a228d68e45/crucible"
72024-12-21T00:16:54.727ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::15]:32345
82024-12-21T00:16:54.727ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-12-21T00:16:54.744ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-12-21T00:16:54.881ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::15]:32345
112024-12-21T00:17:53.690ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:49610
122024-12-21T00:17:53.713ZINFOcrucible-agent (datafile): region dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9 state: Requested
132024-12-21T00:17:53.713ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
142024-12-21T00:17:53.719ZINFOcrucible-agent (dropshot): request completed latency_us = 10078 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49610 req_id = dd67cc01-b1d9-4373-8ee0-b8eece91e88d response_code = 200 uri = /crucible/0/regions
152024-12-21T00:17:53.725ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
162024-12-21T00:17:53.725ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
172024-12-21T00:17:53.777ZINFOcrucible-agent (worker): creating region Region { id: RegionId("dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9" region = dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
182024-12-21T00:17:53.778ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9" "--data" "/data/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
192024-12-21T00:17:53.912ZINFOcrucible-agent (worker): region files created ok region = dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
202024-12-21T00:17:53.912ZINFOcrucible-agent (datafile): region dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9 state: Requested -> Created
212024-12-21T00:17:53.912ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-12-21T00:17:53.912ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
232024-12-21T00:17:53.917ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
242024-12-21T00:17:53.928ZINFOcrucible-agent (worker): creating config property group
252024-12-21T00:17:53.931ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
262024-12-21T00:17:53.931ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
272024-12-21T00:17:53.931ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
282024-12-21T00:17:53.931ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
292024-12-21T00:17:53.931ZINFOcrucible-agent (worker): commit
302024-12-21T00:17:53.936ZINFOcrucible-agent (worker): ok!
312024-12-21T00:17:53.939ZINFOcrucible-agent (worker): SMF ok!
322024-12-21T00:17:53.965ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49610 req_id = 69486c71-3a21-463c-9011-8a90c8d310e0 response_code = 200 uri = /crucible/0/regions
332024-12-21T00:28:14.812ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:64258
342024-12-21T00:28:14.815ZINFOcrucible-agent (dropshot): request completed latency_us = 2952 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:64258 req_id = 1745cc6e-2201-4156-9862-07e9eb23e7cc response_code = 200 uri = /crucible/0/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
352024-12-21T00:28:14.818ZINFOcrucible-agent (datafile): path is "/data/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9"
362024-12-21T00:28:14.827ZINFOcrucible-agent (datafile): dataset is oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
372024-12-21T00:28:14.836ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9@9e83aa1a-7eee-477e-bf34-353a0eb8bb47
382024-12-21T00:28:14.836ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9", "9e83aa1a-7eee-477e-bf34-353a0eb8bb47"]
392024-12-21T00:28:14.848ZINFOcrucible-agent (dropshot): request completed latency_us = 30242 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:64258 req_id = 6cf1707f-68af-41f6-9fa3-9191b5c0c796 response_code = 200 uri = /crucible/0/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9/snapshots/9e83aa1a-7eee-477e-bf34-353a0eb8bb47
402024-12-21T00:28:14.851ZINFOcrucible-agent (datafile): path is "/data/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9"
412024-12-21T00:28:14.860ZINFOcrucible-agent (datafile): dataset is oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
422024-12-21T00:28:14.869ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9@9e83aa1a-7eee-477e-bf34-353a0eb8bb47
432024-12-21T00:28:14.869ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9", "9e83aa1a-7eee-477e-bf34-353a0eb8bb47"]
442024-12-21T00:28:14.880ZINFOcrucible-agent (datafile): requesting running snapshot dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9-9e83aa1a-7eee-477e-bf34-353a0eb8bb47 state: Requested
452024-12-21T00:28:14.880ZINFOcrucible-agent (worker): applying SMF actions for region dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9 running snapshot 9e83aa1a-7eee-477e-bf34-353a0eb8bb47 (state Requested)...
462024-12-21T00:28:14.880ZINFOcrucible-agent (dropshot): request completed latency_us = 28920 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64258 req_id = 52ac8828-3e4b-49c7-a353-12ec1abd1195 response_code = 200 uri = /crucible/0/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9/snapshots/9e83aa1a-7eee-477e-bf34-353a0eb8bb47/run
472024-12-21T00:28:14.892ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9-9e83aa1a-7eee-477e-bf34-353a0eb8bb47
482024-12-21T00:28:14.896ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9-9e83aa1a-7eee-477e-bf34-353a0eb8bb47
492024-12-21T00:28:14.908ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9-9e83aa1a-7eee-477e-bf34-353a0eb8bb47
502024-12-21T00:28:14.908ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9/.zfs/snapshot/9e83aa1a-7eee-477e-bf34-353a0eb8bb47
512024-12-21T00:28:14.908ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
522024-12-21T00:28:14.908ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
532024-12-21T00:28:14.908ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
542024-12-21T00:28:14.908ZINFOcrucible-agent (worker): commit
552024-12-21T00:28:14.912ZINFOcrucible-agent (worker): ok!
562024-12-21T00:28:14.916ZINFOcrucible-agent (worker): SMF ok!
572024-12-21T00:28:14.916ZINFOcrucible-agent (datafile): region dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9 running snapshot 9e83aa1a-7eee-477e-bf34-353a0eb8bb47 state: Requested -> Created
582024-12-21T00:28:15.113ZINFOcrucible-agent (datafile): path is "/data/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9"
592024-12-21T00:28:15.122ZINFOcrucible-agent (datafile): dataset is oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9
602024-12-21T00:28:15.130ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9@9e83aa1a-7eee-477e-bf34-353a0eb8bb47
612024-12-21T00:28:15.130ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_db606984-291b-40d5-964e-14a228d68e45/crucible/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9", "9e83aa1a-7eee-477e-bf34-353a0eb8bb47"]
622024-12-21T00:28:15.139ZINFOcrucible-agent (dropshot): request completed latency_us = 27171 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64258 req_id = 99af0c26-9170-4557-afa5-dd3a618b1a6b response_code = 200 uri = /crucible/0/regions/dde6d80d-ddf1-4fa5-b3f2-027cca69fdc9/snapshots/9e83aa1a-7eee-477e-bf34-353a0eb8bb47/run