Max level shown:
1 [ May 6 18:53:05 Enabled. ]
2 [ May 6 18:53:05 Rereading configuration. ]
3 [ May 6 18:53:10 Rereading configuration. ]
4 [ May 6 18:53:17 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_ad34c00c-d848-4310-92c7-641ce5863a7d/crucible -l [fd00:1122:3344:101::16]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-05-06T18:53:17.720ZINFOcrucible-agent: dataset: "oxp_ad34c00c-d848-4310-92c7-641ce5863a7d/crucible"
72024-05-06T18:53:17.723ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::16]:32345
82024-05-06T18:53:17.723ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-05-06T18:53:17.741ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-05-06T18:53:17.831ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::16]:32345
112024-05-06T18:54:28.849ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:57563
122024-05-06T18:54:28.912ZINFOcrucible-agent (datafile): region ee3e5f20-1770-47da-8ed0-d007900044ff state: Requested
132024-05-06T18:54:28.915ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
142024-05-06T18:54:28.915ZINFOcrucible-agent (dropshot): request completed latency_us = 9413 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:57563 req_id = efdfc93d-bf21-4016-8b5e-0478d781e36e response_code = 200 uri = /crucible/0/regions
152024-05-06T18:54:28.926ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_ad34c00c-d848-4310-92c7-641ce5863a7d/crucible/regions/ee3e5f20-1770-47da-8ed0-d007900044ff
162024-05-06T18:54:28.926ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_ad34c00c-d848-4310-92c7-641ce5863a7d/crucible/regions/ee3e5f20-1770-47da-8ed0-d007900044ff
172024-05-06T18:54:28.976ZINFOcrucible-agent (worker): creating region Region { id: RegionId("ee3e5f20-1770-47da-8ed0-d007900044ff"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/ee3e5f20-1770-47da-8ed0-d007900044ff" region = ee3e5f20-1770-47da-8ed0-d007900044ff
182024-05-06T18:54:28.978ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "ee3e5f20-1770-47da-8ed0-d007900044ff" "--data" "/data/regions/ee3e5f20-1770-47da-8ed0-d007900044ff" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = ee3e5f20-1770-47da-8ed0-d007900044ff
192024-05-06T18:54:29.089ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:57563 req_id = 9d059cb4-4177-4e00-b004-4bd7f01a8544 response_code = 200 uri = /crucible/0/regions
202024-05-06T18:54:29.106ZINFOcrucible-agent (worker): region files created ok region = ee3e5f20-1770-47da-8ed0-d007900044ff
212024-05-06T18:54:29.106ZINFOcrucible-agent (datafile): region ee3e5f20-1770-47da-8ed0-d007900044ff state: Requested -> Created
222024-05-06T18:54:29.106ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-05-06T18:54:29.106ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-ee3e5f20-1770-47da-8ed0-d007900044ff
242024-05-06T18:54:29.110ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-ee3e5f20-1770-47da-8ed0-d007900044ff
252024-05-06T18:54:29.124ZINFOcrucible-agent (worker): creating config property group
262024-05-06T18:54:29.127ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-ee3e5f20-1770-47da-8ed0-d007900044ff
272024-05-06T18:54:29.127ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ee3e5f20-1770-47da-8ed0-d007900044ff
282024-05-06T18:54:29.128ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-05-06T18:54:29.128ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
302024-05-06T18:54:29.128ZINFOcrucible-agent (worker): commit
312024-05-06T18:54:29.131ZINFOcrucible-agent (worker): ok!
322024-05-06T18:54:29.135ZINFOcrucible-agent (worker): SMF ok!
332024-05-06T18:54:29.353ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:57563 req_id = 0c03ed6a-1a6f-4ef2-84c5-1dc8e7b69b02 response_code = 200 uri = /crucible/0/regions
342024-05-06T18:59:55.177ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:61468
352024-05-06T18:59:55.177ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:61468 req_id = a08f30cf-79eb-48f1-9530-a589bdea6ab2 response_code = 200 uri = /crucible/0/regions/ee3e5f20-1770-47da-8ed0-d007900044ff
362024-05-06T18:59:55.178ZINFOcrucible-agent (datafile): path is "/data/regions/ee3e5f20-1770-47da-8ed0-d007900044ff"
372024-05-06T18:59:55.187ZINFOcrucible-agent (datafile): dataset is oxp_ad34c00c-d848-4310-92c7-641ce5863a7d/crucible/regions/ee3e5f20-1770-47da-8ed0-d007900044ff
382024-05-06T18:59:55.195ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ad34c00c-d848-4310-92c7-641ce5863a7d/crucible/regions/ee3e5f20-1770-47da-8ed0-d007900044ff@2741d331-a7ac-4ab3-9b92-e5769c6b8cab
392024-05-06T18:59:55.195ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ad34c00c-d848-4310-92c7-641ce5863a7d/crucible/regions/ee3e5f20-1770-47da-8ed0-d007900044ff", "2741d331-a7ac-4ab3-9b92-e5769c6b8cab"]
402024-05-06T18:59:55.212ZINFOcrucible-agent (dropshot): request completed latency_us = 34139 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:61468 req_id = d9490396-b27a-4ab7-bd01-e5245328cba6 response_code = 200 uri = /crucible/0/regions/ee3e5f20-1770-47da-8ed0-d007900044ff/snapshots/2741d331-a7ac-4ab3-9b92-e5769c6b8cab
412024-05-06T18:59:55.212ZINFOcrucible-agent (datafile): path is "/data/regions/ee3e5f20-1770-47da-8ed0-d007900044ff"
422024-05-06T18:59:55.220ZINFOcrucible-agent (datafile): dataset is oxp_ad34c00c-d848-4310-92c7-641ce5863a7d/crucible/regions/ee3e5f20-1770-47da-8ed0-d007900044ff
432024-05-06T18:59:55.228ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ad34c00c-d848-4310-92c7-641ce5863a7d/crucible/regions/ee3e5f20-1770-47da-8ed0-d007900044ff@2741d331-a7ac-4ab3-9b92-e5769c6b8cab
442024-05-06T18:59:55.228ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ad34c00c-d848-4310-92c7-641ce5863a7d/crucible/regions/ee3e5f20-1770-47da-8ed0-d007900044ff", "2741d331-a7ac-4ab3-9b92-e5769c6b8cab"]
452024-05-06T18:59:55.237ZINFOcrucible-agent (datafile): requesting running snapshot ee3e5f20-1770-47da-8ed0-d007900044ff-2741d331-a7ac-4ab3-9b92-e5769c6b8cab state: Requested
462024-05-06T18:59:55.237ZINFOcrucible-agent (worker): applying SMF actions for region ee3e5f20-1770-47da-8ed0-d007900044ff running snapshot 2741d331-a7ac-4ab3-9b92-e5769c6b8cab (state Requested)...
472024-05-06T18:59:55.237ZINFOcrucible-agent (dropshot): request completed latency_us = 25492 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61468 req_id = 7033202d-868b-4218-bb91-28615dee4245 response_code = 200 uri = /crucible/0/regions/ee3e5f20-1770-47da-8ed0-d007900044ff/snapshots/2741d331-a7ac-4ab3-9b92-e5769c6b8cab/run
482024-05-06T18:59:55.249ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-ee3e5f20-1770-47da-8ed0-d007900044ff-2741d331-a7ac-4ab3-9b92-e5769c6b8cab
492024-05-06T18:59:55.255ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-ee3e5f20-1770-47da-8ed0-d007900044ff-2741d331-a7ac-4ab3-9b92-e5769c6b8cab
502024-05-06T18:59:55.266ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-ee3e5f20-1770-47da-8ed0-d007900044ff-2741d331-a7ac-4ab3-9b92-e5769c6b8cab
512024-05-06T18:59:55.266ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ee3e5f20-1770-47da-8ed0-d007900044ff/.zfs/snapshot/2741d331-a7ac-4ab3-9b92-e5769c6b8cab
522024-05-06T18:59:55.266ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
532024-05-06T18:59:55.266ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
542024-05-06T18:59:55.266ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
552024-05-06T18:59:55.266ZINFOcrucible-agent (worker): commit
562024-05-06T18:59:55.270ZINFOcrucible-agent (worker): ok!
572024-05-06T18:59:55.274ZINFOcrucible-agent (worker): SMF ok!
582024-05-06T18:59:55.274ZINFOcrucible-agent (datafile): region ee3e5f20-1770-47da-8ed0-d007900044ff running snapshot 2741d331-a7ac-4ab3-9b92-e5769c6b8cab state: Requested -> Created