Max level shown:
1 [ Feb 21 06:52:13 Enabled. ]
2 [ Feb 21 06:52:13 Rereading configuration. ]
3 [ Feb 21 06:52:19 Rereading configuration. ]
4 [ Feb 21 06:52:27 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_420ee76b-6585-471a-bbec-521f93b747c7/crucible -l [fd00:1122:3344:101::19]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-02-21T06:52:27.878ZINFOcrucible-agent: dataset: "oxp_420ee76b-6585-471a-bbec-521f93b747c7/crucible"
72025-02-21T06:52:27.881ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::19]:32345
82025-02-21T06:52:27.881ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-02-21T06:52:27.896ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-02-21T06:52:28.010ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::19]:32345
112025-02-21T06:53:19.880ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:50562
122025-02-21T06:53:19.959ZINFOcrucible-agent (datafile): region f263c969-c4eb-4a71-b599-e6609507e8b0 state: Requested
132025-02-21T06:53:19.979ZINFOcrucible-agent (dropshot): request completed latency_us = 39778 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50562 req_id = a72b3720-1d12-4e02-b15e-e0f6fadce9bf response_code = 200 uri = /crucible/0/regions
142025-02-21T06:53:19.979ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-02-21T06:53:19.987ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_420ee76b-6585-471a-bbec-521f93b747c7/crucible/regions/f263c969-c4eb-4a71-b599-e6609507e8b0
162025-02-21T06:53:19.987ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_420ee76b-6585-471a-bbec-521f93b747c7/crucible/regions/f263c969-c4eb-4a71-b599-e6609507e8b0
172025-02-21T06:53:20.141ZINFOcrucible-agent (worker): creating region Region { id: RegionId("f263c969-c4eb-4a71-b599-e6609507e8b0"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/f263c969-c4eb-4a71-b599-e6609507e8b0" region = f263c969-c4eb-4a71-b599-e6609507e8b0
182025-02-21T06:53:20.147ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "f263c969-c4eb-4a71-b599-e6609507e8b0" "--data" "/data/regions/f263c969-c4eb-4a71-b599-e6609507e8b0" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = f263c969-c4eb-4a71-b599-e6609507e8b0
192025-02-21T06:53:20.179ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50562 req_id = 37affef0-4a6c-4d36-aa9d-302eafafcb82 response_code = 200 uri = /crucible/0/regions
202025-02-21T06:53:20.807ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50562 req_id = b9768d73-c218-4513-923d-13be25cb1e8c response_code = 200 uri = /crucible/0/regions
212025-02-21T06:53:21.222ZINFOcrucible-agent (worker): region files created ok region = f263c969-c4eb-4a71-b599-e6609507e8b0
222025-02-21T06:53:21.222ZINFOcrucible-agent (datafile): region f263c969-c4eb-4a71-b599-e6609507e8b0 state: Requested -> Created
232025-02-21T06:53:21.222ZINFOcrucible-agent (worker): applying SMF actions post create...
242025-02-21T06:53:21.223ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-f263c969-c4eb-4a71-b599-e6609507e8b0
252025-02-21T06:53:21.289ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-f263c969-c4eb-4a71-b599-e6609507e8b0
262025-02-21T06:53:21.366ZINFOcrucible-agent (worker): creating config property group
272025-02-21T06:53:21.392ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-f263c969-c4eb-4a71-b599-e6609507e8b0
282025-02-21T06:53:21.392ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f263c969-c4eb-4a71-b599-e6609507e8b0
292025-02-21T06:53:21.403ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302025-02-21T06:53:21.403ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
312025-02-21T06:53:21.403ZINFOcrucible-agent (worker): commit
322025-02-21T06:53:21.486ZINFOcrucible-agent (worker): ok!
332025-02-21T06:53:21.500ZINFOcrucible-agent (worker): SMF ok!
342025-02-21T06:53:22.124ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50562 req_id = 0079e05c-9766-43a8-9a87-a42f38169584 response_code = 200 uri = /crucible/0/regions
352025-02-21T06:53:23.668ZINFOcrucible-agent (dropshot): request completed latency_us = 50 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:50562 req_id = bf34782e-1f2d-49a5-b84c-ae8283b3b888 response_code = 200 uri = /crucible/0/regions/f263c969-c4eb-4a71-b599-e6609507e8b0
362025-02-21T06:53:23.671ZINFOcrucible-agent (datafile): path is "/data/regions/f263c969-c4eb-4a71-b599-e6609507e8b0"
372025-02-21T06:53:23.682ZINFOcrucible-agent (datafile): dataset is oxp_420ee76b-6585-471a-bbec-521f93b747c7/crucible/regions/f263c969-c4eb-4a71-b599-e6609507e8b0
382025-02-21T06:53:23.692ZINFOcrucible-agent (datafile): region f263c969-c4eb-4a71-b599-e6609507e8b0 state: Created -> Tombstoned
392025-02-21T06:53:23.692ZINFOcrucible-agent (worker): applying SMF actions before removal...
402025-02-21T06:53:23.694ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-f263c969-c4eb-4a71-b599-e6609507e8b0 (instance states: (Some(Online), None))
412025-02-21T06:53:23.695ZINFOcrucible-agent (dropshot): request completed latency_us = 23795 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:50562 req_id = 73c6b8b4-636f-40f8-bfdc-5d256f49862e response_code = 204 uri = /crucible/0/regions/f263c969-c4eb-4a71-b599-e6609507e8b0
422025-02-21T06:53:23.698ZINFOcrucible-agent (dropshot): request completed latency_us = 23 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:50562 req_id = 384769ec-baf4-4134-9ef6-c2982e8323f8 response_code = 200 uri = /crucible/0/regions/f263c969-c4eb-4a71-b599-e6609507e8b0
432025-02-21T06:53:23.717ZINFOcrucible-agent (worker): SMF ok!
442025-02-21T06:53:23.730ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_420ee76b-6585-471a-bbec-521f93b747c7/crucible/regions/f263c969-c4eb-4a71-b599-e6609507e8b0" region = f263c969-c4eb-4a71-b599-e6609507e8b0
452025-02-21T06:53:23.844ZINFOcrucible-agent (dropshot): request completed latency_us = 86 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:50562 req_id = 136d7984-e416-4f8c-94d4-a207a6ec18a0 response_code = 200 uri = /crucible/0/regions/f263c969-c4eb-4a71-b599-e6609507e8b0
462025-02-21T06:53:23.935ZINFOcrucible-agent (datafile): region f263c969-c4eb-4a71-b599-e6609507e8b0 state: Tombstoned -> Destroyed
472025-02-21T06:53:24.026ZINFOcrucible-agent (dropshot): request completed latency_us = 44 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:50562 req_id = ff446f03-cefd-4a57-8825-80d98a8625ac response_code = 200 uri = /crucible/0/regions/f263c969-c4eb-4a71-b599-e6609507e8b0
482025-02-21T06:53:28.342ZINFOcrucible-agent (datafile): region ef4c8524-0593-438c-ba5f-25f04a918977 state: Requested
492025-02-21T06:53:28.344ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
502025-02-21T06:53:28.344ZINFOcrucible-agent (dropshot): request completed latency_us = 2459 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50562 req_id = c549accd-6c63-49d8-a21d-cef3e1575676 response_code = 200 uri = /crucible/0/regions
512025-02-21T06:53:28.354ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_420ee76b-6585-471a-bbec-521f93b747c7/crucible/regions/ef4c8524-0593-438c-ba5f-25f04a918977
522025-02-21T06:53:28.354ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_420ee76b-6585-471a-bbec-521f93b747c7/crucible/regions/ef4c8524-0593-438c-ba5f-25f04a918977
532025-02-21T06:53:28.494ZINFOcrucible-agent (worker): creating region Region { id: RegionId("ef4c8524-0593-438c-ba5f-25f04a918977"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/ef4c8524-0593-438c-ba5f-25f04a918977" region = ef4c8524-0593-438c-ba5f-25f04a918977
542025-02-21T06:53:28.496ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "ef4c8524-0593-438c-ba5f-25f04a918977" "--data" "/data/regions/ef4c8524-0593-438c-ba5f-25f04a918977" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = ef4c8524-0593-438c-ba5f-25f04a918977
552025-02-21T06:53:28.520ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50562 req_id = d3928239-97b3-4b3b-b2f3-67807bc12190 response_code = 200 uri = /crucible/0/regions
562025-02-21T06:53:28.784ZINFOcrucible-agent (worker): region files created ok region = ef4c8524-0593-438c-ba5f-25f04a918977
572025-02-21T06:53:28.784ZINFOcrucible-agent (datafile): region ef4c8524-0593-438c-ba5f-25f04a918977 state: Requested -> Created
582025-02-21T06:53:28.785ZINFOcrucible-agent (worker): applying SMF actions post create...
592025-02-21T06:53:28.786ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-ef4c8524-0593-438c-ba5f-25f04a918977
602025-02-21T06:53:28.794ZINFOcrucible-agent (dropshot): request completed latency_us = 51 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:50562 req_id = 9c8450ce-213a-4299-8034-8617fe85f6d6 response_code = 200 uri = /crucible/0/regions
612025-02-21T06:53:28.807ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-ef4c8524-0593-438c-ba5f-25f04a918977
622025-02-21T06:53:28.867ZINFOcrucible-agent (worker): creating config property group
632025-02-21T06:53:28.947ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-ef4c8524-0593-438c-ba5f-25f04a918977
642025-02-21T06:53:28.947ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ef4c8524-0593-438c-ba5f-25f04a918977
652025-02-21T06:53:28.947ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
662025-02-21T06:53:28.947ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
672025-02-21T06:53:28.947ZINFOcrucible-agent (worker): commit
682025-02-21T06:53:29.037ZINFOcrucible-agent (worker): ok!
692025-02-21T06:53:29.109ZINFOcrucible-agent (worker): SMF ok!