Max level shown:
1 [ Mar 11 14:39:06 Enabled. ]
2 [ Mar 11 14:39:06 Rereading configuration. ]
3 [ Mar 11 14:39:07 Rereading configuration. ]
4 [ Mar 11 14:39:09 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible -l [fd00:1122:3344:101::16]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-03-11T14:39:09.756ZINFOcrucible-agent: dataset: "oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible"
72024-03-11T14:39:09.764ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::16]:32345
82024-03-11T14:39:09.764ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-03-11T14:39:09.946ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::16]:32345
102024-03-11T14:39:33.862ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::c]:46469
112024-03-11T14:39:33.865ZINFOcrucible-agent (datafile): region 2760e400-74e8-4695-aca3-42275b3e2d0b state: Requested
122024-03-11T14:39:33.865ZINFOcrucible-agent (dropshot): request completed latency_us = 1570 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:46469 req_id = c98d5f9e-5ccb-4087-bf99-41d889fcd1bd response_code = 200 uri = /crucible/0/regions
132024-03-11T14:39:33.866ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142024-03-11T14:39:33.875ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/2760e400-74e8-4695-aca3-42275b3e2d0b
152024-03-11T14:39:33.875ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/2760e400-74e8-4695-aca3-42275b3e2d0b
162024-03-11T14:39:33.935ZINFOcrucible-agent (worker): creating region Region { id: RegionId("2760e400-74e8-4695-aca3-42275b3e2d0b"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None } at "/data/regions/2760e400-74e8-4695-aca3-42275b3e2d0b" region = 2760e400-74e8-4695-aca3-42275b3e2d0b
172024-03-11T14:39:34.037ZINFOcrucible-agent (worker): region files created ok region = 2760e400-74e8-4695-aca3-42275b3e2d0b
182024-03-11T14:39:34.037ZINFOcrucible-agent (datafile): region 2760e400-74e8-4695-aca3-42275b3e2d0b state: Requested -> Created
192024-03-11T14:39:34.037ZINFOcrucible-agent (worker): applying SMF actions post create...
202024-03-11T14:39:34.038ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-2760e400-74e8-4695-aca3-42275b3e2d0b
212024-03-11T14:39:34.040ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-2760e400-74e8-4695-aca3-42275b3e2d0b
222024-03-11T14:39:34.050ZINFOcrucible-agent (worker): creating config property group
232024-03-11T14:39:34.053ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-2760e400-74e8-4695-aca3-42275b3e2d0b
242024-03-11T14:39:34.053ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2760e400-74e8-4695-aca3-42275b3e2d0b
252024-03-11T14:39:34.054ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
262024-03-11T14:39:34.054ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
272024-03-11T14:39:34.054ZINFOcrucible-agent (worker): commit
282024-03-11T14:39:34.058ZINFOcrucible-agent (worker): ok!
292024-03-11T14:39:34.061ZINFOcrucible-agent (worker): SMF ok!
302024-03-11T14:39:34.242ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:46469 req_id = c8e7fc37-b54a-4348-89cd-b18ab818d192 response_code = 200 uri = /crucible/0/regions
312024-03-11T14:39:34.976ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::c]:52630
322024-03-11T14:39:34.976ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:52630 req_id = 05e15d99-1f35-4fad-b05f-c9ea6de7eaa0 response_code = 200 uri = /crucible/0/regions/2760e400-74e8-4695-aca3-42275b3e2d0b
332024-03-11T14:39:34.977ZINFOcrucible-agent (datafile): path is "/data/regions/2760e400-74e8-4695-aca3-42275b3e2d0b"
342024-03-11T14:39:34.986ZINFOcrucible-agent (datafile): dataset is oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/2760e400-74e8-4695-aca3-42275b3e2d0b
352024-03-11T14:39:34.995ZINFOcrucible-agent (datafile): region 2760e400-74e8-4695-aca3-42275b3e2d0b state: Created -> Tombstoned
362024-03-11T14:39:34.995ZINFOcrucible-agent (worker): applying SMF actions before removal...
372024-03-11T14:39:34.995ZINFOcrucible-agent (dropshot): request completed latency_us = 18369 local_addr = [fd00:1122:3344:101::16]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:52630 req_id = f4e0eeed-98ef-46f4-8b74-fd1a8c2a2218 response_code = 204 uri = /crucible/0/regions/2760e400-74e8-4695-aca3-42275b3e2d0b
382024-03-11T14:39:34.995ZINFOcrucible-agent (dropshot): request completed latency_us = 26 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:52630 req_id = a9b60837-46c5-4c82-8e5e-c96eda50ed01 response_code = 200 uri = /crucible/0/regions/2760e400-74e8-4695-aca3-42275b3e2d0b
392024-03-11T14:39:34.996ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-2760e400-74e8-4695-aca3-42275b3e2d0b (instance states: (Some(Online), None))
402024-03-11T14:39:35.000ZINFOcrucible-agent (worker): SMF ok!
412024-03-11T14:39:35.010ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/2760e400-74e8-4695-aca3-42275b3e2d0b" region = 2760e400-74e8-4695-aca3-42275b3e2d0b
422024-03-11T14:39:35.064ZINFOcrucible-agent (datafile): region 2760e400-74e8-4695-aca3-42275b3e2d0b state: Tombstoned -> Destroyed
432024-03-11T14:39:35.142ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:52630 req_id = 96a04043-532a-4849-84ca-d08eef0d777f response_code = 200 uri = /crucible/0/regions/2760e400-74e8-4695-aca3-42275b3e2d0b
442024-03-11T14:40:21.554ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:32808
452024-03-11T14:40:21.584ZINFOcrucible-agent (datafile): region 4a8916be-3b8b-428a-998a-25b43c780aca state: Requested
462024-03-11T14:40:21.584ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
472024-03-11T14:40:21.584ZINFOcrucible-agent (dropshot): request completed latency_us = 812 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:32808 req_id = 407e6b3a-1c6a-4c31-b368-fc983c7be2a0 response_code = 200 uri = /crucible/0/regions
482024-03-11T14:40:21.593ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/4a8916be-3b8b-428a-998a-25b43c780aca
492024-03-11T14:40:21.593ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_cd70d7f6-2354-4bf2-8012-55bf9eaf7930/crucible/regions/4a8916be-3b8b-428a-998a-25b43c780aca
502024-03-11T14:40:21.663ZINFOcrucible-agent (worker): creating region Region { id: RegionId("4a8916be-3b8b-428a-998a-25b43c780aca"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None } at "/data/regions/4a8916be-3b8b-428a-998a-25b43c780aca" region = 4a8916be-3b8b-428a-998a-25b43c780aca
512024-03-11T14:40:21.743ZINFOcrucible-agent (worker): region files created ok region = 4a8916be-3b8b-428a-998a-25b43c780aca
522024-03-11T14:40:21.743ZINFOcrucible-agent (datafile): region 4a8916be-3b8b-428a-998a-25b43c780aca state: Requested -> Created
532024-03-11T14:40:21.743ZINFOcrucible-agent (worker): applying SMF actions post create...
542024-03-11T14:40:21.744ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-4a8916be-3b8b-428a-998a-25b43c780aca
552024-03-11T14:40:21.748ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-4a8916be-3b8b-428a-998a-25b43c780aca
562024-03-11T14:40:21.758ZINFOcrucible-agent (worker): creating config property group
572024-03-11T14:40:21.762ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4a8916be-3b8b-428a-998a-25b43c780aca
582024-03-11T14:40:21.762ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4a8916be-3b8b-428a-998a-25b43c780aca
592024-03-11T14:40:21.762ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
602024-03-11T14:40:21.762ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
612024-03-11T14:40:21.762ZINFOcrucible-agent (worker): commit
622024-03-11T14:40:21.766ZINFOcrucible-agent (worker): ok!
632024-03-11T14:40:21.770ZINFOcrucible-agent (worker): SMF ok!
642024-03-11T14:40:21.789ZINFOcrucible-agent (dropshot): request completed latency_us = 623 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:32808 req_id = 95cbe83d-2dac-4862-9844-a2273d0acf96 response_code = 200 uri = /crucible/0/regions