Max level shown:
1 [ Nov 22 05:53:27 Enabled. ]
2 [ Nov 22 05:53:27 Rereading configuration. ]
3 [ Nov 22 05:53:32 Rereading configuration. ]
4 [ Nov 22 05:53:38 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_38f500bf-3a39-433c-8745-e20b3ab66215/crucible -l [fd00:1122:3344:101::17]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-11-22T05:53:38.563ZINFOcrucible-agent: dataset: "oxp_38f500bf-3a39-433c-8745-e20b3ab66215/crucible"
72024-11-22T05:53:38.581ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::17]:32345
82024-11-22T05:53:38.581ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-11-22T05:53:38.587ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-11-22T05:53:38.735ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::17]:32345
112024-11-22T05:54:11.308ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:48549
122024-11-22T05:54:11.333ZINFOcrucible-agent (datafile): region 20274a32-2d0e-4fb2-ac9e-ebca1105634b state: Requested
132024-11-22T05:54:11.336ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142024-11-22T05:54:11.336ZINFOcrucible-agent (dropshot): request completed latency_us = 5052 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:48549 req_id = ca91c19e-5608-4998-bd97-2ed944997ee9 response_code = 200 uri = /crucible/0/regions
152024-11-22T05:54:11.346ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_38f500bf-3a39-433c-8745-e20b3ab66215/crucible/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b
162024-11-22T05:54:11.346ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_38f500bf-3a39-433c-8745-e20b3ab66215/crucible/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b
172024-11-22T05:54:11.415ZINFOcrucible-agent (worker): creating region Region { id: RegionId("20274a32-2d0e-4fb2-ac9e-ebca1105634b"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b" region = 20274a32-2d0e-4fb2-ac9e-ebca1105634b
182024-11-22T05:54:11.415ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "20274a32-2d0e-4fb2-ac9e-ebca1105634b" "--data" "/data/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 20274a32-2d0e-4fb2-ac9e-ebca1105634b
192024-11-22T05:54:11.575ZINFOcrucible-agent (dropshot): request completed latency_us = 78 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:48549 req_id = b1084d6b-6202-410e-9341-1e3f724d03d5 response_code = 200 uri = /crucible/0/regions
202024-11-22T05:54:11.592ZINFOcrucible-agent (worker): region files created ok region = 20274a32-2d0e-4fb2-ac9e-ebca1105634b
212024-11-22T05:54:11.592ZINFOcrucible-agent (datafile): region 20274a32-2d0e-4fb2-ac9e-ebca1105634b state: Requested -> Created
222024-11-22T05:54:11.592ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-11-22T05:54:11.592ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-20274a32-2d0e-4fb2-ac9e-ebca1105634b
242024-11-22T05:54:11.598ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-20274a32-2d0e-4fb2-ac9e-ebca1105634b
252024-11-22T05:54:11.611ZINFOcrucible-agent (worker): creating config property group
262024-11-22T05:54:11.614ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-20274a32-2d0e-4fb2-ac9e-ebca1105634b
272024-11-22T05:54:11.616ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b
282024-11-22T05:54:11.616ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-11-22T05:54:11.616ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
302024-11-22T05:54:11.616ZINFOcrucible-agent (worker): commit
312024-11-22T05:54:11.620ZINFOcrucible-agent (worker): ok!
322024-11-22T05:54:11.624ZINFOcrucible-agent (worker): SMF ok!
332024-11-22T05:54:11.855ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:48549 req_id = a1c20484-75ad-46ef-9807-ae4719d714ef response_code = 200 uri = /crucible/0/regions
342024-11-22T05:54:12.376ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:48549 req_id = 90d9f422-32cd-4eb6-83c3-ada5a53a17ec response_code = 200 uri = /crucible/0/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b
352024-11-22T05:54:12.378ZINFOcrucible-agent (datafile): path is "/data/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b"
362024-11-22T05:54:12.389ZINFOcrucible-agent (datafile): dataset is oxp_38f500bf-3a39-433c-8745-e20b3ab66215/crucible/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b
372024-11-22T05:54:12.398ZINFOcrucible-agent (datafile): region 20274a32-2d0e-4fb2-ac9e-ebca1105634b state: Created -> Tombstoned
382024-11-22T05:54:12.399ZINFOcrucible-agent (worker): applying SMF actions before removal...
392024-11-22T05:54:12.399ZINFOcrucible-agent (dropshot): request completed latency_us = 20613 local_addr = [fd00:1122:3344:101::17]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:48549 req_id = bc614cc9-bc7a-4d72-9620-ff90f1a399a7 response_code = 204 uri = /crucible/0/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b
402024-11-22T05:54:12.400ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-20274a32-2d0e-4fb2-ac9e-ebca1105634b (instance states: (Some(Online), None))
412024-11-22T05:54:12.401ZINFOcrucible-agent (dropshot): request completed latency_us = 23 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:48549 req_id = f95d5aec-214f-4df2-b6b9-1db0b2b462a5 response_code = 200 uri = /crucible/0/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b
422024-11-22T05:54:12.404ZINFOcrucible-agent (worker): SMF ok!
432024-11-22T05:54:12.414ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_38f500bf-3a39-433c-8745-e20b3ab66215/crucible/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b" region = 20274a32-2d0e-4fb2-ac9e-ebca1105634b
442024-11-22T05:54:12.518ZINFOcrucible-agent (datafile): region 20274a32-2d0e-4fb2-ac9e-ebca1105634b state: Tombstoned -> Destroyed
452024-11-22T05:54:12.550ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:48549 req_id = d5ee5004-4ce0-440b-88e8-484ca8cf4c2e response_code = 200 uri = /crucible/0/regions/20274a32-2d0e-4fb2-ac9e-ebca1105634b
462024-11-22T05:54:47.944ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:38085
472024-11-22T05:54:47.944ZINFOcrucible-agent (datafile): region 30d5b175-b9c3-4f49-ae6e-5ca0f63049e6 state: Requested
482024-11-22T05:54:47.944ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-11-22T05:54:47.945ZINFOcrucible-agent (dropshot): request completed latency_us = 244 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:38085 req_id = 6ce94af5-8e03-4d14-ba4b-01e522367fe9 response_code = 200 uri = /crucible/0/regions
502024-11-22T05:54:47.955ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_38f500bf-3a39-433c-8745-e20b3ab66215/crucible/regions/30d5b175-b9c3-4f49-ae6e-5ca0f63049e6
512024-11-22T05:54:47.955ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_38f500bf-3a39-433c-8745-e20b3ab66215/crucible/regions/30d5b175-b9c3-4f49-ae6e-5ca0f63049e6
522024-11-22T05:54:48.049ZINFOcrucible-agent (worker): creating region Region { id: RegionId("30d5b175-b9c3-4f49-ae6e-5ca0f63049e6"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/30d5b175-b9c3-4f49-ae6e-5ca0f63049e6" region = 30d5b175-b9c3-4f49-ae6e-5ca0f63049e6
532024-11-22T05:54:48.049ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "30d5b175-b9c3-4f49-ae6e-5ca0f63049e6" "--data" "/data/regions/30d5b175-b9c3-4f49-ae6e-5ca0f63049e6" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 30d5b175-b9c3-4f49-ae6e-5ca0f63049e6
542024-11-22T05:54:48.249ZINFOcrucible-agent (worker): region files created ok region = 30d5b175-b9c3-4f49-ae6e-5ca0f63049e6
552024-11-22T05:54:48.249ZINFOcrucible-agent (datafile): region 30d5b175-b9c3-4f49-ae6e-5ca0f63049e6 state: Requested -> Created
562024-11-22T05:54:48.249ZINFOcrucible-agent (worker): applying SMF actions post create...
572024-11-22T05:54:48.250ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-30d5b175-b9c3-4f49-ae6e-5ca0f63049e6
582024-11-22T05:54:48.251ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:38085 req_id = 99220cb5-019c-47c6-ac47-6f24409e95c9 response_code = 200 uri = /crucible/0/regions
592024-11-22T05:54:48.263ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-30d5b175-b9c3-4f49-ae6e-5ca0f63049e6
602024-11-22T05:54:48.289ZINFOcrucible-agent (worker): creating config property group
612024-11-22T05:54:48.311ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-30d5b175-b9c3-4f49-ae6e-5ca0f63049e6
622024-11-22T05:54:48.311ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/30d5b175-b9c3-4f49-ae6e-5ca0f63049e6
632024-11-22T05:54:48.311ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
642024-11-22T05:54:48.311ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
652024-11-22T05:54:48.311ZINFOcrucible-agent (worker): commit
662024-11-22T05:54:48.326ZINFOcrucible-agent (worker): ok!
672024-11-22T05:54:48.337ZINFOcrucible-agent (worker): SMF ok!