Max level shown:
1 [ Nov 14 00:18:35 Enabled. ]
2 [ Nov 14 00:18:35 Rereading configuration. ]
3 [ Nov 14 00:18:40 Rereading configuration. ]
4 [ Nov 14 00:18:46 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_c2753ec7-decb-4fc2-b85a-bb303a9a0272/crucible -l [fd00:1122:3344:101::14]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-11-14T00:18:46.675ZINFOcrucible-agent: dataset: "oxp_c2753ec7-decb-4fc2-b85a-bb303a9a0272/crucible"
72024-11-14T00:18:46.678ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::14]:32345
82024-11-14T00:18:46.678ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-11-14T00:18:46.694ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-11-14T00:18:46.788ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::14]:32345
112024-11-14T00:29:47.575ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:52508
122024-11-14T00:29:47.618ZINFOcrucible-agent (datafile): region 181a0fb3-9b28-4d00-9212-ad78d66d0ec2 state: Requested
132024-11-14T00:29:47.626ZINFOcrucible-agent (dropshot): request completed latency_us = 22034 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52508 req_id = 5f2f362c-78e6-4b3f-bc33-a99f1696f159 response_code = 200 uri = /crucible/0/regions
142024-11-14T00:29:47.626ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
152024-11-14T00:29:47.639ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_c2753ec7-decb-4fc2-b85a-bb303a9a0272/crucible/regions/181a0fb3-9b28-4d00-9212-ad78d66d0ec2
162024-11-14T00:29:47.639ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_c2753ec7-decb-4fc2-b85a-bb303a9a0272/crucible/regions/181a0fb3-9b28-4d00-9212-ad78d66d0ec2
172024-11-14T00:29:47.689ZINFOcrucible-agent (worker): creating region Region { id: RegionId("181a0fb3-9b28-4d00-9212-ad78d66d0ec2"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/181a0fb3-9b28-4d00-9212-ad78d66d0ec2" region = 181a0fb3-9b28-4d00-9212-ad78d66d0ec2
182024-11-14T00:29:47.693ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "181a0fb3-9b28-4d00-9212-ad78d66d0ec2" "--data" "/data/regions/181a0fb3-9b28-4d00-9212-ad78d66d0ec2" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 181a0fb3-9b28-4d00-9212-ad78d66d0ec2
192024-11-14T00:29:47.923ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52508 req_id = a9abafaa-5bd3-4b08-8776-a9737f476176 response_code = 200 uri = /crucible/0/regions
202024-11-14T00:29:48.034ZINFOcrucible-agent (worker): region files created ok region = 181a0fb3-9b28-4d00-9212-ad78d66d0ec2
212024-11-14T00:29:48.034ZINFOcrucible-agent (datafile): region 181a0fb3-9b28-4d00-9212-ad78d66d0ec2 state: Requested -> Created
222024-11-14T00:29:48.035ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-11-14T00:29:48.035ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-181a0fb3-9b28-4d00-9212-ad78d66d0ec2
242024-11-14T00:29:48.041ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-181a0fb3-9b28-4d00-9212-ad78d66d0ec2
252024-11-14T00:29:48.056ZINFOcrucible-agent (worker): creating config property group
262024-11-14T00:29:48.059ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-181a0fb3-9b28-4d00-9212-ad78d66d0ec2
272024-11-14T00:29:48.060ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/181a0fb3-9b28-4d00-9212-ad78d66d0ec2
282024-11-14T00:29:48.063ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-11-14T00:29:48.063ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
302024-11-14T00:29:48.063ZINFOcrucible-agent (worker): commit
312024-11-14T00:29:48.070ZINFOcrucible-agent (worker): ok!
322024-11-14T00:29:48.074ZINFOcrucible-agent (worker): SMF ok!
332024-11-14T00:29:48.193ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52508 req_id = eed1392e-df70-4c35-83ed-8ed380f6fbb3 response_code = 200 uri = /crucible/0/regions
342024-11-14T00:29:54.329ZINFOcrucible-agent (datafile): region ed7362e3-4d5c-445e-84b0-f250003977c9 state: Requested
352024-11-14T00:29:54.329ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
362024-11-14T00:29:54.329ZINFOcrucible-agent (dropshot): request completed latency_us = 255 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52508 req_id = 96966bb9-f6d8-4504-bf60-b1f86bf6f541 response_code = 200 uri = /crucible/0/regions
372024-11-14T00:29:54.340ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_c2753ec7-decb-4fc2-b85a-bb303a9a0272/crucible/regions/ed7362e3-4d5c-445e-84b0-f250003977c9
382024-11-14T00:29:54.340ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_c2753ec7-decb-4fc2-b85a-bb303a9a0272/crucible/regions/ed7362e3-4d5c-445e-84b0-f250003977c9
392024-11-14T00:29:54.394ZINFOcrucible-agent (worker): creating region Region { id: RegionId("ed7362e3-4d5c-445e-84b0-f250003977c9"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/ed7362e3-4d5c-445e-84b0-f250003977c9" region = ed7362e3-4d5c-445e-84b0-f250003977c9
402024-11-14T00:29:54.394ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "ed7362e3-4d5c-445e-84b0-f250003977c9" "--data" "/data/regions/ed7362e3-4d5c-445e-84b0-f250003977c9" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = ed7362e3-4d5c-445e-84b0-f250003977c9
412024-11-14T00:29:54.468ZINFOcrucible-agent (worker): region files created ok region = ed7362e3-4d5c-445e-84b0-f250003977c9
422024-11-14T00:29:54.468ZINFOcrucible-agent (datafile): region ed7362e3-4d5c-445e-84b0-f250003977c9 state: Requested -> Created
432024-11-14T00:29:54.468ZINFOcrucible-agent (worker): applying SMF actions post create...
442024-11-14T00:29:54.470ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/181a0fb3-9b28-4d00-9212-ad78d66d0ec2
452024-11-14T00:29:54.470ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
462024-11-14T00:29:54.470ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::14
472024-11-14T00:29:54.470ZINFOcrucible-agent (worker): reconfig required
482024-11-14T00:29:54.470ZINFOcrucible-agent (worker): using existing config property group
492024-11-14T00:29:54.470ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-181a0fb3-9b28-4d00-9212-ad78d66d0ec2
502024-11-14T00:29:54.470ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/181a0fb3-9b28-4d00-9212-ad78d66d0ec2
512024-11-14T00:29:54.470ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
522024-11-14T00:29:54.470ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
532024-11-14T00:29:54.470ZINFOcrucible-agent (worker): commit
542024-11-14T00:29:54.475ZINFOcrucible-agent (worker): ok!
552024-11-14T00:29:54.475ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-ed7362e3-4d5c-445e-84b0-f250003977c9
562024-11-14T00:29:54.479ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-ed7362e3-4d5c-445e-84b0-f250003977c9
572024-11-14T00:29:54.489ZINFOcrucible-agent (worker): creating config property group
582024-11-14T00:29:54.492ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-ed7362e3-4d5c-445e-84b0-f250003977c9
592024-11-14T00:29:54.492ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ed7362e3-4d5c-445e-84b0-f250003977c9
602024-11-14T00:29:54.492ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
612024-11-14T00:29:54.492ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
622024-11-14T00:29:54.492ZINFOcrucible-agent (worker): commit
632024-11-14T00:29:54.497ZINFOcrucible-agent (worker): ok!
642024-11-14T00:29:54.502ZINFOcrucible-agent (worker): SMF ok!
652024-11-14T00:29:54.620ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:52508 req_id = 0ad1963a-e73f-4f4d-a7d6-54e7377a3169 response_code = 200 uri = /crucible/0/regions