Max level shown:
1 [ May 1 00:20:05 Enabled. ]
2 [ May 1 00:20:05 Rereading configuration. ]
3 [ May 1 00:20:10 Rereading configuration. ]
4 [ May 1 00:20:17 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_6464a1b3-39ff-4f6d-bb8d-28b483455b0c/crucible -l [fd00:1122:3344:101::19]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-05-01T00:20:17.408ZINFOcrucible-agent: dataset: "oxp_6464a1b3-39ff-4f6d-bb8d-28b483455b0c/crucible"
72024-05-01T00:20:17.413ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::19]:32345
82024-05-01T00:20:17.413ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-05-01T00:20:17.429ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-05-01T00:20:17.513ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::19]:32345
112024-05-01T00:20:38.696ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:58409
122024-05-01T00:20:38.733ZINFOcrucible-agent (datafile): region 3a62cd81-7d51-4dc3-ac4c-db95fda44c50 state: Requested
132024-05-01T00:20:38.738ZINFOcrucible-agent (dropshot): request completed latency_us = 9109 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58409 req_id = e5417537-6ac5-47d6-995d-44baaa257451 response_code = 200 uri = /crucible/0/regions
142024-05-01T00:20:38.738ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-05-01T00:20:38.748ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_6464a1b3-39ff-4f6d-bb8d-28b483455b0c/crucible/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50
162024-05-01T00:20:38.748ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_6464a1b3-39ff-4f6d-bb8d-28b483455b0c/crucible/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50
172024-05-01T00:20:38.817ZINFOcrucible-agent (worker): creating region Region { id: RegionId("3a62cd81-7d51-4dc3-ac4c-db95fda44c50"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50" region = 3a62cd81-7d51-4dc3-ac4c-db95fda44c50
182024-05-01T00:20:38.817ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "3a62cd81-7d51-4dc3-ac4c-db95fda44c50" "--data" "/data/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 3a62cd81-7d51-4dc3-ac4c-db95fda44c50
192024-05-01T00:20:39.028ZINFOcrucible-agent (worker): region files created ok region = 3a62cd81-7d51-4dc3-ac4c-db95fda44c50
202024-05-01T00:20:39.028ZINFOcrucible-agent (datafile): region 3a62cd81-7d51-4dc3-ac4c-db95fda44c50 state: Requested -> Created
212024-05-01T00:20:39.028ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-05-01T00:20:39.029ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-3a62cd81-7d51-4dc3-ac4c-db95fda44c50
232024-05-01T00:20:39.029ZINFOcrucible-agent (dropshot): request completed latency_us = 87 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58409 req_id = 60b191ac-1870-4425-abaf-86a68b835213 response_code = 200 uri = /crucible/0/regions
242024-05-01T00:20:39.032ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-3a62cd81-7d51-4dc3-ac4c-db95fda44c50
252024-05-01T00:20:39.041ZINFOcrucible-agent (worker): creating config property group
262024-05-01T00:20:39.045ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-3a62cd81-7d51-4dc3-ac4c-db95fda44c50
272024-05-01T00:20:39.045ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50
282024-05-01T00:20:39.046ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-05-01T00:20:39.046ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
302024-05-01T00:20:39.046ZINFOcrucible-agent (worker): commit
312024-05-01T00:20:39.049ZINFOcrucible-agent (worker): ok!
322024-05-01T00:20:39.054ZINFOcrucible-agent (worker): SMF ok!
332024-05-01T00:20:39.912ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:58058
342024-05-01T00:20:39.913ZINFOcrucible-agent (dropshot): request completed latency_us = 821 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:58058 req_id = 4d2591a0-023a-434e-bfc5-0519ba4c8acd response_code = 200 uri = /crucible/0/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50
352024-05-01T00:20:39.913ZINFOcrucible-agent (datafile): path is "/data/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50"
362024-05-01T00:20:39.923ZINFOcrucible-agent (datafile): dataset is oxp_6464a1b3-39ff-4f6d-bb8d-28b483455b0c/crucible/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50
372024-05-01T00:20:39.931ZINFOcrucible-agent (datafile): region 3a62cd81-7d51-4dc3-ac4c-db95fda44c50 state: Created -> Tombstoned
382024-05-01T00:20:39.931ZINFOcrucible-agent (worker): applying SMF actions before removal...
392024-05-01T00:20:39.931ZINFOcrucible-agent (dropshot): request completed latency_us = 17902 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:58058 req_id = 444fe8dd-d8b1-410b-94a8-4f9bab91491a response_code = 204 uri = /crucible/0/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50
402024-05-01T00:20:39.931ZINFOcrucible-agent (dropshot): request completed latency_us = 17 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:58058 req_id = eb1927d5-3fa8-4052-8625-a617906ab17b response_code = 200 uri = /crucible/0/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50
412024-05-01T00:20:39.932ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-3a62cd81-7d51-4dc3-ac4c-db95fda44c50 (instance states: (Some(Online), None))
422024-05-01T00:20:39.936ZINFOcrucible-agent (worker): SMF ok!
432024-05-01T00:20:39.945ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_6464a1b3-39ff-4f6d-bb8d-28b483455b0c/crucible/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50" region = 3a62cd81-7d51-4dc3-ac4c-db95fda44c50
442024-05-01T00:20:40.002ZINFOcrucible-agent (datafile): region 3a62cd81-7d51-4dc3-ac4c-db95fda44c50 state: Tombstoned -> Destroyed
452024-05-01T00:20:40.022ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:58058 req_id = efb8b19e-21af-4fd3-bb72-f94672e7d5c9 response_code = 200 uri = /crucible/0/regions/3a62cd81-7d51-4dc3-ac4c-db95fda44c50
462024-05-01T00:26:11.966ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:47487
472024-05-01T00:26:11.966ZINFOcrucible-agent (datafile): region 44ebb4d0-dd33-4181-a2df-085cad9e424b state: Requested
482024-05-01T00:26:11.967ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-05-01T00:26:11.967ZINFOcrucible-agent (dropshot): request completed latency_us = 1044 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:47487 req_id = 074959b3-2785-44d5-8514-ea8c27a9638e response_code = 200 uri = /crucible/0/regions
502024-05-01T00:26:11.979ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_6464a1b3-39ff-4f6d-bb8d-28b483455b0c/crucible/regions/44ebb4d0-dd33-4181-a2df-085cad9e424b
512024-05-01T00:26:11.979ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_6464a1b3-39ff-4f6d-bb8d-28b483455b0c/crucible/regions/44ebb4d0-dd33-4181-a2df-085cad9e424b
522024-05-01T00:26:12.030ZINFOcrucible-agent (worker): creating region Region { id: RegionId("44ebb4d0-dd33-4181-a2df-085cad9e424b"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/44ebb4d0-dd33-4181-a2df-085cad9e424b" region = 44ebb4d0-dd33-4181-a2df-085cad9e424b
532024-05-01T00:26:12.032ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "44ebb4d0-dd33-4181-a2df-085cad9e424b" "--data" "/data/regions/44ebb4d0-dd33-4181-a2df-085cad9e424b" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 44ebb4d0-dd33-4181-a2df-085cad9e424b
542024-05-01T00:26:12.206ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:47487 req_id = fccad6f1-f368-49ba-8cf5-1b4d285dc03a response_code = 200 uri = /crucible/0/regions
552024-05-01T00:26:12.278ZINFOcrucible-agent (worker): region files created ok region = 44ebb4d0-dd33-4181-a2df-085cad9e424b
562024-05-01T00:26:12.278ZINFOcrucible-agent (datafile): region 44ebb4d0-dd33-4181-a2df-085cad9e424b state: Requested -> Created
572024-05-01T00:26:12.278ZINFOcrucible-agent (worker): applying SMF actions post create...
582024-05-01T00:26:12.279ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-44ebb4d0-dd33-4181-a2df-085cad9e424b
592024-05-01T00:26:12.283ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-44ebb4d0-dd33-4181-a2df-085cad9e424b
602024-05-01T00:26:12.299ZINFOcrucible-agent (worker): creating config property group
612024-05-01T00:26:12.301ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-44ebb4d0-dd33-4181-a2df-085cad9e424b
622024-05-01T00:26:12.302ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/44ebb4d0-dd33-4181-a2df-085cad9e424b
632024-05-01T00:26:12.302ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
642024-05-01T00:26:12.302ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
652024-05-01T00:26:12.302ZINFOcrucible-agent (worker): commit
662024-05-01T00:26:12.305ZINFOcrucible-agent (worker): ok!
672024-05-01T00:26:12.310ZINFOcrucible-agent (worker): SMF ok!
682024-05-01T00:26:12.602ZINFOcrucible-agent (dropshot): request completed latency_us = 790 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:47487 req_id = 80c96c47-b23d-4e1c-b397-7901a0c82b86 response_code = 200 uri = /crucible/0/regions