Max level shown:
1 [ Oct 12 03:53:12 Enabled. ]
2 [ Oct 12 03:53:12 Rereading configuration. ]
3 [ Oct 12 03:53:14 Rereading configuration. ]
4 [ Oct 12 03:53:16 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_c0c860af-959d-47ac-8156-a6981a984705/crucible -l [fd00:1122:3344:101::15]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-10-12T03:53:16.292ZINFOcrucible-agent: dataset: "oxp_c0c860af-959d-47ac-8156-a6981a984705/crucible"
72024-10-12T03:53:16.300ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::15]:32345
82024-10-12T03:53:16.300ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-10-12T03:53:16.310ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-10-12T03:53:16.497ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::15]:32345
112024-10-12T03:53:39.584ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::c]:64237
122024-10-12T03:53:39.593ZINFOcrucible-agent (datafile): region 849d203c-c779-4efa-aae5-cdc5b891a2ef state: Requested
132024-10-12T03:53:39.594ZINFOcrucible-agent (dropshot): request completed latency_us = 4376 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:64237 req_id = ef6b9b22-ab66-446b-9335-76f02bca28bb response_code = 200 uri = /crucible/0/regions
142024-10-12T03:53:39.598ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-10-12T03:53:39.608ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_c0c860af-959d-47ac-8156-a6981a984705/crucible/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef
162024-10-12T03:53:39.608ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_c0c860af-959d-47ac-8156-a6981a984705/crucible/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef
172024-10-12T03:53:39.670ZINFOcrucible-agent (worker): creating region Region { id: RegionId("849d203c-c779-4efa-aae5-cdc5b891a2ef"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef" region = 849d203c-c779-4efa-aae5-cdc5b891a2ef
182024-10-12T03:53:39.670ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "849d203c-c779-4efa-aae5-cdc5b891a2ef" "--data" "/data/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 849d203c-c779-4efa-aae5-cdc5b891a2ef
192024-10-12T03:53:39.938ZINFOcrucible-agent (worker): region files created ok region = 849d203c-c779-4efa-aae5-cdc5b891a2ef
202024-10-12T03:53:39.938ZINFOcrucible-agent (datafile): region 849d203c-c779-4efa-aae5-cdc5b891a2ef state: Requested -> Created
212024-10-12T03:53:39.938ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-10-12T03:53:39.941ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-849d203c-c779-4efa-aae5-cdc5b891a2ef
232024-10-12T03:53:39.945ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-849d203c-c779-4efa-aae5-cdc5b891a2ef
242024-10-12T03:53:39.947ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:64237 req_id = df1b5498-5482-41b7-850b-f5c37e0ed37c response_code = 200 uri = /crucible/0/regions
252024-10-12T03:53:39.956ZINFOcrucible-agent (worker): creating config property group
262024-10-12T03:53:39.960ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-849d203c-c779-4efa-aae5-cdc5b891a2ef
272024-10-12T03:53:39.966ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef
282024-10-12T03:53:39.966ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-10-12T03:53:39.966ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
302024-10-12T03:53:39.966ZINFOcrucible-agent (worker): commit
312024-10-12T03:53:39.971ZINFOcrucible-agent (worker): ok!
322024-10-12T03:53:39.974ZINFOcrucible-agent (worker): SMF ok!
332024-10-12T03:53:40.308ZINFOcrucible-agent (dropshot): request completed latency_us = 81 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:64237 req_id = ce536689-a438-450f-83c0-ddcf046e8e3a response_code = 200 uri = /crucible/0/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef
342024-10-12T03:53:40.311ZINFOcrucible-agent (datafile): path is "/data/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef"
352024-10-12T03:53:40.323ZINFOcrucible-agent (datafile): dataset is oxp_c0c860af-959d-47ac-8156-a6981a984705/crucible/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef
362024-10-12T03:53:40.333ZINFOcrucible-agent (datafile): region 849d203c-c779-4efa-aae5-cdc5b891a2ef state: Created -> Tombstoned
372024-10-12T03:53:40.333ZINFOcrucible-agent (worker): applying SMF actions before removal...
382024-10-12T03:53:40.333ZINFOcrucible-agent (dropshot): request completed latency_us = 22001 local_addr = [fd00:1122:3344:101::15]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:64237 req_id = 9b291b9e-1db5-4aa6-810d-8778bf6a3403 response_code = 204 uri = /crucible/0/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef
392024-10-12T03:53:40.334ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-849d203c-c779-4efa-aae5-cdc5b891a2ef (instance states: (Some(Online), None))
402024-10-12T03:53:40.336ZINFOcrucible-agent (dropshot): request completed latency_us = 38 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:64237 req_id = 97ba16e9-b372-4d79-bc70-f933dc68f666 response_code = 200 uri = /crucible/0/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef
412024-10-12T03:53:40.339ZINFOcrucible-agent (worker): SMF ok!
422024-10-12T03:53:40.350ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_c0c860af-959d-47ac-8156-a6981a984705/crucible/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef" region = 849d203c-c779-4efa-aae5-cdc5b891a2ef
432024-10-12T03:53:40.424ZINFOcrucible-agent (datafile): region 849d203c-c779-4efa-aae5-cdc5b891a2ef state: Tombstoned -> Destroyed
442024-10-12T03:53:40.460ZINFOcrucible-agent (dropshot): request completed latency_us = 751 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:64237 req_id = d7b059c9-bb52-443f-a7d2-ca7e805bf804 response_code = 200 uri = /crucible/0/regions/849d203c-c779-4efa-aae5-cdc5b891a2ef
452024-10-12T03:57:55.931ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:47740
462024-10-12T03:57:55.932ZINFOcrucible-agent (datafile): region 4089f7f3-b9c1-4ecc-a893-9c4a02450d57 state: Requested
472024-10-12T03:57:55.932ZINFOcrucible-agent (dropshot): request completed latency_us = 609 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:47740 req_id = a499d090-a423-4ffa-8a6e-e06c58e0f91f response_code = 200 uri = /crucible/0/regions
482024-10-12T03:57:55.932ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-10-12T03:57:55.942ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_c0c860af-959d-47ac-8156-a6981a984705/crucible/regions/4089f7f3-b9c1-4ecc-a893-9c4a02450d57
502024-10-12T03:57:55.942ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_c0c860af-959d-47ac-8156-a6981a984705/crucible/regions/4089f7f3-b9c1-4ecc-a893-9c4a02450d57
512024-10-12T03:57:55.993ZINFOcrucible-agent (worker): creating region Region { id: RegionId("4089f7f3-b9c1-4ecc-a893-9c4a02450d57"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/4089f7f3-b9c1-4ecc-a893-9c4a02450d57" region = 4089f7f3-b9c1-4ecc-a893-9c4a02450d57
522024-10-12T03:57:55.995ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "4089f7f3-b9c1-4ecc-a893-9c4a02450d57" "--data" "/data/regions/4089f7f3-b9c1-4ecc-a893-9c4a02450d57" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 4089f7f3-b9c1-4ecc-a893-9c4a02450d57
532024-10-12T03:57:56.064ZINFOcrucible-agent (worker): region files created ok region = 4089f7f3-b9c1-4ecc-a893-9c4a02450d57
542024-10-12T03:57:56.064ZINFOcrucible-agent (datafile): region 4089f7f3-b9c1-4ecc-a893-9c4a02450d57 state: Requested -> Created
552024-10-12T03:57:56.064ZINFOcrucible-agent (worker): applying SMF actions post create...
562024-10-12T03:57:56.065ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-4089f7f3-b9c1-4ecc-a893-9c4a02450d57
572024-10-12T03:57:56.070ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-4089f7f3-b9c1-4ecc-a893-9c4a02450d57
582024-10-12T03:57:56.082ZINFOcrucible-agent (worker): creating config property group
592024-10-12T03:57:56.087ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4089f7f3-b9c1-4ecc-a893-9c4a02450d57
602024-10-12T03:57:56.088ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4089f7f3-b9c1-4ecc-a893-9c4a02450d57
612024-10-12T03:57:56.088ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
622024-10-12T03:57:56.088ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
632024-10-12T03:57:56.088ZINFOcrucible-agent (worker): commit
642024-10-12T03:57:56.091ZINFOcrucible-agent (worker): ok!
652024-10-12T03:57:56.096ZINFOcrucible-agent (worker): SMF ok!
662024-10-12T03:57:56.208ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:47740 req_id = 357f9fa8-6226-42eb-a76e-489bdc7bbc5e response_code = 200 uri = /crucible/0/regions