Max level shown:
12024-09-03T21:30:36.972ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::b]:43533
22024-09-03T21:30:37.008ZINFOcrucible-agent (datafile): region 5bf0467d-392f-41ad-b8ad-0405ce0abe72 state: Requested
32024-09-03T21:30:37.012ZINFOcrucible-agent (dropshot): request completed latency_us = 22707 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43533 req_id = a7258980-0aa5-4aa3-b7c6-07ce59b823bf response_code = 200 uri = /crucible/0/regions
42024-09-03T21:30:37.017ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
52024-09-03T21:30:37.028ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_38359a1c-1b04-4272-8ebb-dbd0b95c2650/crucible/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72
62024-09-03T21:30:37.028ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_38359a1c-1b04-4272-8ebb-dbd0b95c2650/crucible/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72
72024-09-03T21:30:37.106ZINFOcrucible-agent (worker): creating region Region { id: RegionId("5bf0467d-392f-41ad-b8ad-0405ce0abe72"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72" region = 5bf0467d-392f-41ad-b8ad-0405ce0abe72
82024-09-03T21:30:37.110ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "5bf0467d-392f-41ad-b8ad-0405ce0abe72" "--data" "/data/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 5bf0467d-392f-41ad-b8ad-0405ce0abe72
92024-09-03T21:30:37.327ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43533 req_id = 38868a81-3ca1-4e60-9dfb-9c834bdbb527 response_code = 200 uri = /crucible/0/regions
102024-09-03T21:30:37.425ZINFOcrucible-agent (worker): region files created ok region = 5bf0467d-392f-41ad-b8ad-0405ce0abe72
112024-09-03T21:30:37.425ZINFOcrucible-agent (datafile): region 5bf0467d-392f-41ad-b8ad-0405ce0abe72 state: Requested -> Created
122024-09-03T21:30:37.425ZINFOcrucible-agent (worker): applying SMF actions post create...
132024-09-03T21:30:37.429ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-5bf0467d-392f-41ad-b8ad-0405ce0abe72
142024-09-03T21:30:37.434ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-5bf0467d-392f-41ad-b8ad-0405ce0abe72
152024-09-03T21:30:37.452ZINFOcrucible-agent (worker): creating config property group
162024-09-03T21:30:37.458ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-5bf0467d-392f-41ad-b8ad-0405ce0abe72
172024-09-03T21:30:37.470ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72
182024-09-03T21:30:37.472ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192024-09-03T21:30:37.472ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
202024-09-03T21:30:37.472ZINFOcrucible-agent (worker): commit
212024-09-03T21:30:37.480ZINFOcrucible-agent (worker): ok!
222024-09-03T21:30:37.486ZINFOcrucible-agent (worker): SMF ok!
232024-09-03T21:30:37.664ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43533 req_id = d3893b05-93f9-47d8-bcd0-06428bce4f29 response_code = 200 uri = /crucible/0/regions
242024-09-03T21:30:38.606ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43533 req_id = 363c35e2-b873-4a5f-969f-7e2894aeaa96 response_code = 200 uri = /crucible/0/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72
252024-09-03T21:30:38.608ZINFOcrucible-agent (datafile): path is "/data/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72"
262024-09-03T21:30:38.618ZINFOcrucible-agent (datafile): dataset is oxp_38359a1c-1b04-4272-8ebb-dbd0b95c2650/crucible/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72
272024-09-03T21:30:38.627ZINFOcrucible-agent (datafile): region 5bf0467d-392f-41ad-b8ad-0405ce0abe72 state: Created -> Tombstoned
282024-09-03T21:30:38.627ZINFOcrucible-agent (worker): applying SMF actions before removal...
292024-09-03T21:30:38.627ZINFOcrucible-agent (dropshot): request completed latency_us = 19013 local_addr = [fd00:1122:3344:101::12]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:43533 req_id = 58b99669-b7e0-427e-9cda-0628674d9fcf response_code = 204 uri = /crucible/0/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72
302024-09-03T21:30:38.628ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-5bf0467d-392f-41ad-b8ad-0405ce0abe72 (instance states: (Some(Online), None))
312024-09-03T21:30:38.631ZINFOcrucible-agent (dropshot): request completed latency_us = 41 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43533 req_id = e400b39b-4b21-4a7c-bf89-ad9b40644211 response_code = 200 uri = /crucible/0/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72
322024-09-03T21:30:38.635ZINFOcrucible-agent (worker): SMF ok!
332024-09-03T21:30:38.645ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_38359a1c-1b04-4272-8ebb-dbd0b95c2650/crucible/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72" region = 5bf0467d-392f-41ad-b8ad-0405ce0abe72
342024-09-03T21:30:38.705ZINFOcrucible-agent (datafile): region 5bf0467d-392f-41ad-b8ad-0405ce0abe72 state: Tombstoned -> Destroyed
352024-09-03T21:30:38.779ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43533 req_id = 2c23b7ce-92a4-4d7d-b8b8-2d6412aca941 response_code = 200 uri = /crucible/0/regions/5bf0467d-392f-41ad-b8ad-0405ce0abe72
362024-09-03T21:36:45.384ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:33560
372024-09-03T21:36:45.384ZINFOcrucible-agent (datafile): region dad39eae-139f-4559-8bf7-3e7e902ce5af state: Requested
382024-09-03T21:36:45.385ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
392024-09-03T21:36:45.385ZINFOcrucible-agent (dropshot): request completed latency_us = 615 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33560 req_id = c6cef5fa-a6ad-47e3-948e-d19abaebebb9 response_code = 200 uri = /crucible/0/regions
402024-09-03T21:36:45.395ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_38359a1c-1b04-4272-8ebb-dbd0b95c2650/crucible/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af
412024-09-03T21:36:45.395ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_38359a1c-1b04-4272-8ebb-dbd0b95c2650/crucible/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af
422024-09-03T21:36:45.453ZINFOcrucible-agent (worker): creating region Region { id: RegionId("dad39eae-139f-4559-8bf7-3e7e902ce5af"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af" region = dad39eae-139f-4559-8bf7-3e7e902ce5af
432024-09-03T21:36:45.453ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "dad39eae-139f-4559-8bf7-3e7e902ce5af" "--data" "/data/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = dad39eae-139f-4559-8bf7-3e7e902ce5af
442024-09-03T21:36:45.538ZINFOcrucible-agent (worker): region files created ok region = dad39eae-139f-4559-8bf7-3e7e902ce5af
452024-09-03T21:36:45.538ZINFOcrucible-agent (datafile): region dad39eae-139f-4559-8bf7-3e7e902ce5af state: Requested -> Created
462024-09-03T21:36:45.538ZINFOcrucible-agent (worker): applying SMF actions post create...
472024-09-03T21:36:45.539ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-dad39eae-139f-4559-8bf7-3e7e902ce5af
482024-09-03T21:36:45.548ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-dad39eae-139f-4559-8bf7-3e7e902ce5af
492024-09-03T21:36:45.561ZINFOcrucible-agent (worker): creating config property group
502024-09-03T21:36:45.565ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-dad39eae-139f-4559-8bf7-3e7e902ce5af
512024-09-03T21:36:45.566ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af
522024-09-03T21:36:45.566ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
532024-09-03T21:36:45.566ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
542024-09-03T21:36:45.566ZINFOcrucible-agent (worker): commit
552024-09-03T21:36:45.569ZINFOcrucible-agent (worker): ok!
562024-09-03T21:36:45.573ZINFOcrucible-agent (worker): SMF ok!
572024-09-03T21:36:45.609ZINFOcrucible-agent (dropshot): request completed latency_us = 75 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33560 req_id = 818a6c46-1303-48dd-8132-81ef80e4a9fe response_code = 200 uri = /crucible/0/regions
582024-09-03T21:37:10.559ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33560 req_id = 2f0e5a68-97ff-4458-81aa-5cf871cb0beb response_code = 200 uri = /crucible/0/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af
592024-09-03T21:37:10.563ZINFOcrucible-agent (datafile): path is "/data/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af"
602024-09-03T21:37:10.574ZINFOcrucible-agent (datafile): dataset is oxp_38359a1c-1b04-4272-8ebb-dbd0b95c2650/crucible/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af
612024-09-03T21:37:10.581ZINFOcrucible-agent (datafile): region dad39eae-139f-4559-8bf7-3e7e902ce5af state: Created -> Tombstoned
622024-09-03T21:37:10.582ZINFOcrucible-agent (worker): applying SMF actions before removal...
632024-09-03T21:37:10.582ZINFOcrucible-agent (dropshot): request completed latency_us = 18994 local_addr = [fd00:1122:3344:101::12]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:33560 req_id = 186bdd15-c7e5-4ad3-b08d-8bcd3fea8a1d response_code = 204 uri = /crucible/0/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af
642024-09-03T21:37:10.583ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-dad39eae-139f-4559-8bf7-3e7e902ce5af (instance states: (Some(Online), None))
652024-09-03T21:37:10.585ZINFOcrucible-agent (dropshot): request completed latency_us = 34 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33560 req_id = 3ac87cf0-5fc0-41db-8f0c-9ce3534ed4c3 response_code = 200 uri = /crucible/0/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af
662024-09-03T21:37:10.587ZINFOcrucible-agent (worker): SMF ok!
672024-09-03T21:37:10.596ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_38359a1c-1b04-4272-8ebb-dbd0b95c2650/crucible/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af" region = dad39eae-139f-4559-8bf7-3e7e902ce5af
682024-09-03T21:37:10.652ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33560 req_id = 011328f0-0f96-4b78-8cbf-0767d6a8389e response_code = 200 uri = /crucible/0/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af
692024-09-03T21:37:10.672ZINFOcrucible-agent (datafile): region dad39eae-139f-4559-8bf7-3e7e902ce5af state: Tombstoned -> Destroyed
702024-09-03T21:37:10.795ZINFOcrucible-agent (dropshot): request completed latency_us = 28 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33560 req_id = f7dd1f8b-e644-4b88-a905-b7f62d09e877 response_code = 200 uri = /crucible/0/regions/dad39eae-139f-4559-8bf7-3e7e902ce5af