Max level shown:
1 [ Feb 1 02:09:04 Enabled. ]
2 [ Feb 1 02:09:04 Rereading configuration. ]
3 [ Feb 1 02:09:09 Rereading configuration. ]
4 [ Feb 1 02:09:14 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_b531588a-3566-4399-8139-f6e67067de42/crucible -l [fd00:1122:3344:101::17]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-02-01T02:09:14.389ZINFOcrucible-agent: dataset: "oxp_b531588a-3566-4399-8139-f6e67067de42/crucible"
72025-02-01T02:09:14.392ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::17]:32345
82025-02-01T02:09:14.392ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-02-01T02:09:14.411ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-02-01T02:09:14.512ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::17]:32345
112025-02-01T02:09:29.371ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:64333
122025-02-01T02:09:29.380ZINFOcrucible-agent (datafile): region 8bd6e221-9231-4d12-8665-2859715b7feb state: Requested
132025-02-01T02:09:29.381ZINFOcrucible-agent (dropshot): request completed latency_us = 1721 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64333 req_id = 8d92e8be-6e66-4466-859a-f06e01b09f52 response_code = 200 uri = /crucible/0/regions
142025-02-01T02:09:29.381ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-02-01T02:09:29.391ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_b531588a-3566-4399-8139-f6e67067de42/crucible/regions/8bd6e221-9231-4d12-8665-2859715b7feb
162025-02-01T02:09:29.391ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_b531588a-3566-4399-8139-f6e67067de42/crucible/regions/8bd6e221-9231-4d12-8665-2859715b7feb
172025-02-01T02:09:29.470ZINFOcrucible-agent (worker): creating region Region { id: RegionId("8bd6e221-9231-4d12-8665-2859715b7feb"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/8bd6e221-9231-4d12-8665-2859715b7feb" region = 8bd6e221-9231-4d12-8665-2859715b7feb
182025-02-01T02:09:29.473ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "8bd6e221-9231-4d12-8665-2859715b7feb" "--data" "/data/regions/8bd6e221-9231-4d12-8665-2859715b7feb" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 8bd6e221-9231-4d12-8665-2859715b7feb
192025-02-01T02:09:29.653ZINFOcrucible-agent (worker): region files created ok region = 8bd6e221-9231-4d12-8665-2859715b7feb
202025-02-01T02:09:29.653ZINFOcrucible-agent (datafile): region 8bd6e221-9231-4d12-8665-2859715b7feb state: Requested -> Created
212025-02-01T02:09:29.653ZINFOcrucible-agent (worker): applying SMF actions post create...
222025-02-01T02:09:29.653ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-8bd6e221-9231-4d12-8665-2859715b7feb
232025-02-01T02:09:29.662ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-8bd6e221-9231-4d12-8665-2859715b7feb
242025-02-01T02:09:29.678ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64333 req_id = 7804c6d2-d4d2-446a-b766-99d182e21b35 response_code = 200 uri = /crucible/0/regions
252025-02-01T02:09:29.680ZINFOcrucible-agent (worker): creating config property group
262025-02-01T02:09:29.688ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8bd6e221-9231-4d12-8665-2859715b7feb
272025-02-01T02:09:29.689ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8bd6e221-9231-4d12-8665-2859715b7feb
282025-02-01T02:09:29.689ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292025-02-01T02:09:29.689ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
302025-02-01T02:09:29.689ZINFOcrucible-agent (worker): commit
312025-02-01T02:09:29.697ZINFOcrucible-agent (worker): ok!
322025-02-01T02:09:29.705ZINFOcrucible-agent (worker): SMF ok!
332025-02-01T02:09:30.458ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:64333 req_id = f0deca09-7e57-461a-bf2c-e07fdc440680 response_code = 200 uri = /crucible/0/regions/8bd6e221-9231-4d12-8665-2859715b7feb
342025-02-01T02:09:30.462ZINFOcrucible-agent (datafile): path is "/data/regions/8bd6e221-9231-4d12-8665-2859715b7feb"
352025-02-01T02:09:30.473ZINFOcrucible-agent (datafile): dataset is oxp_b531588a-3566-4399-8139-f6e67067de42/crucible/regions/8bd6e221-9231-4d12-8665-2859715b7feb
362025-02-01T02:09:30.485ZINFOcrucible-agent (datafile): region 8bd6e221-9231-4d12-8665-2859715b7feb state: Created -> Tombstoned
372025-02-01T02:09:30.485ZINFOcrucible-agent (worker): applying SMF actions before removal...
382025-02-01T02:09:30.486ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-8bd6e221-9231-4d12-8665-2859715b7feb (instance states: (Some(Online), None))
392025-02-01T02:09:30.487ZINFOcrucible-agent (dropshot): request completed latency_us = 25607 local_addr = [fd00:1122:3344:101::17]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:64333 req_id = 7836e91b-afe9-4642-ad1c-9bb1866978b9 response_code = 204 uri = /crucible/0/regions/8bd6e221-9231-4d12-8665-2859715b7feb
402025-02-01T02:09:30.490ZINFOcrucible-agent (dropshot): request completed latency_us = 33 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:64333 req_id = 39b8db37-aebf-4fbb-859b-c60fcd71bddc response_code = 200 uri = /crucible/0/regions/8bd6e221-9231-4d12-8665-2859715b7feb
412025-02-01T02:09:30.493ZINFOcrucible-agent (worker): SMF ok!
422025-02-01T02:09:30.507ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_b531588a-3566-4399-8139-f6e67067de42/crucible/regions/8bd6e221-9231-4d12-8665-2859715b7feb" region = 8bd6e221-9231-4d12-8665-2859715b7feb
432025-02-01T02:09:30.562ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:64333 req_id = 42fc2ace-41b2-44ff-b539-97e73f2a8563 response_code = 200 uri = /crucible/0/regions/8bd6e221-9231-4d12-8665-2859715b7feb
442025-02-01T02:09:30.598ZINFOcrucible-agent (datafile): region 8bd6e221-9231-4d12-8665-2859715b7feb state: Tombstoned -> Destroyed
452025-02-01T02:09:30.633ZINFOcrucible-agent (dropshot): request completed latency_us = 45 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:64333 req_id = cbcf2051-50d8-4a5d-81e8-50c0de0b3143 response_code = 200 uri = /crucible/0/regions/8bd6e221-9231-4d12-8665-2859715b7feb
462025-02-01T02:09:32.996ZINFOcrucible-agent (datafile): region 12a8af30-44cd-45b3-9c2f-e8de1e7eee0e state: Requested
472025-02-01T02:09:32.999ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
482025-02-01T02:09:32.999ZINFOcrucible-agent (dropshot): request completed latency_us = 2489 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64333 req_id = 339b6456-2d33-4b69-a379-80ffc14a44b4 response_code = 200 uri = /crucible/0/regions
492025-02-01T02:09:33.008ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b531588a-3566-4399-8139-f6e67067de42/crucible/regions/12a8af30-44cd-45b3-9c2f-e8de1e7eee0e
502025-02-01T02:09:33.008ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b531588a-3566-4399-8139-f6e67067de42/crucible/regions/12a8af30-44cd-45b3-9c2f-e8de1e7eee0e
512025-02-01T02:09:33.103ZINFOcrucible-agent (worker): creating region Region { id: RegionId("12a8af30-44cd-45b3-9c2f-e8de1e7eee0e"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/12a8af30-44cd-45b3-9c2f-e8de1e7eee0e" region = 12a8af30-44cd-45b3-9c2f-e8de1e7eee0e
522025-02-01T02:09:33.105ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "12a8af30-44cd-45b3-9c2f-e8de1e7eee0e" "--data" "/data/regions/12a8af30-44cd-45b3-9c2f-e8de1e7eee0e" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 12a8af30-44cd-45b3-9c2f-e8de1e7eee0e
532025-02-01T02:09:33.209ZINFOcrucible-agent (worker): region files created ok region = 12a8af30-44cd-45b3-9c2f-e8de1e7eee0e
542025-02-01T02:09:33.209ZINFOcrucible-agent (datafile): region 12a8af30-44cd-45b3-9c2f-e8de1e7eee0e state: Requested -> Created
552025-02-01T02:09:33.209ZINFOcrucible-agent (worker): applying SMF actions post create...
562025-02-01T02:09:33.210ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-12a8af30-44cd-45b3-9c2f-e8de1e7eee0e
572025-02-01T02:09:33.214ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-12a8af30-44cd-45b3-9c2f-e8de1e7eee0e
582025-02-01T02:09:33.220ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64333 req_id = 8805ca8e-f96f-4f54-ad55-32e99224dfc2 response_code = 200 uri = /crucible/0/regions
592025-02-01T02:09:33.226ZINFOcrucible-agent (worker): creating config property group
602025-02-01T02:09:33.234ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-12a8af30-44cd-45b3-9c2f-e8de1e7eee0e
612025-02-01T02:09:33.234ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/12a8af30-44cd-45b3-9c2f-e8de1e7eee0e
622025-02-01T02:09:33.234ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632025-02-01T02:09:33.234ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
642025-02-01T02:09:33.234ZINFOcrucible-agent (worker): commit
652025-02-01T02:09:33.245ZINFOcrucible-agent (worker): ok!
662025-02-01T02:09:33.250ZINFOcrucible-agent (worker): SMF ok!