Max level shown:
1 [ Aug 1 21:33:06 Enabled. ]
2 [ Aug 1 21:33:06 Rereading configuration. ]
3 [ Aug 1 21:33:12 Rereading configuration. ]
4 [ Aug 1 21:33:22 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_ca92b575-71c6-42e2-b0e0-517f7851f430/crucible -l [fd00:1122:3344:101::12]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-08-01T21:33:23.007ZINFOcrucible-agent: dataset: "oxp_ca92b575-71c6-42e2-b0e0-517f7851f430/crucible"
72025-08-01T21:33:23.010ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::12]:32345
82025-08-01T21:33:23.010ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-08-01T21:33:23.028ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-08-01T21:33:23.162ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::12]:32345
112025-08-01T21:44:18.716ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::c]:62577
122025-08-01T21:44:18.786ZINFOcrucible-agent (datafile): region 1983f2d5-6666-469e-a8d5-50d6731f6a05 state: Requested
132025-08-01T21:44:18.794ZINFOcrucible-agent (dropshot): request completed latency_us = 53856 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:62577 req_id = 8a0d6978-8fa7-48ae-8428-9550cc6a3857 response_code = 200 uri = /crucible/0/regions
142025-08-01T21:44:18.796ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
152025-08-01T21:44:18.815ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_ca92b575-71c6-42e2-b0e0-517f7851f430/crucible/regions/1983f2d5-6666-469e-a8d5-50d6731f6a05
162025-08-01T21:44:18.815ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_ca92b575-71c6-42e2-b0e0-517f7851f430/crucible/regions/1983f2d5-6666-469e-a8d5-50d6731f6a05
172025-08-01T21:44:18.917ZINFOcrucible-agent (worker): creating region Region { id: RegionId("1983f2d5-6666-469e-a8d5-50d6731f6a05"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/1983f2d5-6666-469e-a8d5-50d6731f6a05" region = 1983f2d5-6666-469e-a8d5-50d6731f6a05
182025-08-01T21:44:18.917ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "1983f2d5-6666-469e-a8d5-50d6731f6a05" "--data" "/data/regions/1983f2d5-6666-469e-a8d5-50d6731f6a05" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 1983f2d5-6666-469e-a8d5-50d6731f6a05
192025-08-01T21:44:18.945ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:62577 req_id = ef7c79a3-0aaa-4db6-8c7e-2ea8387a7e2f response_code = 200 uri = /crucible/0/regions
202025-08-01T21:44:19.471ZINFOcrucible-agent (worker): region files created ok region = 1983f2d5-6666-469e-a8d5-50d6731f6a05
212025-08-01T21:44:19.471ZINFOcrucible-agent (datafile): region 1983f2d5-6666-469e-a8d5-50d6731f6a05 state: Requested -> Created
222025-08-01T21:44:19.471ZINFOcrucible-agent (worker): applying SMF actions post create...
232025-08-01T21:44:19.471ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-1983f2d5-6666-469e-a8d5-50d6731f6a05
242025-08-01T21:44:19.505ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-1983f2d5-6666-469e-a8d5-50d6731f6a05
252025-08-01T21:44:19.558ZINFOcrucible-agent (worker): creating config property group
262025-08-01T21:44:19.572ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-1983f2d5-6666-469e-a8d5-50d6731f6a05
272025-08-01T21:44:19.573ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/1983f2d5-6666-469e-a8d5-50d6731f6a05
282025-08-01T21:44:19.573ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292025-08-01T21:44:19.573ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
302025-08-01T21:44:19.573ZINFOcrucible-agent (worker): commit
312025-08-01T21:44:19.585ZINFOcrucible-agent (worker): ok!
322025-08-01T21:44:19.635ZINFOcrucible-agent (worker): SMF ok!
332025-08-01T21:44:19.658ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:62577 req_id = 7a22772f-00c8-421b-8470-78c5a287f2b0 response_code = 200 uri = /crucible/0/regions
342025-08-01T21:44:26.478ZINFOcrucible-agent (datafile): region abdad58c-201c-49c8-9472-59027011f023 state: Requested
352025-08-01T21:44:26.478ZINFOcrucible-agent (dropshot): request completed latency_us = 216 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:62577 req_id = 6d16157b-ad42-4d75-b868-966be540fde9 response_code = 200 uri = /crucible/0/regions
362025-08-01T21:44:26.478ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
372025-08-01T21:44:26.488ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_ca92b575-71c6-42e2-b0e0-517f7851f430/crucible/regions/abdad58c-201c-49c8-9472-59027011f023
382025-08-01T21:44:26.488ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_ca92b575-71c6-42e2-b0e0-517f7851f430/crucible/regions/abdad58c-201c-49c8-9472-59027011f023
392025-08-01T21:44:26.600ZINFOcrucible-agent (worker): creating region Region { id: RegionId("abdad58c-201c-49c8-9472-59027011f023"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/abdad58c-201c-49c8-9472-59027011f023" region = abdad58c-201c-49c8-9472-59027011f023
402025-08-01T21:44:26.600ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "abdad58c-201c-49c8-9472-59027011f023" "--data" "/data/regions/abdad58c-201c-49c8-9472-59027011f023" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = abdad58c-201c-49c8-9472-59027011f023
412025-08-01T21:44:26.777ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:62577 req_id = 5ff4331d-10d4-44b0-b7c7-e515829eebae response_code = 200 uri = /crucible/0/regions
422025-08-01T21:44:26.801ZINFOcrucible-agent (worker): region files created ok region = abdad58c-201c-49c8-9472-59027011f023
432025-08-01T21:44:26.801ZINFOcrucible-agent (datafile): region abdad58c-201c-49c8-9472-59027011f023 state: Requested -> Created
442025-08-01T21:44:26.801ZINFOcrucible-agent (worker): applying SMF actions post create...
452025-08-01T21:44:26.802ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-abdad58c-201c-49c8-9472-59027011f023
462025-08-01T21:44:26.815ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-abdad58c-201c-49c8-9472-59027011f023
472025-08-01T21:44:26.842ZINFOcrucible-agent (worker): creating config property group
482025-08-01T21:44:26.851ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-abdad58c-201c-49c8-9472-59027011f023
492025-08-01T21:44:26.851ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/abdad58c-201c-49c8-9472-59027011f023
502025-08-01T21:44:26.851ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
512025-08-01T21:44:26.851ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
522025-08-01T21:44:26.851ZINFOcrucible-agent (worker): commit
532025-08-01T21:44:26.875ZINFOcrucible-agent (worker): ok!
542025-08-01T21:44:26.925ZINFOcrucible-agent (worker): SMF ok!
552025-08-01T21:44:27.148ZINFOcrucible-agent (dropshot): request completed latency_us = 47 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:62577 req_id = 13e1da64-f979-4a75-9575-eacefcd266cd response_code = 200 uri = /crucible/0/regions
562025-08-01T21:44:58.761ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::c]:55228
572025-08-01T21:44:58.764ZINFOcrucible-agent (dropshot): request completed latency_us = 2691 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:55228 req_id = 7e4b8541-1c52-4492-8eec-524e13134771 response_code = 200 uri = /crucible/0/regions/abdad58c-201c-49c8-9472-59027011f023
582025-08-01T21:44:58.766ZINFOcrucible-agent (datafile): path is "/data/regions/abdad58c-201c-49c8-9472-59027011f023"
592025-08-01T21:44:58.780ZINFOcrucible-agent (datafile): dataset is oxp_ca92b575-71c6-42e2-b0e0-517f7851f430/crucible/regions/abdad58c-201c-49c8-9472-59027011f023
602025-08-01T21:44:58.790ZINFOcrucible-agent (datafile): region abdad58c-201c-49c8-9472-59027011f023 state: Created -> Tombstoned
612025-08-01T21:44:58.791ZINFOcrucible-agent (worker): applying SMF actions before removal...
622025-08-01T21:44:58.791ZINFOcrucible-agent (dropshot): request completed latency_us = 24905 local_addr = [fd00:1122:3344:101::12]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:55228 req_id = 49e05cea-c1d8-4dde-89d5-090a7d3ef542 response_code = 204 uri = /crucible/0/regions/abdad58c-201c-49c8-9472-59027011f023
632025-08-01T21:44:58.792ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-abdad58c-201c-49c8-9472-59027011f023 (instance states: (Some(Online), None))
642025-08-01T21:44:58.794ZINFOcrucible-agent (dropshot): request completed latency_us = 27 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:55228 req_id = c904f2ce-60f7-4928-b8f8-d0ae38af1378 response_code = 200 uri = /crucible/0/regions/abdad58c-201c-49c8-9472-59027011f023
652025-08-01T21:44:58.809ZINFOcrucible-agent (worker): SMF ok!
662025-08-01T21:44:58.822ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_ca92b575-71c6-42e2-b0e0-517f7851f430/crucible/regions/abdad58c-201c-49c8-9472-59027011f023" region = abdad58c-201c-49c8-9472-59027011f023
672025-08-01T21:44:58.893ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:55228 req_id = d91999b7-3418-4d1e-b524-bc50c0c1079b response_code = 200 uri = /crucible/0/regions/abdad58c-201c-49c8-9472-59027011f023
682025-08-01T21:44:58.900ZINFOcrucible-agent (datafile): region abdad58c-201c-49c8-9472-59027011f023 state: Tombstoned -> Destroyed
692025-08-01T21:44:59.050ZINFOcrucible-agent (dropshot): request completed latency_us = 26 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:55228 req_id = 6f034e7f-2860-41bb-9ee9-c66cb76586c6 response_code = 200 uri = /crucible/0/regions/abdad58c-201c-49c8-9472-59027011f023