Max level shown:
12026-09-03T03:54:42.529ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:33202
22026-09-03T03:54:42.529ZINFOcrucible-agent (datafile): region a312df24-e6b1-49bd-8843-b66f3d8078a8 state: Requested
32026-09-03T03:54:42.530ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-09-03T03:54:42.530ZINFOcrucible-agent (dropshot): request completed latency_us = 1210 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:33202 req_id = 3bd3ed4f-bb29-4575-85ab-461462153ed3 response_code = 200 uri = /crucible/0/regions
52026-09-03T03:54:42.540ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/a312df24-e6b1-49bd-8843-b66f3d8078a8
62026-09-03T03:54:42.540ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/a312df24-e6b1-49bd-8843-b66f3d8078a8
72026-09-03T03:54:42.586ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a312df24-e6b1-49bd-8843-b66f3d8078a8"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a312df24-e6b1-49bd-8843-b66f3d8078a8" region = a312df24-e6b1-49bd-8843-b66f3d8078a8
82026-09-03T03:54:42.586ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a312df24-e6b1-49bd-8843-b66f3d8078a8" "--data" "/data/regions/a312df24-e6b1-49bd-8843-b66f3d8078a8" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = a312df24-e6b1-49bd-8843-b66f3d8078a8
92026-09-03T03:54:42.635ZINFOcrucible-agent (worker): region files created ok region = a312df24-e6b1-49bd-8843-b66f3d8078a8
102026-09-03T03:54:42.635ZINFOcrucible-agent (datafile): region a312df24-e6b1-49bd-8843-b66f3d8078a8 state: Requested -> Created
112026-09-03T03:54:42.635ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-09-03T03:54:42.642ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a312df24-e6b1-49bd-8843-b66f3d8078a8
132026-09-03T03:54:42.644ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a312df24-e6b1-49bd-8843-b66f3d8078a8
142026-09-03T03:54:42.650ZINFOcrucible-agent (worker): creating config property group
152026-09-03T03:54:42.651ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a312df24-e6b1-49bd-8843-b66f3d8078a8
162026-09-03T03:54:42.651ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a312df24-e6b1-49bd-8843-b66f3d8078a8
172026-09-03T03:54:42.651ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-09-03T03:54:42.651ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192026-09-03T03:54:42.651ZINFOcrucible-agent (worker): commit
202026-09-03T03:54:42.653ZINFOcrucible-agent (worker): ok!
212026-09-03T03:54:42.656ZINFOcrucible-agent (worker): SMF ok!
222026-09-03T03:54:42.881ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:33202 req_id = 19d5c6f7-206e-45b9-87bb-33be117692f2 response_code = 200 uri = /crucible/0/regions
232026-09-03T03:54:43.739ZINFOcrucible-agent (dropshot): request completed latency_us = 75 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:33202 req_id = 7ae18a2c-9c19-411f-a44f-beffd88b63b3 response_code = 200 uri = /crucible/0/regions/71a877f9-bdc4-4c62-8282-1e14a2730303
242026-09-03T03:54:43.743ZINFOcrucible-agent (datafile): path is "/data/regions/71a877f9-bdc4-4c62-8282-1e14a2730303"
252026-09-03T03:54:43.749ZINFOcrucible-agent (datafile): dataset is oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/71a877f9-bdc4-4c62-8282-1e14a2730303
262026-09-03T03:54:43.756ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/71a877f9-bdc4-4c62-8282-1e14a2730303@7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff
272026-09-03T03:54:43.756ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/71a877f9-bdc4-4c62-8282-1e14a2730303", "7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff"]
282026-09-03T03:54:43.767ZINFOcrucible-agent (dropshot): request completed latency_us = 24323 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:33202 req_id = 02b312cf-6f40-4558-a1e0-a5b423d17c83 response_code = 200 uri = /crucible/0/regions/71a877f9-bdc4-4c62-8282-1e14a2730303/snapshots/7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff
292026-09-03T03:54:43.777ZINFOcrucible-agent (datafile): path is "/data/regions/71a877f9-bdc4-4c62-8282-1e14a2730303"
302026-09-03T03:54:43.784ZINFOcrucible-agent (datafile): dataset is oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/71a877f9-bdc4-4c62-8282-1e14a2730303
312026-09-03T03:54:43.790ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/71a877f9-bdc4-4c62-8282-1e14a2730303@7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff
322026-09-03T03:54:43.790ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/71a877f9-bdc4-4c62-8282-1e14a2730303", "7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff"]
332026-09-03T03:54:43.795ZINFOcrucible-agent (datafile): requesting running snapshot 71a877f9-bdc4-4c62-8282-1e14a2730303-7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff state: Requested
342026-09-03T03:54:43.800ZINFOcrucible-agent (worker): applying SMF actions for region 71a877f9-bdc4-4c62-8282-1e14a2730303 running snapshot 7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff (state Requested)...
352026-09-03T03:54:43.800ZINFOcrucible-agent (dropshot): request completed latency_us = 23134 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:33202 req_id = 561c19f3-4e79-40cb-842d-a262ed1efaff response_code = 200 uri = /crucible/0/regions/71a877f9-bdc4-4c62-8282-1e14a2730303/snapshots/7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff/run
362026-09-03T03:54:43.802ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-71a877f9-bdc4-4c62-8282-1e14a2730303-7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff
372026-09-03T03:54:43.805ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-71a877f9-bdc4-4c62-8282-1e14a2730303-7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff
382026-09-03T03:54:43.810ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-71a877f9-bdc4-4c62-8282-1e14a2730303-7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff
392026-09-03T03:54:43.810ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/71a877f9-bdc4-4c62-8282-1e14a2730303/.zfs/snapshot/7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff
402026-09-03T03:54:43.810ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-09-03T03:54:43.810ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-09-03T03:54:43.810ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432026-09-03T03:54:43.810ZINFOcrucible-agent (worker): commit
442026-09-03T03:54:43.812ZINFOcrucible-agent (worker): ok!
452026-09-03T03:54:43.815ZINFOcrucible-agent (worker): SMF ok!
462026-09-03T03:54:43.815ZINFOcrucible-agent (datafile): region 71a877f9-bdc4-4c62-8282-1e14a2730303 running snapshot 7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff state: Requested -> Created
472026-09-03T03:54:44.012ZINFOcrucible-agent (datafile): path is "/data/regions/71a877f9-bdc4-4c62-8282-1e14a2730303"
482026-09-03T03:54:44.019ZINFOcrucible-agent (datafile): dataset is oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/71a877f9-bdc4-4c62-8282-1e14a2730303
492026-09-03T03:54:44.025ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/71a877f9-bdc4-4c62-8282-1e14a2730303@7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff
502026-09-03T03:54:44.025ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ea71a7e-a4a4-49c4-b170-71be74919469/crucible/regions/71a877f9-bdc4-4c62-8282-1e14a2730303", "7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff"]
512026-09-03T03:54:44.032ZINFOcrucible-agent (dropshot): request completed latency_us = 19295 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:33202 req_id = a0ac2f47-6aee-4ac5-bb58-95b3b2a7bb75 response_code = 200 uri = /crucible/0/regions/71a877f9-bdc4-4c62-8282-1e14a2730303/snapshots/7b03a4a3-5f1d-4aec-b325-f7fb099ce3ff/run