Max level shown:
1 [ Jan 16 19:48:46 Enabled. ]
2 [ Jan 16 19:48:46 Rereading configuration. ]
3 [ Jan 16 19:48:49 Rereading configuration. ]
4 [ Jan 16 19:48:54 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_b79d39ee-f7b6-49ee-84ea-90c91349c2c9/crucible -l [fd00:1122:3344:101::18]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-01-16T19:48:54.108ZINFOcrucible-agent: dataset: "oxp_b79d39ee-f7b6-49ee-84ea-90c91349c2c9/crucible"
72025-01-16T19:48:54.109ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::18]:32345
82025-01-16T19:48:54.109ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-01-16T19:48:54.126ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-01-16T19:48:54.290ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::18]:32345
112025-01-16T19:49:17.568ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::c]:61036
122025-01-16T19:49:17.579ZINFOcrucible-agent (datafile): region 9b4cbc45-4194-4e61-8ea8-662a399c917a state: Requested
132025-01-16T19:49:17.583ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142025-01-16T19:49:17.587ZINFOcrucible-agent (dropshot): request completed latency_us = 12455 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:61036 req_id = c091bbaa-491d-4f23-a42e-dac2e648f6c3 response_code = 200 uri = /crucible/0/regions
152025-01-16T19:49:17.594ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_b79d39ee-f7b6-49ee-84ea-90c91349c2c9/crucible/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a
162025-01-16T19:49:17.594ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_b79d39ee-f7b6-49ee-84ea-90c91349c2c9/crucible/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a
172025-01-16T19:49:17.669ZINFOcrucible-agent (worker): creating region Region { id: RegionId("9b4cbc45-4194-4e61-8ea8-662a399c917a"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a" region = 9b4cbc45-4194-4e61-8ea8-662a399c917a
182025-01-16T19:49:17.669ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "9b4cbc45-4194-4e61-8ea8-662a399c917a" "--data" "/data/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 9b4cbc45-4194-4e61-8ea8-662a399c917a
192025-01-16T19:49:17.833ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:61036 req_id = f7e00a35-a603-4899-afc1-24bec915989a response_code = 200 uri = /crucible/0/regions
202025-01-16T19:49:17.853ZINFOcrucible-agent (worker): region files created ok region = 9b4cbc45-4194-4e61-8ea8-662a399c917a
212025-01-16T19:49:17.853ZINFOcrucible-agent (datafile): region 9b4cbc45-4194-4e61-8ea8-662a399c917a state: Requested -> Created
222025-01-16T19:49:17.853ZINFOcrucible-agent (worker): applying SMF actions post create...
232025-01-16T19:49:17.853ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-9b4cbc45-4194-4e61-8ea8-662a399c917a
242025-01-16T19:49:17.858ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-9b4cbc45-4194-4e61-8ea8-662a399c917a
252025-01-16T19:49:17.878ZINFOcrucible-agent (worker): creating config property group
262025-01-16T19:49:17.894ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-9b4cbc45-4194-4e61-8ea8-662a399c917a
272025-01-16T19:49:17.894ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a
282025-01-16T19:49:17.896ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292025-01-16T19:49:17.897ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
302025-01-16T19:49:17.897ZINFOcrucible-agent (worker): commit
312025-01-16T19:49:17.904ZINFOcrucible-agent (worker): ok!
322025-01-16T19:49:17.911ZINFOcrucible-agent (worker): SMF ok!
332025-01-16T19:49:18.411ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:61036 req_id = f396fa9e-f9b1-4ded-9798-fc42e263e8aa response_code = 200 uri = /crucible/0/regions
342025-01-16T19:49:18.932ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:61036 req_id = 8852df7d-f2ac-4e6b-81fa-13c349545be4 response_code = 200 uri = /crucible/0/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a
352025-01-16T19:49:18.935ZINFOcrucible-agent (datafile): path is "/data/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a"
362025-01-16T19:49:18.947ZINFOcrucible-agent (datafile): dataset is oxp_b79d39ee-f7b6-49ee-84ea-90c91349c2c9/crucible/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a
372025-01-16T19:49:18.958ZINFOcrucible-agent (datafile): region 9b4cbc45-4194-4e61-8ea8-662a399c917a state: Created -> Tombstoned
382025-01-16T19:49:18.958ZINFOcrucible-agent (worker): applying SMF actions before removal...
392025-01-16T19:49:18.958ZINFOcrucible-agent (dropshot): request completed latency_us = 23038 local_addr = [fd00:1122:3344:101::18]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:61036 req_id = ede229fe-d23e-4489-9e6f-a2a1f4db7c4f response_code = 204 uri = /crucible/0/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a
402025-01-16T19:49:18.959ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-9b4cbc45-4194-4e61-8ea8-662a399c917a (instance states: (Some(Online), None))
412025-01-16T19:49:18.960ZINFOcrucible-agent (dropshot): request completed latency_us = 30 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:61036 req_id = bb50f56d-ad1a-4000-ad2c-9bfc49967663 response_code = 200 uri = /crucible/0/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a
422025-01-16T19:49:18.967ZINFOcrucible-agent (worker): SMF ok!
432025-01-16T19:49:18.980ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_b79d39ee-f7b6-49ee-84ea-90c91349c2c9/crucible/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a" region = 9b4cbc45-4194-4e61-8ea8-662a399c917a
442025-01-16T19:49:19.070ZINFOcrucible-agent (datafile): region 9b4cbc45-4194-4e61-8ea8-662a399c917a state: Tombstoned -> Destroyed
452025-01-16T19:49:19.101ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:61036 req_id = 7fc7f221-dbf5-4e87-93da-06090dd1f19b response_code = 200 uri = /crucible/0/regions/9b4cbc45-4194-4e61-8ea8-662a399c917a
462025-01-16T19:59:42.449ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::c]:38631
472025-01-16T19:59:42.449ZINFOcrucible-agent (datafile): region 21b590c8-1889-4ca2-a749-00d531e779d7 state: Requested
482025-01-16T19:59:42.449ZINFOcrucible-agent (dropshot): request completed latency_us = 703 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:38631 req_id = 2b649a2b-3ac9-44d3-aa99-b8e61c0001b8 response_code = 200 uri = /crucible/0/regions
492025-01-16T19:59:42.449ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
502025-01-16T19:59:42.466ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b79d39ee-f7b6-49ee-84ea-90c91349c2c9/crucible/regions/21b590c8-1889-4ca2-a749-00d531e779d7
512025-01-16T19:59:42.466ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b79d39ee-f7b6-49ee-84ea-90c91349c2c9/crucible/regions/21b590c8-1889-4ca2-a749-00d531e779d7
522025-01-16T19:59:42.519ZINFOcrucible-agent (worker): creating region Region { id: RegionId("21b590c8-1889-4ca2-a749-00d531e779d7"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/21b590c8-1889-4ca2-a749-00d531e779d7" region = 21b590c8-1889-4ca2-a749-00d531e779d7
532025-01-16T19:59:42.521ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "21b590c8-1889-4ca2-a749-00d531e779d7" "--data" "/data/regions/21b590c8-1889-4ca2-a749-00d531e779d7" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 21b590c8-1889-4ca2-a749-00d531e779d7
542025-01-16T19:59:42.602ZINFOcrucible-agent (dropshot): request completed latency_us = 88 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:38631 req_id = b91167c4-6ddc-4040-85c3-5a4676be11a8 response_code = 200 uri = /crucible/0/regions
552025-01-16T19:59:42.798ZINFOcrucible-agent (worker): region files created ok region = 21b590c8-1889-4ca2-a749-00d531e779d7
562025-01-16T19:59:42.799ZINFOcrucible-agent (datafile): region 21b590c8-1889-4ca2-a749-00d531e779d7 state: Requested -> Created
572025-01-16T19:59:42.799ZINFOcrucible-agent (worker): applying SMF actions post create...
582025-01-16T19:59:42.800ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-21b590c8-1889-4ca2-a749-00d531e779d7
592025-01-16T19:59:42.808ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-21b590c8-1889-4ca2-a749-00d531e779d7
602025-01-16T19:59:42.820ZINFOcrucible-agent (worker): creating config property group
612025-01-16T19:59:42.824ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-21b590c8-1889-4ca2-a749-00d531e779d7
622025-01-16T19:59:42.824ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/21b590c8-1889-4ca2-a749-00d531e779d7
632025-01-16T19:59:42.824ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
642025-01-16T19:59:42.824ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
652025-01-16T19:59:42.824ZINFOcrucible-agent (worker): commit
662025-01-16T19:59:42.828ZINFOcrucible-agent (worker): ok!
672025-01-16T19:59:42.833ZINFOcrucible-agent (worker): SMF ok!
682025-01-16T19:59:42.997ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:38631 req_id = 81a1d473-4502-492e-a32a-2552263e2c6f response_code = 200 uri = /crucible/0/regions