Max level shown:
1 [ Nov 27 20:46:55 Enabled. ]
2 [ Nov 27 20:46:55 Rereading configuration. ]
3 [ Nov 27 20:47:00 Rereading configuration. ]
4 [ Nov 27 20:47:06 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_bd1e9859-f391-4410-ac6a-94d84a761bf8/crucible -l [fd00:1122:3344:101::18]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-11-27T20:47:06.222ZINFOcrucible-agent: dataset: "oxp_bd1e9859-f391-4410-ac6a-94d84a761bf8/crucible"
72024-11-27T20:47:06.223ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::18]:32345
82024-11-27T20:47:06.223ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-11-27T20:47:06.241ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-11-27T20:47:06.499ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::18]:32345
112024-11-27T20:58:33.797ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:43391
122024-11-27T20:58:33.842ZINFOcrucible-agent (datafile): region 98db1605-a09a-401f-a405-f3fd96cc98dd state: Requested
132024-11-27T20:58:33.847ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
142024-11-27T20:58:33.852ZINFOcrucible-agent (dropshot): request completed latency_us = 16460 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43391 req_id = 03e1380c-b2fc-4285-874f-ab17bd69a197 response_code = 200 uri = /crucible/0/regions
152024-11-27T20:58:33.861ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_bd1e9859-f391-4410-ac6a-94d84a761bf8/crucible/regions/98db1605-a09a-401f-a405-f3fd96cc98dd
162024-11-27T20:58:33.861ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_bd1e9859-f391-4410-ac6a-94d84a761bf8/crucible/regions/98db1605-a09a-401f-a405-f3fd96cc98dd
172024-11-27T20:58:33.911ZINFOcrucible-agent (worker): creating region Region { id: RegionId("98db1605-a09a-401f-a405-f3fd96cc98dd"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/98db1605-a09a-401f-a405-f3fd96cc98dd" region = 98db1605-a09a-401f-a405-f3fd96cc98dd
182024-11-27T20:58:33.913ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "98db1605-a09a-401f-a405-f3fd96cc98dd" "--data" "/data/regions/98db1605-a09a-401f-a405-f3fd96cc98dd" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 98db1605-a09a-401f-a405-f3fd96cc98dd
192024-11-27T20:58:34.076ZINFOcrucible-agent (dropshot): request completed latency_us = 87 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43391 req_id = 00eeaab7-5c2d-4f84-8e5e-8060c6aad8bc response_code = 200 uri = /crucible/0/regions
202024-11-27T20:58:34.239ZINFOcrucible-agent (worker): region files created ok region = 98db1605-a09a-401f-a405-f3fd96cc98dd
212024-11-27T20:58:34.239ZINFOcrucible-agent (datafile): region 98db1605-a09a-401f-a405-f3fd96cc98dd state: Requested -> Created
222024-11-27T20:58:34.239ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-11-27T20:58:34.240ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-98db1605-a09a-401f-a405-f3fd96cc98dd
242024-11-27T20:58:34.252ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-98db1605-a09a-401f-a405-f3fd96cc98dd
252024-11-27T20:58:34.269ZINFOcrucible-agent (worker): creating config property group
262024-11-27T20:58:34.272ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-98db1605-a09a-401f-a405-f3fd96cc98dd
272024-11-27T20:58:34.273ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/98db1605-a09a-401f-a405-f3fd96cc98dd
282024-11-27T20:58:34.273ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-11-27T20:58:34.273ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
302024-11-27T20:58:34.273ZINFOcrucible-agent (worker): commit
312024-11-27T20:58:34.276ZINFOcrucible-agent (worker): ok!
322024-11-27T20:58:34.281ZINFOcrucible-agent (worker): SMF ok!
332024-11-27T20:58:34.344ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43391 req_id = ac80d853-b1af-4eb6-b086-f5f9797107a6 response_code = 200 uri = /crucible/0/regions
342024-11-27T20:58:39.336ZINFOcrucible-agent (datafile): region ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc state: Requested
352024-11-27T20:58:39.336ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
362024-11-27T20:58:39.337ZINFOcrucible-agent (dropshot): request completed latency_us = 225 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43391 req_id = 6bbe4da4-5cf1-42b1-9b5c-aa458e9beb7e response_code = 200 uri = /crucible/0/regions
372024-11-27T20:58:39.347ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_bd1e9859-f391-4410-ac6a-94d84a761bf8/crucible/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
382024-11-27T20:58:39.347ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_bd1e9859-f391-4410-ac6a-94d84a761bf8/crucible/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
392024-11-27T20:58:39.396ZINFOcrucible-agent (worker): creating region Region { id: RegionId("ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc" region = ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
402024-11-27T20:58:39.396ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc" "--data" "/data/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
412024-11-27T20:58:39.466ZINFOcrucible-agent (worker): region files created ok region = ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
422024-11-27T20:58:39.466ZINFOcrucible-agent (datafile): region ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc state: Requested -> Created
432024-11-27T20:58:39.466ZINFOcrucible-agent (worker): applying SMF actions post create...
442024-11-27T20:58:39.470ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
452024-11-27T20:58:39.473ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
462024-11-27T20:58:39.481ZINFOcrucible-agent (worker): creating config property group
472024-11-27T20:58:39.484ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
482024-11-27T20:58:39.484ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
492024-11-27T20:58:39.485ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
502024-11-27T20:58:39.485ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
512024-11-27T20:58:39.485ZINFOcrucible-agent (worker): commit
522024-11-27T20:58:39.488ZINFOcrucible-agent (worker): ok!
532024-11-27T20:58:39.493ZINFOcrucible-agent (worker): SMF ok!
542024-11-27T20:58:39.618ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43391 req_id = 7b6452e2-a96f-4b75-b068-16bb59dcec3a response_code = 200 uri = /crucible/0/regions
552024-11-27T20:59:04.378ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:43391 req_id = 8b1e6c06-7019-4f43-a9b4-4d7f2f9e7f88 response_code = 200 uri = /crucible/0/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
562024-11-27T20:59:04.380ZINFOcrucible-agent (datafile): path is "/data/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc"
572024-11-27T20:59:04.393ZINFOcrucible-agent (datafile): dataset is oxp_bd1e9859-f391-4410-ac6a-94d84a761bf8/crucible/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
582024-11-27T20:59:04.403ZINFOcrucible-agent (datafile): region ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc state: Created -> Tombstoned
592024-11-27T20:59:04.403ZINFOcrucible-agent (worker): applying SMF actions before removal...
602024-11-27T20:59:04.403ZINFOcrucible-agent (dropshot): request completed latency_us = 22999 local_addr = [fd00:1122:3344:101::18]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:43391 req_id = 7ab19f96-41ab-4f23-83af-6b7b65f1cb8d response_code = 204 uri = /crucible/0/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
612024-11-27T20:59:04.404ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc (instance states: (Some(Online), None))
622024-11-27T20:59:04.408ZINFOcrucible-agent (dropshot): request completed latency_us = 617 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:43391 req_id = 9567d1e3-5bd2-4b85-858c-ac08f6791a6e response_code = 200 uri = /crucible/0/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
632024-11-27T20:59:04.410ZINFOcrucible-agent (worker): SMF ok!
642024-11-27T20:59:04.422ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_bd1e9859-f391-4410-ac6a-94d84a761bf8/crucible/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc" region = ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc
652024-11-27T20:59:04.467ZINFOcrucible-agent (datafile): region ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc state: Tombstoned -> Destroyed
662024-11-27T20:59:04.521ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:43391 req_id = c92556c8-51e8-44cf-b79d-ae3b88922966 response_code = 200 uri = /crucible/0/regions/ddbe0c9d-8a0d-4e47-a429-7efbdf18e1cc