Max level shown:
1 [ Aug 27 19:02:02 Enabled. ]
2 [ Aug 27 19:02:02 Rereading configuration. ]
3 [ Aug 27 19:02:09 Rereading configuration. ]
4 [ Aug 27 19:02:17 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_ece87f46-247e-409b-ad26-6e8e96c7e756/crucible -l [fd00:1122:3344:101::16]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-08-27T19:02:17.974ZINFOcrucible-agent: dataset: "oxp_ece87f46-247e-409b-ad26-6e8e96c7e756/crucible"
72024-08-27T19:02:17.975ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::16]:32345
82024-08-27T19:02:17.975ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-08-27T19:02:17.990ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-08-27T19:02:18.191ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::16]:32345
112024-08-27T19:03:03.700ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:62365
122024-08-27T19:03:03.718ZINFOcrucible-agent (datafile): region 357bc787-dfaa-4337-a8a3-7a35d6a66e3e state: Requested
132024-08-27T19:03:03.723ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142024-08-27T19:03:03.726ZINFOcrucible-agent (dropshot): request completed latency_us = 14453 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62365 req_id = fec7ff54-1783-48c5-954a-2fa0da96e562 response_code = 200 uri = /crucible/0/regions
152024-08-27T19:03:03.733ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_ece87f46-247e-409b-ad26-6e8e96c7e756/crucible/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e
162024-08-27T19:03:03.733ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_ece87f46-247e-409b-ad26-6e8e96c7e756/crucible/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e
172024-08-27T19:03:03.786ZINFOcrucible-agent (worker): creating region Region { id: RegionId("357bc787-dfaa-4337-a8a3-7a35d6a66e3e"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e" region = 357bc787-dfaa-4337-a8a3-7a35d6a66e3e
182024-08-27T19:03:03.786ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "357bc787-dfaa-4337-a8a3-7a35d6a66e3e" "--data" "/data/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 357bc787-dfaa-4337-a8a3-7a35d6a66e3e
192024-08-27T19:03:03.972ZINFOcrucible-agent (worker): region files created ok region = 357bc787-dfaa-4337-a8a3-7a35d6a66e3e
202024-08-27T19:03:03.972ZINFOcrucible-agent (datafile): region 357bc787-dfaa-4337-a8a3-7a35d6a66e3e state: Requested -> Created
212024-08-27T19:03:03.972ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-08-27T19:03:03.973ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-357bc787-dfaa-4337-a8a3-7a35d6a66e3e
232024-08-27T19:03:03.977ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-357bc787-dfaa-4337-a8a3-7a35d6a66e3e
242024-08-27T19:03:03.993ZINFOcrucible-agent (worker): creating config property group
252024-08-27T19:03:03.996ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-357bc787-dfaa-4337-a8a3-7a35d6a66e3e
262024-08-27T19:03:04.002ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e
272024-08-27T19:03:04.004ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
282024-08-27T19:03:04.004ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
292024-08-27T19:03:04.004ZINFOcrucible-agent (worker): commit
302024-08-27T19:03:04.008ZINFOcrucible-agent (worker): ok!
312024-08-27T19:03:04.013ZINFOcrucible-agent (worker): SMF ok!
322024-08-27T19:03:04.017ZINFOcrucible-agent (dropshot): request completed latency_us = 85 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62365 req_id = 9aabd449-a7d4-4a47-a621-dda4e31a2cd0 response_code = 200 uri = /crucible/0/regions
332024-08-27T19:03:04.506ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62365 req_id = b162b1ef-e6b0-483b-88c1-6f8134bdcd16 response_code = 200 uri = /crucible/0/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e
342024-08-27T19:03:04.508ZINFOcrucible-agent (datafile): path is "/data/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e"
352024-08-27T19:03:04.518ZINFOcrucible-agent (datafile): dataset is oxp_ece87f46-247e-409b-ad26-6e8e96c7e756/crucible/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e
362024-08-27T19:03:04.527ZINFOcrucible-agent (datafile): region 357bc787-dfaa-4337-a8a3-7a35d6a66e3e state: Created -> Tombstoned
372024-08-27T19:03:04.527ZINFOcrucible-agent (worker): applying SMF actions before removal...
382024-08-27T19:03:04.527ZINFOcrucible-agent (dropshot): request completed latency_us = 19619 local_addr = [fd00:1122:3344:101::16]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:62365 req_id = 5c29ab91-f14b-4eca-b335-ef2030864225 response_code = 204 uri = /crucible/0/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e
392024-08-27T19:03:04.528ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-357bc787-dfaa-4337-a8a3-7a35d6a66e3e (instance states: (Some(Online), None))
402024-08-27T19:03:04.529ZINFOcrucible-agent (dropshot): request completed latency_us = 25 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62365 req_id = dfdd5ac7-bd1e-434a-85ff-025404c1475a response_code = 200 uri = /crucible/0/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e
412024-08-27T19:03:04.532ZINFOcrucible-agent (worker): SMF ok!
422024-08-27T19:03:04.543ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_ece87f46-247e-409b-ad26-6e8e96c7e756/crucible/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e" region = 357bc787-dfaa-4337-a8a3-7a35d6a66e3e
432024-08-27T19:03:04.597ZINFOcrucible-agent (datafile): region 357bc787-dfaa-4337-a8a3-7a35d6a66e3e state: Tombstoned -> Destroyed
442024-08-27T19:03:04.641ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62365 req_id = 7e67792c-4d7b-4338-bbf4-7893ac6b7f3f response_code = 200 uri = /crucible/0/regions/357bc787-dfaa-4337-a8a3-7a35d6a66e3e
452024-08-27T19:08:40.612ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:40722
462024-08-27T19:08:40.612ZINFOcrucible-agent (datafile): region 13355130-ad00-4f06-b604-7c966f985712 state: Requested
472024-08-27T19:08:40.618ZINFOcrucible-agent (dropshot): request completed latency_us = 6262 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:40722 req_id = 6acc2bdd-539f-4903-a30c-9718c7e7f35a response_code = 200 uri = /crucible/0/regions
482024-08-27T19:08:40.618ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-08-27T19:08:40.629ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_ece87f46-247e-409b-ad26-6e8e96c7e756/crucible/regions/13355130-ad00-4f06-b604-7c966f985712
502024-08-27T19:08:40.629ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_ece87f46-247e-409b-ad26-6e8e96c7e756/crucible/regions/13355130-ad00-4f06-b604-7c966f985712
512024-08-27T19:08:40.745ZINFOcrucible-agent (worker): creating region Region { id: RegionId("13355130-ad00-4f06-b604-7c966f985712"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/13355130-ad00-4f06-b604-7c966f985712" region = 13355130-ad00-4f06-b604-7c966f985712
522024-08-27T19:08:40.745ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "13355130-ad00-4f06-b604-7c966f985712" "--data" "/data/regions/13355130-ad00-4f06-b604-7c966f985712" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 13355130-ad00-4f06-b604-7c966f985712
532024-08-27T19:08:40.965ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:40722 req_id = 3f1a288b-23a2-4743-a3d9-2ee84bc1883f response_code = 200 uri = /crucible/0/regions
542024-08-27T19:08:41.103ZINFOcrucible-agent (worker): region files created ok region = 13355130-ad00-4f06-b604-7c966f985712
552024-08-27T19:08:41.103ZINFOcrucible-agent (datafile): region 13355130-ad00-4f06-b604-7c966f985712 state: Requested -> Created
562024-08-27T19:08:41.103ZINFOcrucible-agent (worker): applying SMF actions post create...
572024-08-27T19:08:41.104ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-13355130-ad00-4f06-b604-7c966f985712
582024-08-27T19:08:41.114ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-13355130-ad00-4f06-b604-7c966f985712
592024-08-27T19:08:41.180ZINFOcrucible-agent (worker): creating config property group
602024-08-27T19:08:41.254ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-13355130-ad00-4f06-b604-7c966f985712
612024-08-27T19:08:41.254ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/13355130-ad00-4f06-b604-7c966f985712
622024-08-27T19:08:41.254ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632024-08-27T19:08:41.254ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
642024-08-27T19:08:41.254ZINFOcrucible-agent (worker): commit
652024-08-27T19:08:41.280ZINFOcrucible-agent (worker): ok!
662024-08-27T19:08:41.290ZINFOcrucible-agent (worker): SMF ok!
672024-08-27T19:08:41.465ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:40722 req_id = 281f3b04-06c9-4b59-ad43-922ce007e981 response_code = 200 uri = /crucible/0/regions