Max level shown:
1 [ Aug 9 21:13:30 Enabled. ]
2 [ Aug 9 21:13:30 Rereading configuration. ]
3 [ Aug 9 21:13:32 Rereading configuration. ]
4 [ Aug 9 21:13:34 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible -l [fd00:1122:3344:101::19]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-08-09T21:13:34.199ZINFOcrucible-agent: dataset: "oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible"
72024-08-09T21:13:34.200ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::19]:32345
82024-08-09T21:13:34.200ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-08-09T21:13:34.221ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-08-09T21:13:34.326ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::19]:32345
112024-08-09T21:13:53.072ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:38094
122024-08-09T21:13:53.076ZINFOcrucible-agent (datafile): region eaaa057b-ac17-42b7-afd4-622ff0310a57 state: Requested
132024-08-09T21:13:53.077ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142024-08-09T21:13:53.077ZINFOcrucible-agent (dropshot): request completed latency_us = 3734 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:38094 req_id = 12ea2f6d-87ba-44e9-83d1-ff0c525c90a9 response_code = 200 uri = /crucible/0/regions
152024-08-09T21:13:53.087ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57
162024-08-09T21:13:53.087ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57
172024-08-09T21:13:53.143ZINFOcrucible-agent (worker): creating region Region { id: RegionId("eaaa057b-ac17-42b7-afd4-622ff0310a57"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57" region = eaaa057b-ac17-42b7-afd4-622ff0310a57
182024-08-09T21:13:53.143ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "eaaa057b-ac17-42b7-afd4-622ff0310a57" "--data" "/data/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = eaaa057b-ac17-42b7-afd4-622ff0310a57
192024-08-09T21:13:53.246ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:38094 req_id = 5b33916b-208d-4771-ada0-e3cf2ed71cb4 response_code = 200 uri = /crucible/0/regions
202024-08-09T21:13:53.258ZINFOcrucible-agent (worker): region files created ok region = eaaa057b-ac17-42b7-afd4-622ff0310a57
212024-08-09T21:13:53.258ZINFOcrucible-agent (datafile): region eaaa057b-ac17-42b7-afd4-622ff0310a57 state: Requested -> Created
222024-08-09T21:13:53.258ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-08-09T21:13:53.258ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-eaaa057b-ac17-42b7-afd4-622ff0310a57
242024-08-09T21:13:53.262ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-eaaa057b-ac17-42b7-afd4-622ff0310a57
252024-08-09T21:13:53.271ZINFOcrucible-agent (worker): creating config property group
262024-08-09T21:13:53.275ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-eaaa057b-ac17-42b7-afd4-622ff0310a57
272024-08-09T21:13:53.276ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57
282024-08-09T21:13:53.276ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-08-09T21:13:53.276ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
302024-08-09T21:13:53.276ZINFOcrucible-agent (worker): commit
312024-08-09T21:13:53.281ZINFOcrucible-agent (worker): ok!
322024-08-09T21:13:53.287ZINFOcrucible-agent (worker): SMF ok!
332024-08-09T21:13:53.769ZINFOcrucible-agent (dropshot): request completed latency_us = 88 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:38094 req_id = 39c20ed2-0929-4f73-a39f-d55ace206d55 response_code = 200 uri = /crucible/0/regions
342024-08-09T21:13:54.414ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:38094 req_id = 99a56abb-d0ca-4250-9bb9-c1c39493b9cc response_code = 200 uri = /crucible/0/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57
352024-08-09T21:13:54.417ZINFOcrucible-agent (datafile): path is "/data/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57"
362024-08-09T21:13:54.427ZINFOcrucible-agent (datafile): dataset is oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57
372024-08-09T21:13:54.436ZINFOcrucible-agent (datafile): region eaaa057b-ac17-42b7-afd4-622ff0310a57 state: Created -> Tombstoned
382024-08-09T21:13:54.436ZINFOcrucible-agent (worker): applying SMF actions before removal...
392024-08-09T21:13:54.436ZINFOcrucible-agent (dropshot): request completed latency_us = 18942 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:38094 req_id = 5aad860f-12ac-4de6-8b35-d3e803351912 response_code = 204 uri = /crucible/0/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57
402024-08-09T21:13:54.437ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-eaaa057b-ac17-42b7-afd4-622ff0310a57 (instance states: (Some(Online), None))
412024-08-09T21:13:54.439ZINFOcrucible-agent (dropshot): request completed latency_us = 26 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:38094 req_id = 2e6b8352-bb75-444c-b651-0667683ddde6 response_code = 200 uri = /crucible/0/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57
422024-08-09T21:13:54.441ZINFOcrucible-agent (worker): SMF ok!
432024-08-09T21:13:54.452ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57" region = eaaa057b-ac17-42b7-afd4-622ff0310a57
442024-08-09T21:13:54.494ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:38094 req_id = 68344ff9-6a9e-46c5-987f-7d7bfe7300a0 response_code = 200 uri = /crucible/0/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57
452024-08-09T21:13:54.507ZINFOcrucible-agent (datafile): region eaaa057b-ac17-42b7-afd4-622ff0310a57 state: Tombstoned -> Destroyed
462024-08-09T21:13:54.578ZINFOcrucible-agent (dropshot): request completed latency_us = 693 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:38094 req_id = 1de97969-3fc6-4e15-ac75-0b4814ecadf7 response_code = 200 uri = /crucible/0/regions/eaaa057b-ac17-42b7-afd4-622ff0310a57
472024-08-09T21:13:55.861ZINFOcrucible-agent (datafile): region 4c8c9fcf-617c-4e28-b9ca-384c3eec448c state: Requested
482024-08-09T21:13:55.861ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-08-09T21:13:55.861ZINFOcrucible-agent (dropshot): request completed latency_us = 256 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:38094 req_id = e1db4f61-3578-4e56-b049-fee0ea02f70e response_code = 200 uri = /crucible/0/regions
502024-08-09T21:13:55.872ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c
512024-08-09T21:13:55.872ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_639c9258-e6ff-4184-8616-816ac5dd8f66/crucible/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c
522024-08-09T21:13:55.951ZINFOcrucible-agent (worker): creating region Region { id: RegionId("4c8c9fcf-617c-4e28-b9ca-384c3eec448c"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c" region = 4c8c9fcf-617c-4e28-b9ca-384c3eec448c
532024-08-09T21:13:55.953ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "4c8c9fcf-617c-4e28-b9ca-384c3eec448c" "--data" "/data/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 4c8c9fcf-617c-4e28-b9ca-384c3eec448c
542024-08-09T21:13:56.051ZINFOcrucible-agent (worker): region files created ok region = 4c8c9fcf-617c-4e28-b9ca-384c3eec448c
552024-08-09T21:13:56.051ZINFOcrucible-agent (datafile): region 4c8c9fcf-617c-4e28-b9ca-384c3eec448c state: Requested -> Created
562024-08-09T21:13:56.051ZINFOcrucible-agent (worker): applying SMF actions post create...
572024-08-09T21:13:56.052ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-4c8c9fcf-617c-4e28-b9ca-384c3eec448c
582024-08-09T21:13:56.056ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-4c8c9fcf-617c-4e28-b9ca-384c3eec448c
592024-08-09T21:13:56.073ZINFOcrucible-agent (worker): creating config property group
602024-08-09T21:13:56.082ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4c8c9fcf-617c-4e28-b9ca-384c3eec448c
612024-08-09T21:13:56.082ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4c8c9fcf-617c-4e28-b9ca-384c3eec448c
622024-08-09T21:13:56.082ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632024-08-09T21:13:56.082ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
642024-08-09T21:13:56.082ZINFOcrucible-agent (worker): commit
652024-08-09T21:13:56.088ZINFOcrucible-agent (worker): ok!
662024-08-09T21:13:56.093ZINFOcrucible-agent (worker): SMF ok!
672024-08-09T21:13:56.123ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:38094 req_id = 590aef09-7a8b-4451-88cb-518be6592cf9 response_code = 200 uri = /crucible/0/regions