Max level shown:
1 [ Aug 27 19:48:17 Enabled. ]
2 [ Aug 27 19:48:17 Rereading configuration. ]
3 [ Aug 27 19:48:19 Rereading configuration. ]
4 [ Aug 27 19:48:20 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_d598a515-1659-4717-862f-e7cfd6119c10/crucible -l [fd00:1122:3344:101::12]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-08-27T19:48:20.947ZINFOcrucible-agent: dataset: "oxp_d598a515-1659-4717-862f-e7cfd6119c10/crucible"
72024-08-27T19:48:20.954ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::12]:32345
82024-08-27T19:48:20.954ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-08-27T19:48:20.980ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-08-27T19:48:21.143ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::12]:32345
112024-08-27T19:49:22.164ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::b]:48984
122024-08-27T19:49:22.182ZINFOcrucible-agent (datafile): region 9177dfe5-1045-40a3-887c-8aa2edbf5480 state: Requested
132024-08-27T19:49:22.183ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142024-08-27T19:49:22.187ZINFOcrucible-agent (dropshot): request completed latency_us = 11249 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:48984 req_id = 2119d324-9446-42fa-8d96-a260265a4aae response_code = 200 uri = /crucible/0/regions
152024-08-27T19:49:22.193ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_d598a515-1659-4717-862f-e7cfd6119c10/crucible/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480
162024-08-27T19:49:22.193ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_d598a515-1659-4717-862f-e7cfd6119c10/crucible/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480
172024-08-27T19:49:22.266ZINFOcrucible-agent (worker): creating region Region { id: RegionId("9177dfe5-1045-40a3-887c-8aa2edbf5480"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480" region = 9177dfe5-1045-40a3-887c-8aa2edbf5480
182024-08-27T19:49:22.268ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "9177dfe5-1045-40a3-887c-8aa2edbf5480" "--data" "/data/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 9177dfe5-1045-40a3-887c-8aa2edbf5480
192024-08-27T19:49:22.394ZINFOcrucible-agent (worker): region files created ok region = 9177dfe5-1045-40a3-887c-8aa2edbf5480
202024-08-27T19:49:22.394ZINFOcrucible-agent (datafile): region 9177dfe5-1045-40a3-887c-8aa2edbf5480 state: Requested -> Created
212024-08-27T19:49:22.394ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-08-27T19:49:22.395ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-9177dfe5-1045-40a3-887c-8aa2edbf5480
232024-08-27T19:49:22.401ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-9177dfe5-1045-40a3-887c-8aa2edbf5480
242024-08-27T19:49:22.415ZINFOcrucible-agent (worker): creating config property group
252024-08-27T19:49:22.418ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-9177dfe5-1045-40a3-887c-8aa2edbf5480
262024-08-27T19:49:22.419ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480
272024-08-27T19:49:22.419ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
282024-08-27T19:49:22.419ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
292024-08-27T19:49:22.419ZINFOcrucible-agent (worker): commit
302024-08-27T19:49:22.424ZINFOcrucible-agent (worker): ok!
312024-08-27T19:49:22.428ZINFOcrucible-agent (worker): SMF ok!
322024-08-27T19:49:22.491ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:48984 req_id = edf867ea-b103-4b13-a116-2d7ccb970e85 response_code = 200 uri = /crucible/0/regions
332024-08-27T19:49:22.787ZINFOcrucible-agent (dropshot): request completed latency_us = 94 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:48984 req_id = b8ee0283-1c70-413c-8504-c027cc82985e response_code = 200 uri = /crucible/0/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480
342024-08-27T19:49:22.791ZINFOcrucible-agent (datafile): path is "/data/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480"
352024-08-27T19:49:22.800ZINFOcrucible-agent (datafile): dataset is oxp_d598a515-1659-4717-862f-e7cfd6119c10/crucible/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480
362024-08-27T19:49:22.809ZINFOcrucible-agent (datafile): region 9177dfe5-1045-40a3-887c-8aa2edbf5480 state: Created -> Tombstoned
372024-08-27T19:49:22.809ZINFOcrucible-agent (worker): applying SMF actions before removal...
382024-08-27T19:49:22.809ZINFOcrucible-agent (dropshot): request completed latency_us = 18632 local_addr = [fd00:1122:3344:101::12]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:48984 req_id = 88439d3b-6c4d-473d-a860-dd528ba8e703 response_code = 204 uri = /crucible/0/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480
392024-08-27T19:49:22.810ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-9177dfe5-1045-40a3-887c-8aa2edbf5480 (instance states: (Some(Online), None))
402024-08-27T19:49:22.812ZINFOcrucible-agent (dropshot): request completed latency_us = 30 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:48984 req_id = f58cf83e-3962-445e-bdc3-9368307bcc5f response_code = 200 uri = /crucible/0/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480
412024-08-27T19:49:22.814ZINFOcrucible-agent (worker): SMF ok!
422024-08-27T19:49:22.824ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_d598a515-1659-4717-862f-e7cfd6119c10/crucible/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480" region = 9177dfe5-1045-40a3-887c-8aa2edbf5480
432024-08-27T19:49:22.889ZINFOcrucible-agent (datafile): region 9177dfe5-1045-40a3-887c-8aa2edbf5480 state: Tombstoned -> Destroyed
442024-08-27T19:49:22.948ZINFOcrucible-agent (dropshot): request completed latency_us = 89 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:48984 req_id = fdca78c7-7441-4e65-bac0-7c84e057fd92 response_code = 200 uri = /crucible/0/regions/9177dfe5-1045-40a3-887c-8aa2edbf5480
452024-08-27T19:55:03.448ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:60706
462024-08-27T19:55:03.448ZINFOcrucible-agent (datafile): region 84df2b4b-6cc7-4b0a-83b3-c1b0d638001e state: Requested
472024-08-27T19:55:03.449ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
482024-08-27T19:55:03.449ZINFOcrucible-agent (dropshot): request completed latency_us = 989 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60706 req_id = 39ce91ff-2131-4b8c-a4e7-21de8769c938 response_code = 200 uri = /crucible/0/regions
492024-08-27T19:55:03.460ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_d598a515-1659-4717-862f-e7cfd6119c10/crucible/regions/84df2b4b-6cc7-4b0a-83b3-c1b0d638001e
502024-08-27T19:55:03.460ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_d598a515-1659-4717-862f-e7cfd6119c10/crucible/regions/84df2b4b-6cc7-4b0a-83b3-c1b0d638001e
512024-08-27T19:55:03.535ZINFOcrucible-agent (worker): creating region Region { id: RegionId("84df2b4b-6cc7-4b0a-83b3-c1b0d638001e"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/84df2b4b-6cc7-4b0a-83b3-c1b0d638001e" region = 84df2b4b-6cc7-4b0a-83b3-c1b0d638001e
522024-08-27T19:55:03.535ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "84df2b4b-6cc7-4b0a-83b3-c1b0d638001e" "--data" "/data/regions/84df2b4b-6cc7-4b0a-83b3-c1b0d638001e" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 84df2b4b-6cc7-4b0a-83b3-c1b0d638001e
532024-08-27T19:55:03.781ZINFOcrucible-agent (dropshot): request completed latency_us = 96 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60706 req_id = 2e69f035-f615-4194-9756-910594177020 response_code = 200 uri = /crucible/0/regions
542024-08-27T19:55:03.832ZINFOcrucible-agent (worker): region files created ok region = 84df2b4b-6cc7-4b0a-83b3-c1b0d638001e
552024-08-27T19:55:03.832ZINFOcrucible-agent (datafile): region 84df2b4b-6cc7-4b0a-83b3-c1b0d638001e state: Requested -> Created
562024-08-27T19:55:03.832ZINFOcrucible-agent (worker): applying SMF actions post create...
572024-08-27T19:55:03.834ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-84df2b4b-6cc7-4b0a-83b3-c1b0d638001e
582024-08-27T19:55:03.842ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-84df2b4b-6cc7-4b0a-83b3-c1b0d638001e
592024-08-27T19:55:03.857ZINFOcrucible-agent (worker): creating config property group
602024-08-27T19:55:03.864ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-84df2b4b-6cc7-4b0a-83b3-c1b0d638001e
612024-08-27T19:55:03.864ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/84df2b4b-6cc7-4b0a-83b3-c1b0d638001e
622024-08-27T19:55:03.864ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632024-08-27T19:55:03.865ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
642024-08-27T19:55:03.865ZINFOcrucible-agent (worker): commit
652024-08-27T19:55:03.870ZINFOcrucible-agent (worker): ok!
662024-08-27T19:55:03.877ZINFOcrucible-agent (worker): SMF ok!
672024-08-27T19:55:04.533ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60706 req_id = fe686c30-c9d6-4603-a2b9-fd4fc0923cf5 response_code = 200 uri = /crucible/0/regions