Max level shown:
1 [ Sep 5 00:42:07 Enabled. ]
2 [ Sep 5 00:42:07 Rereading configuration. ]
3 [ Sep 5 00:42:09 Rereading configuration. ]
4 [ Sep 5 00:42:12 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_0992886b-7bd4-4261-afe1-662fbe130da8/crucible -l [fd00:1122:3344:101::17]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-09-05T00:42:12.614ZINFOcrucible-agent: dataset: "oxp_0992886b-7bd4-4261-afe1-662fbe130da8/crucible"
72024-09-05T00:42:12.615ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::17]:32345
82024-09-05T00:42:12.616ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-09-05T00:42:12.634ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-09-05T00:42:12.772ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::17]:32345
112024-09-05T00:42:49.946ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:49033
122024-09-05T00:42:49.974ZINFOcrucible-agent (datafile): region 2ec8696c-9975-46ef-8aec-8212bb724f9f state: Requested
132024-09-05T00:42:49.982ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142024-09-05T00:42:49.986ZINFOcrucible-agent (dropshot): request completed latency_us = 19150 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:49033 req_id = a9b79cd5-dd58-4b0a-a12b-ac46f863957d response_code = 200 uri = /crucible/0/regions
152024-09-05T00:42:49.994ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_0992886b-7bd4-4261-afe1-662fbe130da8/crucible/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f
162024-09-05T00:42:49.994ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_0992886b-7bd4-4261-afe1-662fbe130da8/crucible/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f
172024-09-05T00:42:50.164ZINFOcrucible-agent (worker): creating region Region { id: RegionId("2ec8696c-9975-46ef-8aec-8212bb724f9f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f" region = 2ec8696c-9975-46ef-8aec-8212bb724f9f
182024-09-05T00:42:50.167ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "2ec8696c-9975-46ef-8aec-8212bb724f9f" "--data" "/data/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 2ec8696c-9975-46ef-8aec-8212bb724f9f
192024-09-05T00:42:50.365ZINFOcrucible-agent (dropshot): request completed latency_us = 86 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:49033 req_id = 8696839f-3843-4b4e-86d0-50ac0125356d response_code = 200 uri = /crucible/0/regions
202024-09-05T00:42:50.409ZINFOcrucible-agent (worker): region files created ok region = 2ec8696c-9975-46ef-8aec-8212bb724f9f
212024-09-05T00:42:50.409ZINFOcrucible-agent (datafile): region 2ec8696c-9975-46ef-8aec-8212bb724f9f state: Requested -> Created
222024-09-05T00:42:50.409ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-09-05T00:42:50.409ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-2ec8696c-9975-46ef-8aec-8212bb724f9f
242024-09-05T00:42:50.445ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-2ec8696c-9975-46ef-8aec-8212bb724f9f
252024-09-05T00:42:50.516ZINFOcrucible-agent (worker): creating config property group
262024-09-05T00:42:50.567ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-2ec8696c-9975-46ef-8aec-8212bb724f9f
272024-09-05T00:42:50.590ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f
282024-09-05T00:42:50.593ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-09-05T00:42:50.593ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
302024-09-05T00:42:50.593ZINFOcrucible-agent (worker): commit
312024-09-05T00:42:50.613ZINFOcrucible-agent (worker): ok!
322024-09-05T00:42:50.642ZINFOcrucible-agent (worker): SMF ok!
332024-09-05T00:42:50.924ZINFOcrucible-agent (dropshot): request completed latency_us = 79 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:49033 req_id = c198e123-ff9a-4491-844d-370e704791be response_code = 200 uri = /crucible/0/regions
342024-09-05T00:42:53.302ZINFOcrucible-agent (dropshot): request completed latency_us = 540 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:49033 req_id = 86ede738-bbb6-4c7e-81af-acb09890cd84 response_code = 200 uri = /crucible/0/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f
352024-09-05T00:42:53.305ZINFOcrucible-agent (datafile): path is "/data/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f"
362024-09-05T00:42:53.314ZINFOcrucible-agent (datafile): dataset is oxp_0992886b-7bd4-4261-afe1-662fbe130da8/crucible/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f
372024-09-05T00:42:53.324ZINFOcrucible-agent (datafile): region 2ec8696c-9975-46ef-8aec-8212bb724f9f state: Created -> Tombstoned
382024-09-05T00:42:53.324ZINFOcrucible-agent (worker): applying SMF actions before removal...
392024-09-05T00:42:53.324ZINFOcrucible-agent (dropshot): request completed latency_us = 19427 local_addr = [fd00:1122:3344:101::17]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:49033 req_id = 899b7e28-bd59-478c-be40-29b2c6bba910 response_code = 204 uri = /crucible/0/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f
402024-09-05T00:42:53.325ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-2ec8696c-9975-46ef-8aec-8212bb724f9f (instance states: (Some(Online), None))
412024-09-05T00:42:53.327ZINFOcrucible-agent (dropshot): request completed latency_us = 22 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:49033 req_id = f37c92d7-aa48-42da-a119-8235d3cf65a2 response_code = 200 uri = /crucible/0/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f
422024-09-05T00:42:53.358ZINFOcrucible-agent (worker): SMF ok!
432024-09-05T00:42:53.366ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_0992886b-7bd4-4261-afe1-662fbe130da8/crucible/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f" region = 2ec8696c-9975-46ef-8aec-8212bb724f9f
442024-09-05T00:42:53.455ZINFOcrucible-agent (dropshot): request completed latency_us = 67 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:49033 req_id = 74c8c681-a023-4ed3-b756-112b5971ddfe response_code = 200 uri = /crucible/0/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f
452024-09-05T00:42:53.459ZINFOcrucible-agent (datafile): region 2ec8696c-9975-46ef-8aec-8212bb724f9f state: Tombstoned -> Destroyed
462024-09-05T00:42:53.606ZINFOcrucible-agent (dropshot): request completed latency_us = 28 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:49033 req_id = 1dfd169d-efe1-4209-aac7-823de3be4a51 response_code = 200 uri = /crucible/0/regions/2ec8696c-9975-46ef-8aec-8212bb724f9f
472024-09-05T00:43:36.503ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:42555
482024-09-05T00:43:36.503ZINFOcrucible-agent (datafile): region a419490a-5d51-421e-b082-161e2afe8972 state: Requested
492024-09-05T00:43:36.503ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
502024-09-05T00:43:36.503ZINFOcrucible-agent (dropshot): request completed latency_us = 218 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42555 req_id = 00549f74-e81e-49e3-8778-70db3e7bf457 response_code = 200 uri = /crucible/0/regions
512024-09-05T00:43:36.513ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_0992886b-7bd4-4261-afe1-662fbe130da8/crucible/regions/a419490a-5d51-421e-b082-161e2afe8972
522024-09-05T00:43:36.513ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_0992886b-7bd4-4261-afe1-662fbe130da8/crucible/regions/a419490a-5d51-421e-b082-161e2afe8972
532024-09-05T00:43:36.569ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a419490a-5d51-421e-b082-161e2afe8972"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a419490a-5d51-421e-b082-161e2afe8972" region = a419490a-5d51-421e-b082-161e2afe8972
542024-09-05T00:43:36.571ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a419490a-5d51-421e-b082-161e2afe8972" "--data" "/data/regions/a419490a-5d51-421e-b082-161e2afe8972" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = a419490a-5d51-421e-b082-161e2afe8972
552024-09-05T00:43:36.678ZINFOcrucible-agent (worker): region files created ok region = a419490a-5d51-421e-b082-161e2afe8972
562024-09-05T00:43:36.678ZINFOcrucible-agent (datafile): region a419490a-5d51-421e-b082-161e2afe8972 state: Requested -> Created
572024-09-05T00:43:36.678ZINFOcrucible-agent (worker): applying SMF actions post create...
582024-09-05T00:43:36.679ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a419490a-5d51-421e-b082-161e2afe8972
592024-09-05T00:43:36.685ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a419490a-5d51-421e-b082-161e2afe8972
602024-09-05T00:43:36.699ZINFOcrucible-agent (worker): creating config property group
612024-09-05T00:43:36.706ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a419490a-5d51-421e-b082-161e2afe8972
622024-09-05T00:43:36.706ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a419490a-5d51-421e-b082-161e2afe8972
632024-09-05T00:43:36.706ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
642024-09-05T00:43:36.706ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
652024-09-05T00:43:36.707ZINFOcrucible-agent (worker): commit
662024-09-05T00:43:36.716ZINFOcrucible-agent (worker): ok!
672024-09-05T00:43:36.726ZINFOcrucible-agent (worker): SMF ok!
682024-09-05T00:43:36.819ZINFOcrucible-agent (dropshot): request completed latency_us = 84 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42555 req_id = 8061c817-37a5-44eb-a630-2efc94d06398 response_code = 200 uri = /crucible/0/regions