Max level shown:
1 [ Jul 10 20:02:45 Enabled. ]
2 [ Jul 10 20:02:45 Rereading configuration. ]
3 [ Jul 10 20:02:47 Rereading configuration. ]
4 [ Jul 10 20:02:49 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_a1d5dfa8-5a21-417c-aa81-606313589625/crucible -l [fd00:1122:3344:101::15]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-07-10T20:02:49.258ZINFOcrucible-agent: dataset: "oxp_a1d5dfa8-5a21-417c-aa81-606313589625/crucible"
72024-07-10T20:02:49.264ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::15]:32345
82024-07-10T20:02:49.264ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-07-10T20:02:49.287ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-07-10T20:02:49.360ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::15]:32345
112024-07-10T20:04:01.388ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:61821
122024-07-10T20:04:01.409ZINFOcrucible-agent (datafile): region a21d4d03-976a-46fc-8f5e-51d23bf402af state: Requested
132024-07-10T20:04:01.418ZINFOcrucible-agent (dropshot): request completed latency_us = 17726 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61821 req_id = 5fb5f31c-860f-4807-971a-5e38a2cf90db response_code = 200 uri = /crucible/0/regions
142024-07-10T20:04:01.418ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
152024-07-10T20:04:01.430ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_a1d5dfa8-5a21-417c-aa81-606313589625/crucible/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af
162024-07-10T20:04:01.430ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_a1d5dfa8-5a21-417c-aa81-606313589625/crucible/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af
172024-07-10T20:04:01.494ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a21d4d03-976a-46fc-8f5e-51d23bf402af"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af" region = a21d4d03-976a-46fc-8f5e-51d23bf402af
182024-07-10T20:04:01.496ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a21d4d03-976a-46fc-8f5e-51d23bf402af" "--data" "/data/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = a21d4d03-976a-46fc-8f5e-51d23bf402af
192024-07-10T20:04:01.652ZINFOcrucible-agent (dropshot): request completed latency_us = 79 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61821 req_id = 93e0bf81-1d2c-4aae-9b2f-cfb3a3f9f09f response_code = 200 uri = /crucible/0/regions
202024-07-10T20:04:01.751ZINFOcrucible-agent (worker): region files created ok region = a21d4d03-976a-46fc-8f5e-51d23bf402af
212024-07-10T20:04:01.751ZINFOcrucible-agent (datafile): region a21d4d03-976a-46fc-8f5e-51d23bf402af state: Requested -> Created
222024-07-10T20:04:01.751ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-07-10T20:04:01.751ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a21d4d03-976a-46fc-8f5e-51d23bf402af
242024-07-10T20:04:01.761ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a21d4d03-976a-46fc-8f5e-51d23bf402af
252024-07-10T20:04:01.782ZINFOcrucible-agent (worker): creating config property group
262024-07-10T20:04:01.789ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a21d4d03-976a-46fc-8f5e-51d23bf402af
272024-07-10T20:04:01.789ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af
282024-07-10T20:04:01.793ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-07-10T20:04:01.794ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
302024-07-10T20:04:01.794ZINFOcrucible-agent (worker): commit
312024-07-10T20:04:01.806ZINFOcrucible-agent (worker): ok!
322024-07-10T20:04:01.815ZINFOcrucible-agent (worker): SMF ok!
332024-07-10T20:04:02.036ZINFOcrucible-agent (dropshot): request completed latency_us = 97 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:61821 req_id = f64a7d57-ff23-4046-8d75-16f185c704c6 response_code = 200 uri = /crucible/0/regions
342024-07-10T20:09:22.288ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:60044
352024-07-10T20:09:22.291ZINFOcrucible-agent (dropshot): request completed latency_us = 2576 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60044 req_id = 2f1d3bcd-c0e9-4709-bcb3-3a392f150460 response_code = 200 uri = /crucible/0/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af
362024-07-10T20:09:22.294ZINFOcrucible-agent (datafile): path is "/data/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af"
372024-07-10T20:09:22.303ZINFOcrucible-agent (datafile): dataset is oxp_a1d5dfa8-5a21-417c-aa81-606313589625/crucible/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af
382024-07-10T20:09:22.311ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a1d5dfa8-5a21-417c-aa81-606313589625/crucible/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af@9e302fab-288d-4912-8d43-d6946fbfde66
392024-07-10T20:09:22.311ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a1d5dfa8-5a21-417c-aa81-606313589625/crucible/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af", "9e302fab-288d-4912-8d43-d6946fbfde66"]
402024-07-10T20:09:22.321ZINFOcrucible-agent (dropshot): request completed latency_us = 29763 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60044 req_id = 0a5bdc93-9525-41d8-a4f4-dc6d9312ed8f response_code = 200 uri = /crucible/0/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af/snapshots/9e302fab-288d-4912-8d43-d6946fbfde66
412024-07-10T20:09:22.321ZINFOcrucible-agent (datafile): path is "/data/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af"
422024-07-10T20:09:22.328ZINFOcrucible-agent (datafile): dataset is oxp_a1d5dfa8-5a21-417c-aa81-606313589625/crucible/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af
432024-07-10T20:09:22.336ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a1d5dfa8-5a21-417c-aa81-606313589625/crucible/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af@9e302fab-288d-4912-8d43-d6946fbfde66
442024-07-10T20:09:22.336ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a1d5dfa8-5a21-417c-aa81-606313589625/crucible/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af", "9e302fab-288d-4912-8d43-d6946fbfde66"]
452024-07-10T20:09:22.345ZINFOcrucible-agent (datafile): requesting running snapshot a21d4d03-976a-46fc-8f5e-51d23bf402af-9e302fab-288d-4912-8d43-d6946fbfde66 state: Requested
462024-07-10T20:09:22.346ZINFOcrucible-agent (worker): applying SMF actions for region a21d4d03-976a-46fc-8f5e-51d23bf402af running snapshot 9e302fab-288d-4912-8d43-d6946fbfde66 (state Requested)...
472024-07-10T20:09:22.346ZINFOcrucible-agent (dropshot): request completed latency_us = 24727 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60044 req_id = 394de2d7-30b0-4b40-9dae-d1b28346bd61 response_code = 200 uri = /crucible/0/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af/snapshots/9e302fab-288d-4912-8d43-d6946fbfde66/run
482024-07-10T20:09:22.357ZINFOcrucible-agent (worker): existing directory value /dev/null does not match /data/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af
492024-07-10T20:09:22.357ZINFOcrucible-agent (worker): existing port value 0 does not match 19000
502024-07-10T20:09:22.357ZINFOcrucible-agent (worker): existing address value 0.0.0.0 does not match fd00:1122:3344:101::15
512024-07-10T20:09:22.357ZINFOcrucible-agent (worker): reconfig required
522024-07-10T20:09:22.357ZINFOcrucible-agent (worker): using existing config property group
532024-07-10T20:09:22.357ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a21d4d03-976a-46fc-8f5e-51d23bf402af
542024-07-10T20:09:22.357ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af
552024-07-10T20:09:22.357ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
562024-07-10T20:09:22.357ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
572024-07-10T20:09:22.357ZINFOcrucible-agent (worker): commit
582024-07-10T20:09:22.366ZINFOcrucible-agent (worker): ok!
592024-07-10T20:09:22.367ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-a21d4d03-976a-46fc-8f5e-51d23bf402af-9e302fab-288d-4912-8d43-d6946fbfde66
602024-07-10T20:09:22.370ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-a21d4d03-976a-46fc-8f5e-51d23bf402af-9e302fab-288d-4912-8d43-d6946fbfde66
612024-07-10T20:09:22.382ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-a21d4d03-976a-46fc-8f5e-51d23bf402af-9e302fab-288d-4912-8d43-d6946fbfde66
622024-07-10T20:09:22.382ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a21d4d03-976a-46fc-8f5e-51d23bf402af/.zfs/snapshot/9e302fab-288d-4912-8d43-d6946fbfde66
632024-07-10T20:09:22.382ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
642024-07-10T20:09:22.382ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
652024-07-10T20:09:22.382ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
662024-07-10T20:09:22.382ZINFOcrucible-agent (worker): commit
672024-07-10T20:09:22.386ZINFOcrucible-agent (worker): ok!
682024-07-10T20:09:22.390ZINFOcrucible-agent (worker): SMF ok!
692024-07-10T20:09:22.390ZINFOcrucible-agent (datafile): region a21d4d03-976a-46fc-8f5e-51d23bf402af running snapshot 9e302fab-288d-4912-8d43-d6946fbfde66 state: Requested -> Created