/var/tmp/bins/crutest verify -t 127.0.0.1:8810 -t 127.0.0.1:8820 -t 127.0.0.1:8830 --verify-out /tmp/test_repair_verify.data --verify-in /tmp/test_repair_verify.data --range -q -g 199 {"msg":"Upstairs starts","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.779682212Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958} {"msg":"Crucible Version: BuildInfo {\n version: \"0.0.1\",\n git_sha: \"2ed025a1b445016a0c73d0c9f3b9d9cefc7e2924\",\n git_commit_timestamp: \"2023-12-10T18:22:02.000000000Z\",\n git_branch: \"renovate/rusqlite-0.x\",\n rustc_semver: \"1.73.0\",\n rustc_channel: \"stable\",\n rustc_host_triple: \"x86_64-unknown-illumos\",\n rustc_commit_sha: \"cc66ad468955717ab92600c770da8c1601a4ff33\",\n cargo_triple: \"x86_64-unknown-illumos\",\n debug: true,\n opt_level: 0,\n}","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780077959Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958} {"msg":"Upstairs <-> Downstairs Message Version: 5","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780105169Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958} {"msg":"Crucible f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 has session id: c9a85fe0-8af2-47fe-a897-455ed079cea0","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780126544Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Upstairs opts: Upstairs UUID: f62d94ba-2d01-4e79-af70-fd9fb1c8eb45, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780160956Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Crucible stats registered with UUID: f62d94ba-2d01-4e79-af70-fd9fb1c8eb45","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780184292Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} Crucible runtime is spawned The guest has requested activation with gen:199 {"msg":"up_listen starts","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780497568Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0","task":"up_listen"} {"msg":"[1] connecting to 127.0.0.1:8820","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780539936Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"looper":"1","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] connecting to 127.0.0.1:8830","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780588852Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"looper":"2","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] connecting to 127.0.0.1:8810","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780632245Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"looper":"0","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Wait for all three downstairs to come online","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78075716Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Flush timeout: 0.5","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78079779Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 active request set","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780835256Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 looper connected","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780873941Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"looper":"1","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Set desired generation to :199","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780924505Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 looper connected","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780966831Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"looper":"0","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 looper connected","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.780993378Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"looper":"2","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Proc runs for 127.0.0.1:8820 in state New","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.781015019Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] Proc runs for 127.0.0.1:8810 in state New","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.781346027Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Proc runs for 127.0.0.1:8830 in state New","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.781461515Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 (c9a85fe0-8af2-47fe-a897-455ed079cea0) New New New ds_transition to WaitActive","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.781613983Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.781662513Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client is_active_req TRUE, promote! session c9a85fe0-8af2-47fe-a897-455ed079cea0","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.781705898Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 (c9a85fe0-8af2-47fe-a897-455ed079cea0) New WaitActive New ds_transition to WaitActive","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.781749951Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.782028365Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client is_active_req TRUE, promote! session c9a85fe0-8af2-47fe-a897-455ed079cea0","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.782126604Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.782543965Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.782579454Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 20, shift: 12 }, extent_count: 30, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.782606129Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 20, shift: 12 }, extent_count: 30, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.782636817Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 WaitActive WaitActive New","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78281637Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 WaitActive WaitActive New","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.782930894Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 (c9a85fe0-8af2-47fe-a897-455ed079cea0) WaitActive WaitActive New ds_transition to WaitActive","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.782977356Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783001198Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client is_active_req TRUE, promote! session c9a85fe0-8af2-47fe-a897-455ed079cea0","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783023114Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 (c9a85fe0-8af2-47fe-a897-455ed079cea0) WaitActive WaitActive WaitActive ds_transition to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78345084Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783487226Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] new RM replaced this: None","v":0,"name":"crucible","level":40,"time":"2023-12-10T18:59:27.783519994Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 (c9a85fe0-8af2-47fe-a897-455ed079cea0) WaitQuorum WaitActive WaitActive ds_transition to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783559314Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78358322Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] new RM replaced this: None","v":0,"name":"crucible","level":40,"time":"2023-12-10T18:59:27.783611674Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Starts reconcile loop","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783650434Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] Starts reconcile loop","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783697055Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] 127.0.0.1:8810 task reports connection:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783740351Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 WaitQuorum WaitQuorum WaitActive","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78378413Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Waiting for 1 more clients to be ready","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783823102Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Set check for repair","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783863979Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783906431Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] 127.0.0.1:8820 task reports connection:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783949907Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 20, shift: 12 }, extent_count: 30, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.783983113Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 WaitQuorum WaitQuorum WaitActive","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784004822Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Waiting for 1 more clients to be ready","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784025228Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Set check for repair","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784045183Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 WaitQuorum WaitQuorum WaitActive","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784066277Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 (c9a85fe0-8af2-47fe-a897-455ed079cea0) WaitQuorum WaitQuorum WaitActive ds_transition to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784561144Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784591199Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] new RM replaced this: None","v":0,"name":"crucible","level":40,"time":"2023-12-10T18:59:27.784612255Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Starts reconcile loop","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784637593Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] 127.0.0.1:8830 task reports connection:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784659333Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 WaitQuorum WaitQuorum WaitQuorum","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784680773Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0]R flush_numbers[0..12]: [346, 343, 345, 349, 346, 346, 351, 351, 349, 350, 349, 343]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784702544Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0]R generation[0..12]: [196, 194, 195, 198, 196, 196, 198, 198, 198, 198, 198, 194]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784724203Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0]R dirty[0..12]: [true, true, false, true, false, false, false, false, false, false, false, true]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784746468Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1]R flush_numbers[0..12]: [346, 343, 345, 349, 346, 346, 351, 351, 349, 350, 349, 343]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784768207Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1]R generation[0..12]: [196, 194, 195, 198, 196, 196, 198, 198, 198, 198, 198, 194]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784789827Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1]R dirty[0..12]: [true, true, false, true, false, false, false, false, false, false, false, true]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784811757Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2]R flush_numbers[0..12]: [346, 343, 345, 329, 346, 346, 346, 342, 344, 345, 341, 343]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784833711Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2]R generation[0..12]: [196, 194, 195, 188, 196, 196, 196, 194, 194, 195, 193, 194]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784855475Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2]R dirty[0..12]: [false, false, false, true, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784877711Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Max found gen is 199","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784897698Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Generation requested: 199 >= found:199","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784918195Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Next flush: 352","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784938188Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extents 0 dirty","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784961385Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 0","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.784983279Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:0 gens: 196 196 196","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78500568Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:0 flush: 346 346 346 scs: [ClientId(0), ClientId(1), ClientId(2)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785029042Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1), ClientId(2)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785055536Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:0 dirty: true true false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78507842Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 0","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785100975Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 1 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785123398Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785145512Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extents 1 dirty","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785168313Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 1","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785189787Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:1 gens: 194 194 194","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785211809Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:1 flush: 343 343 343 scs: [ClientId(0), ClientId(1), ClientId(2)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785268064Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1), ClientId(2)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785320776Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:1 dirty: true true false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785349136Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 1","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785397229Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 1 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785432256Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785460423Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extents 3 dirty","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785508098Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 3","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785534035Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:3 gens: 198 198 188","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785580635Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:3 flush: 349 349 329 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785609104Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785657063Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:3 dirty: true true true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785706491Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 3","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785733083Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 1 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785755721Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785777874Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extents 11 dirty","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785823139Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 11","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785848741Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:11 gens: 194 194 194","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785871229Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:11 flush: 343 343 343 scs: [ClientId(0), ClientId(1), ClientId(2)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785894165Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1), ClientId(2)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785921231Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:11 dirty: true true false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785944223Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 11","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785966462Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 1 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.785988459Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786010505Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extents 21 dirty","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786032389Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 21","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786053473Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:21 gens: 197 197 197","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786075367Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:21 flush: 348 348 348 scs: [ClientId(0), ClientId(1), ClientId(2)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786098308Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1), ClientId(2)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78612098Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:21 dirty: true true false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786144127Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 21","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786185274Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 1 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786229518Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786272064Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extents 22 dirty","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786315596Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 22","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786358158Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:22 gens: 192 192 192","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786407571Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:22 flush: 340 340 340 scs: [ClientId(0), ClientId(1), ClientId(2)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786445211Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1), ClientId(2)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786469405Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:22 dirty: true true false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786491334Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 22","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786513553Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 1 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786535398Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 source flush","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786557273Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"dirty","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 6 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786579828Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 6","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78660098Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:6 gens: 198 198 196","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786623513Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:6 flush: 351 351 346 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78664641Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786668906Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:6 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78669112Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786713706Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 6","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786736079Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786762166Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 7 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786802157Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 7","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786833917Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:7 gens: 198 198 194","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786863783Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:7 flush: 351 351 342 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786887573Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786910054Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:7 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786932155Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786954879Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 7","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.786984566Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787007687Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 8 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787030374Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 8","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787051386Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:8 gens: 198 198 194","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787073285Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:8 flush: 349 349 344 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787095883Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787118135Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:8 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787144118Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787167754Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 8","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787190145Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787212038Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 9 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787234228Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 9","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787255161Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:9 gens: 198 198 195","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787277257Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:9 flush: 350 350 345 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787299854Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787330657Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:9 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787373849Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787418048Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 9","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787463653Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787498376Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 10 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78752243Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 10","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78754359Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:10 gens: 198 198 193","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787570493Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:10 flush: 349 349 341 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787594247Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787616674Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:10 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787638778Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787661326Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 10","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787683869Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787705738Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 12 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787728187Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 12","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787749028Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:12 gens: 198 198 190","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787771085Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:12 flush: 351 351 335 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787793835Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787816309Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:12 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787838299Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787860922Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 12","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787886682Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787909273Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 14 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787931706Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 14","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787952638Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:14 gens: 198 198 188","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78797448Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:14 flush: 351 351 329 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.787997245Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788019639Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:14 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78804156Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788064137Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 14","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788086416Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788108272Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 19 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788130555Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 19","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788151551Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:19 gens: 198 198 194","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788173546Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:19 flush: 349 349 342 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.7881963Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788222099Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:19 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788244735Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788267335Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 19","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788289646Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788311538Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 20 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788333898Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 20","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788354861Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:20 gens: 198 198 192","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788376732Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:20 flush: 350 350 340 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78839924Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788421645Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:20 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788443551Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788466131Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 20","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788488244Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788510073Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 27 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788532277Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 27","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788556149Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:27 gens: 198 198 188","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788578814Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:27 flush: 349 349 329 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788601658Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788624001Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:27 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788646087Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788668687Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 27","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788691068Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788713075Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Extent 28 has flush number mismatch","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788735299Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"First source client ID for extent 28","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788756206Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:28 gens: 198 198 196","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788777941Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:28 flush: 349 349 346 scs: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78880063Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"max_flush now has: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788823055Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"extent:28 dirty: false false false","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788845131Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No maxes found, left with: [ClientId(0), ClientId(1)]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788867682Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"find dest for source 0 for extent at index 28","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788892913Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"source 0, add dest 2 gen","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788915341Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"mrl":"flush_mismatch","":"mend","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Transition from WaitQuorum to Repair","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788937757Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Transition from WaitQuorum to Repair","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788957832Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Transition from WaitQuorum to Repair","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788977767Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Found 17 extents that need repair","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.788997713Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Full repair list: {19: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 3: ExtentFix { source: ClientId(0), dest: [ClientId(1), ClientId(2)] }, 28: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 27: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 22: ExtentFix { source: ClientId(0), dest: [ClientId(1), ClientId(2)] }, 8: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 9: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 0: ExtentFix { source: ClientId(0), dest: [ClientId(1), ClientId(2)] }, 6: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 12: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 7: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 21: ExtentFix { source: ClientId(0), dest: [ClientId(1), ClientId(2)] }, 20: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 11: ExtentFix { source: ClientId(0), dest: [ClientId(1), ClientId(2)] }, 10: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 14: ExtentFix { source: ClientId(0), dest: [ClientId(2)] }, 1: ExtentFix { source: ClientId(0), dest: [ClientId(1), ClientId(2)] }}","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789048411Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Task list: [ReconcileIO { id: ReconciliationId(0), op: ExtentFlush { repair_id: ReconciliationId(0), extent_id: 19, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(1), op: ExtentClose { repair_id: ReconciliationId(1), extent_id: 19 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(2), op: ExtentRepair { repair_id: ReconciliationId(2), extent_id: 19, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(3), op: ExtentReopen { repair_id: ReconciliationId(3), extent_id: 19 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(4), op: ExtentFlush { repair_id: ReconciliationId(4), extent_id: 3, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(5), op: ExtentClose { repair_id: ReconciliationId(5), extent_id: 3 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(6), op: ExtentRepair { repair_id: ReconciliationId(6), extent_id: 3, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(7), op: ExtentReopen { repair_id: ReconciliationId(7), extent_id: 3 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(8), op: ExtentFlush { repair_id: ReconciliationId(8), extent_id: 28, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(9), op: ExtentClose { repair_id: ReconciliationId(9), extent_id: 28 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(10), op: ExtentRepair { repair_id: ReconciliationId(10), extent_id: 28, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(11), op: ExtentReopen { repair_id: ReconciliationId(11), extent_id: 28 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(12), op: ExtentFlush { repair_id: ReconciliationId(12), extent_id: 27, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(13), op: ExtentClose { repair_id: ReconciliationId(13), extent_id: 27 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(14), op: ExtentRepair { repair_id: ReconciliationId(14), extent_id: 27, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(15), op: ExtentReopen { repair_id: ReconciliationId(15), extent_id: 27 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(16), op: ExtentFlush { repair_id: ReconciliationId(16), extent_id: 22, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(17), op: ExtentClose { repair_id: ReconciliationId(17), extent_id: 22 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(18), op: ExtentRepair { repair_id: ReconciliationId(18), extent_id: 22, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(19), op: ExtentReopen { repair_id: ReconciliationId(19), extent_id: 22 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(20), op: ExtentFlush { repair_id: ReconciliationId(20), extent_id: 8, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(21), op: ExtentClose { repair_id: ReconciliationId(21), extent_id: 8 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(22), op: ExtentRepair { repair_id: ReconciliationId(22), extent_id: 8, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(23), op: ExtentReopen { repair_id: ReconciliationId(23), extent_id: 8 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(24), op: ExtentFlush { repair_id: ReconciliationId(24), extent_id: 9, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(25), op: ExtentClose { repair_id: ReconciliationId(25), extent_id: 9 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(26), op: ExtentRepair { repair_id: ReconciliationId(26), extent_id: 9, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(27), op: ExtentReopen { repair_id: ReconciliationId(27), extent_id: 9 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(28), op: ExtentFlush { repair_id: ReconciliationId(28), extent_id: 0, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(29), op: ExtentClose { repair_id: ReconciliationId(29), extent_id: 0 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(30), op: ExtentRepair { repair_id: ReconciliationId(30), extent_id: 0, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(31), op: ExtentReopen { repair_id: ReconciliationId(31), extent_id: 0 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(32), op: ExtentFlush { repair_id: ReconciliationId(32), extent_id: 6, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(33), op: ExtentClose { repair_id: ReconciliationId(33), extent_id: 6 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(34), op: ExtentRepair { repair_id: ReconciliationId(34), extent_id: 6, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(35), op: ExtentReopen { repair_id: ReconciliationId(35), extent_id: 6 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(36), op: ExtentFlush { repair_id: ReconciliationId(36), extent_id: 12, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(37), op: ExtentClose { repair_id: ReconciliationId(37), extent_id: 12 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(38), op: ExtentRepair { repair_id: ReconciliationId(38), extent_id: 12, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(39), op: ExtentReopen { repair_id: ReconciliationId(39), extent_id: 12 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(40), op: ExtentFlush { repair_id: ReconciliationId(40), extent_id: 7, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(41), op: ExtentClose { repair_id: ReconciliationId(41), extent_id: 7 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(42), op: ExtentRepair { repair_id: ReconciliationId(42), extent_id: 7, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(43), op: ExtentReopen { repair_id: ReconciliationId(43), extent_id: 7 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(44), op: ExtentFlush { repair_id: ReconciliationId(44), extent_id: 21, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(45), op: ExtentClose { repair_id: ReconciliationId(45), extent_id: 21 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(46), op: ExtentRepair { repair_id: ReconciliationId(46), extent_id: 21, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(47), op: ExtentReopen { repair_id: ReconciliationId(47), extent_id: 21 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(48), op: ExtentFlush { repair_id: ReconciliationId(48), extent_id: 20, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(49), op: ExtentClose { repair_id: ReconciliationId(49), extent_id: 20 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(50), op: ExtentRepair { repair_id: ReconciliationId(50), extent_id: 20, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(51), op: ExtentReopen { repair_id: ReconciliationId(51), extent_id: 20 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(52), op: ExtentFlush { repair_id: ReconciliationId(52), extent_id: 11, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(53), op: ExtentClose { repair_id: ReconciliationId(53), extent_id: 11 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(54), op: ExtentRepair { repair_id: ReconciliationId(54), extent_id: 11, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(55), op: ExtentReopen { repair_id: ReconciliationId(55), extent_id: 11 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(56), op: ExtentFlush { repair_id: ReconciliationId(56), extent_id: 10, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(57), op: ExtentClose { repair_id: ReconciliationId(57), extent_id: 10 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(58), op: ExtentRepair { repair_id: ReconciliationId(58), extent_id: 10, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(59), op: ExtentReopen { repair_id: ReconciliationId(59), extent_id: 10 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(60), op: ExtentFlush { repair_id: ReconciliationId(60), extent_id: 14, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(61), op: ExtentClose { repair_id: ReconciliationId(61), extent_id: 14 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(62), op: ExtentRepair { repair_id: ReconciliationId(62), extent_id: 14, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(63), op: ExtentReopen { repair_id: ReconciliationId(63), extent_id: 14 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(64), op: ExtentFlush { repair_id: ReconciliationId(64), extent_id: 1, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(65), op: ExtentClose { repair_id: ReconciliationId(65), extent_id: 1 }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(66), op: ExtentRepair { repair_id: ReconciliationId(66), extent_id: 1, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }, ReconcileIO { id: ReconciliationId(67), op: ExtentReopen { repair_id: ReconciliationId(67), extent_id: 1 }, state: ClientData([New, New, New]) }]","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789406894Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Begin repair with 68 commands","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789432845Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(0), op: ExtentFlush { repair_id: ReconciliationId(0), extent_id: 19, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78945804Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789481534Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789502448Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(0), op: ExtentFlush { repair_id: ReconciliationId(0), extent_id: 19, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78952766Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(0), extent_id: 19, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789551701Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789571883Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(0), op: ExtentFlush { repair_id: ReconciliationId(0), extent_id: 19, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, New, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789596914Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.78961798Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(0), extent_id: 19, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789640552Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(0), op: ExtentFlush { repair_id: ReconciliationId(0), extent_id: 19, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789666087Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(0), extent_id: 19, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789689516Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:0 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789709796Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789730009Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(1), op: ExtentClose { repair_id: ReconciliationId(1), extent_id: 19 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789753075Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789776095Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789796819Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(1), op: ExtentClose { repair_id: ReconciliationId(1), extent_id: 19 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789821263Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(1), extent_id: 19 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789843578Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789863928Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(1), op: ExtentClose { repair_id: ReconciliationId(1), extent_id: 19 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789911644Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(1), extent_id: 19 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789934637Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789954802Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(1), op: ExtentClose { repair_id: ReconciliationId(1), extent_id: 19 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.789978864Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(1), extent_id: 19 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790000795Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:1 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790020858Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790041085Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(2), op: ExtentRepair { repair_id: ReconciliationId(2), extent_id: 19, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790066562Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790090354Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790111361Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(2), op: ExtentRepair { repair_id: ReconciliationId(2), extent_id: 19, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790137721Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(2), extent_id: 19, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790162563Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(2)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790183209Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790203329Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(2), op: ExtentRepair { repair_id: ReconciliationId(2), extent_id: 19, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790229638Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(2), extent_id: 19, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790254611Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(2)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790275093Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790295225Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(2), op: ExtentRepair { repair_id: ReconciliationId(2), extent_id: 19, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790321616Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(2), extent_id: 19, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790346235Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(2)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.790369856Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:2 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.876883626Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[3/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.876923307Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(3), op: ExtentReopen { repair_id: ReconciliationId(3), extent_id: 19 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.876949058Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.876970014Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.876990773Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(3), op: ExtentReopen { repair_id: ReconciliationId(3), extent_id: 19 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877014812Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(3), extent_id: 19 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877038875Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877076319Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(3), op: ExtentReopen { repair_id: ReconciliationId(3), extent_id: 19 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877123749Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(3), extent_id: 19 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877171867Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877213507Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(3), op: ExtentReopen { repair_id: ReconciliationId(3), extent_id: 19 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877259052Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(3), extent_id: 19 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877305444Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:3 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.87755084Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[4/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877580901Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(4), op: ExtentFlush { repair_id: ReconciliationId(4), extent_id: 3, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877606739Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877627109Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877647262Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(4), op: ExtentFlush { repair_id: ReconciliationId(4), extent_id: 3, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877672532Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(4), extent_id: 3, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.87769639Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877716469Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(4), op: ExtentFlush { repair_id: ReconciliationId(4), extent_id: 3, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877749552Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(4), extent_id: 3, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877795552Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877839249Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(4), op: ExtentFlush { repair_id: ReconciliationId(4), extent_id: 3, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, Done, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.877890498Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(4), extent_id: 3, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.87793735Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:4 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879154994Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[5/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879189966Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(5), op: ExtentClose { repair_id: ReconciliationId(5), extent_id: 3 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879214395Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879234815Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879255058Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(5), op: ExtentClose { repair_id: ReconciliationId(5), extent_id: 3 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879278925Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(5), extent_id: 3 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879313311Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879354194Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(5), op: ExtentClose { repair_id: ReconciliationId(5), extent_id: 3 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879404037Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(5), extent_id: 3 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879451226Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879491154Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(5), op: ExtentClose { repair_id: ReconciliationId(5), extent_id: 3 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.87951914Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(5), extent_id: 3 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879541768Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:5 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879690759Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[6/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879725339Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(6), op: ExtentRepair { repair_id: ReconciliationId(6), extent_id: 3, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879753252Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879774087Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879794247Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(6), op: ExtentRepair { repair_id: ReconciliationId(6), extent_id: 3, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879821077Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(6), extent_id: 3, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879865687Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(6)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879896238Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879919877Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(6), op: ExtentRepair { repair_id: ReconciliationId(6), extent_id: 3, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.879966748Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(6), extent_id: 3, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.880008915Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(6)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.880032488Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.88005247Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(6), op: ExtentRepair { repair_id: ReconciliationId(6), extent_id: 3, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.880079106Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(6), extent_id: 3, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.880108653Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Sending repair request ReconciliationId(6)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.880130359Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:6 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956112119Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[7/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956150855Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(7), op: ExtentReopen { repair_id: ReconciliationId(7), extent_id: 3 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956176341Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956197393Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.95621778Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(7), op: ExtentReopen { repair_id: ReconciliationId(7), extent_id: 3 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956242143Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(7), extent_id: 3 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956281716Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956310743Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(7), op: ExtentReopen { repair_id: ReconciliationId(7), extent_id: 3 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956335449Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(7), extent_id: 3 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956357542Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956377538Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(7), op: ExtentReopen { repair_id: ReconciliationId(7), extent_id: 3 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956407017Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(7), extent_id: 3 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956430119Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:7 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956682375Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[8/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956712494Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(8), op: ExtentFlush { repair_id: ReconciliationId(8), extent_id: 28, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956737866Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956758539Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956778827Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(8), op: ExtentFlush { repair_id: ReconciliationId(8), extent_id: 28, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.95682075Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(8), extent_id: 28, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956901491Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956927623Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956948424Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(8), op: ExtentFlush { repair_id: ReconciliationId(8), extent_id: 28, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956973947Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(8), extent_id: 28, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.956998072Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(8), op: ExtentFlush { repair_id: ReconciliationId(8), extent_id: 28, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957028844Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(8), extent_id: 28, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.95705411Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:8 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957095094Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[9/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957117973Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(9), op: ExtentClose { repair_id: ReconciliationId(9), extent_id: 28 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957141797Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957163443Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957186457Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(9), op: ExtentClose { repair_id: ReconciliationId(9), extent_id: 28 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957210834Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(9), extent_id: 28 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957233276Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957253762Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(9), op: ExtentClose { repair_id: ReconciliationId(9), extent_id: 28 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957277912Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(9), extent_id: 28 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957304178Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.95732976Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(9), op: ExtentClose { repair_id: ReconciliationId(9), extent_id: 28 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957358619Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(9), extent_id: 28 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957381506Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:9 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957675149Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[10/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957703475Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(10), op: ExtentRepair { repair_id: ReconciliationId(10), extent_id: 28, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957730022Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957750639Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957771137Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(10), op: ExtentRepair { repair_id: ReconciliationId(10), extent_id: 28, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957797545Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(10), extent_id: 28, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957822706Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(10)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957898442Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957926796Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(10), op: ExtentRepair { repair_id: ReconciliationId(10), extent_id: 28, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957954278Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(10), extent_id: 28, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.957979223Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(10)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.95800452Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.958025769Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(10), op: ExtentRepair { repair_id: ReconciliationId(10), extent_id: 28, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.958052662Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(10), extent_id: 28, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.958077751Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(10)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:27.958098316Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:10 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033025374Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[11/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033064432Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(11), op: ExtentReopen { repair_id: ReconciliationId(11), extent_id: 28 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033090046Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.03311098Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033131781Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(11), op: ExtentReopen { repair_id: ReconciliationId(11), extent_id: 28 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.03317278Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(11), extent_id: 28 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033218189Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033261153Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(11), op: ExtentReopen { repair_id: ReconciliationId(11), extent_id: 28 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033314984Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(11), extent_id: 28 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033359127Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033400327Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(11), op: ExtentReopen { repair_id: ReconciliationId(11), extent_id: 28 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033446305Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(11), extent_id: 28 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033475686Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:11 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033696059Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[12/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033725803Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(12), op: ExtentFlush { repair_id: ReconciliationId(12), extent_id: 27, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.03375171Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033772367Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.03380007Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(12), op: ExtentFlush { repair_id: ReconciliationId(12), extent_id: 27, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033893901Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(12), extent_id: 27, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033927307Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.03394832Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033968512Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(12), op: ExtentFlush { repair_id: ReconciliationId(12), extent_id: 27, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.033999863Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(12), extent_id: 27, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034024876Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(12), op: ExtentFlush { repair_id: ReconciliationId(12), extent_id: 27, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034050748Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(12), extent_id: 27, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034074644Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:12 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034098878Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[13/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.03411956Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(13), op: ExtentClose { repair_id: ReconciliationId(13), extent_id: 27 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.03414293Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034170346Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034191714Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(13), op: ExtentClose { repair_id: ReconciliationId(13), extent_id: 27 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034215673Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(13), extent_id: 27 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.0342437Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.03428038Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(13), op: ExtentClose { repair_id: ReconciliationId(13), extent_id: 27 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034327111Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(13), extent_id: 27 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034378275Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034420169Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(13), op: ExtentClose { repair_id: ReconciliationId(13), extent_id: 27 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034463444Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(13), extent_id: 27 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034490444Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:13 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034690049Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[14/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034718555Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(14), op: ExtentRepair { repair_id: ReconciliationId(14), extent_id: 27, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034745139Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034769152Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034804969Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(14), op: ExtentRepair { repair_id: ReconciliationId(14), extent_id: 27, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034862759Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(14), extent_id: 27, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034893282Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(14)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034927866Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.034967369Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(14), op: ExtentRepair { repair_id: ReconciliationId(14), extent_id: 27, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.0350242Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(14), extent_id: 27, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.035079403Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(14)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.035122344Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.035162095Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(14), op: ExtentRepair { repair_id: ReconciliationId(14), extent_id: 27, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.035213678Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(14), extent_id: 27, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.035257966Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(14)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.035294745Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:14 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106181377Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[15/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106221703Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(15), op: ExtentReopen { repair_id: ReconciliationId(15), extent_id: 27 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106247613Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106268447Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106288669Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(15), op: ExtentReopen { repair_id: ReconciliationId(15), extent_id: 27 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106312832Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(15), extent_id: 27 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106357268Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106389128Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(15), op: ExtentReopen { repair_id: ReconciliationId(15), extent_id: 27 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106414452Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(15), extent_id: 27 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106436745Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106457175Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(15), op: ExtentReopen { repair_id: ReconciliationId(15), extent_id: 27 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106481253Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(15), extent_id: 27 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106503551Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:15 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.10683774Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[16/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106876118Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(16), op: ExtentFlush { repair_id: ReconciliationId(16), extent_id: 22, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106903346Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106923902Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106946623Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(16), op: ExtentFlush { repair_id: ReconciliationId(16), extent_id: 22, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.106972381Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(16), extent_id: 22, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.107001748Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.107023093Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(16), op: ExtentFlush { repair_id: ReconciliationId(16), extent_id: 22, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.107048721Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(16), extent_id: 22, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.10707243Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.107092692Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(16), op: ExtentFlush { repair_id: ReconciliationId(16), extent_id: 22, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, Done, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.107117852Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(16), extent_id: 22, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.107141298Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:16 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108462642Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[17/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108492221Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(17), op: ExtentClose { repair_id: ReconciliationId(17), extent_id: 22 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108516405Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108536997Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108557572Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(17), op: ExtentClose { repair_id: ReconciliationId(17), extent_id: 22 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.10858171Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(17), extent_id: 22 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108608916Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108630217Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(17), op: ExtentClose { repair_id: ReconciliationId(17), extent_id: 22 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108654287Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(17), extent_id: 22 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108676672Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108705263Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(17), op: ExtentClose { repair_id: ReconciliationId(17), extent_id: 22 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108730923Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(17), extent_id: 22 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.108753304Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:17 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109012826Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[18/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109040186Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(18), op: ExtentRepair { repair_id: ReconciliationId(18), extent_id: 22, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109066918Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109087484Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109107674Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(18), op: ExtentRepair { repair_id: ReconciliationId(18), extent_id: 22, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109134486Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(18), extent_id: 22, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109163976Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(18)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109185392Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109205804Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(18), op: ExtentRepair { repair_id: ReconciliationId(18), extent_id: 22, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.10923254Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(18), extent_id: 22, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109257623Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(18)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109278344Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109298556Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(18), op: ExtentRepair { repair_id: ReconciliationId(18), extent_id: 22, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109325266Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(18), extent_id: 22, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109350127Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Sending repair request ReconciliationId(18)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.109370683Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:18 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187362504Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[19/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187407676Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(19), op: ExtentReopen { repair_id: ReconciliationId(19), extent_id: 22 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187433348Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.18745927Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187481006Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(19), op: ExtentReopen { repair_id: ReconciliationId(19), extent_id: 22 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187505331Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(19), extent_id: 22 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187528139Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187548477Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(19), op: ExtentReopen { repair_id: ReconciliationId(19), extent_id: 22 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187578295Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(19), extent_id: 22 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187604148Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187638597Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(19), op: ExtentReopen { repair_id: ReconciliationId(19), extent_id: 22 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187684762Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(19), extent_id: 22 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.187730791Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:19 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188056868Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[20/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188096748Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(20), op: ExtentFlush { repair_id: ReconciliationId(20), extent_id: 8, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188124108Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188151019Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188172804Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(20), op: ExtentFlush { repair_id: ReconciliationId(20), extent_id: 8, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188198635Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(20), extent_id: 8, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.18822337Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188244282Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(20), op: ExtentFlush { repair_id: ReconciliationId(20), extent_id: 8, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188269802Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188291094Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(20), extent_id: 8, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188313962Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(20), op: ExtentFlush { repair_id: ReconciliationId(20), extent_id: 8, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188339311Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(20), extent_id: 8, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188363031Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:20 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188500253Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[21/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.18852955Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(21), op: ExtentClose { repair_id: ReconciliationId(21), extent_id: 8 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188554003Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188579197Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188600523Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(21), op: ExtentClose { repair_id: ReconciliationId(21), extent_id: 8 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188624902Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(21), extent_id: 8 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188647529Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.18868078Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(21), op: ExtentClose { repair_id: ReconciliationId(21), extent_id: 8 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188707222Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(21), extent_id: 8 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188729501Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188749573Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(21), op: ExtentClose { repair_id: ReconciliationId(21), extent_id: 8 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.188773552Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(21), extent_id: 8 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.1887956Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:21 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189025057Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[22/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.18905365Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(22), op: ExtentRepair { repair_id: ReconciliationId(22), extent_id: 8, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189080262Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189105479Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189127032Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(22), op: ExtentRepair { repair_id: ReconciliationId(22), extent_id: 8, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189153662Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(22), extent_id: 8, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189178798Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(22)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189212962Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189246852Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(22), op: ExtentRepair { repair_id: ReconciliationId(22), extent_id: 8, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189275534Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(22), extent_id: 8, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189300598Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(22)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189321062Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189341223Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(22), op: ExtentRepair { repair_id: ReconciliationId(22), extent_id: 8, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189367597Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(22), extent_id: 8, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189392135Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(22)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.189416941Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:22 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.261928614Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[23/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.261968384Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(23), op: ExtentReopen { repair_id: ReconciliationId(23), extent_id: 8 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.261994332Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262015448Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262036061Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(23), op: ExtentReopen { repair_id: ReconciliationId(23), extent_id: 8 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262060162Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(23), extent_id: 8 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262082917Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262112972Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(23), op: ExtentReopen { repair_id: ReconciliationId(23), extent_id: 8 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262158988Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(23), extent_id: 8 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262206625Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262250156Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(23), op: ExtentReopen { repair_id: ReconciliationId(23), extent_id: 8 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262294124Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(23), extent_id: 8 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.2623396Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:23 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262580761Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[24/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262611926Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(24), op: ExtentFlush { repair_id: ReconciliationId(24), extent_id: 9, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262637687Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262658176Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262678583Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(24), op: ExtentFlush { repair_id: ReconciliationId(24), extent_id: 9, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262704211Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(24), extent_id: 9, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262728171Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262748627Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(24), op: ExtentFlush { repair_id: ReconciliationId(24), extent_id: 9, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262773883Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(24), extent_id: 9, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262809831Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262850286Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(24), op: ExtentFlush { repair_id: ReconciliationId(24), extent_id: 9, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, Done, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.262931178Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(24), extent_id: 9, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.26298184Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:24 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.26308675Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[25/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263200146Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(25), op: ExtentClose { repair_id: ReconciliationId(25), extent_id: 9 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263326919Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.26341476Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263450506Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(25), op: ExtentClose { repair_id: ReconciliationId(25), extent_id: 9 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263496906Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(25), extent_id: 9 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263543643Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263585047Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(25), op: ExtentClose { repair_id: ReconciliationId(25), extent_id: 9 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263626365Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(25), extent_id: 9 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263651357Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263671392Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(25), op: ExtentClose { repair_id: ReconciliationId(25), extent_id: 9 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263695381Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(25), extent_id: 9 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263717221Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:25 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263738451Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[26/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263764522Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(26), op: ExtentRepair { repair_id: ReconciliationId(26), extent_id: 9, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263791296Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263811892Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.26383227Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(26), op: ExtentRepair { repair_id: ReconciliationId(26), extent_id: 9, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263858779Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263887273Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(26), op: ExtentRepair { repair_id: ReconciliationId(26), extent_id: 9, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263914752Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(26), extent_id: 9, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263939394Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(26), extent_id: 9, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263963069Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(26)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.263983313Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(26)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.264003833Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.264023893Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(26), op: ExtentRepair { repair_id: ReconciliationId(26), extent_id: 9, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.26405Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(26), extent_id: 9, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.264078365Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(26)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.264099569Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:26 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.338878577Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[27/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.338914337Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(27), op: ExtentReopen { repair_id: ReconciliationId(27), extent_id: 9 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.338939475Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.338959769Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.338979923Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(27), op: ExtentReopen { repair_id: ReconciliationId(27), extent_id: 9 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339003996Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(27), extent_id: 9 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339026288Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339058879Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(27), op: ExtentReopen { repair_id: ReconciliationId(27), extent_id: 9 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339096038Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(27), extent_id: 9 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339131478Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339170373Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(27), op: ExtentReopen { repair_id: ReconciliationId(27), extent_id: 9 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339225072Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(27), extent_id: 9 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339270561Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:27 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339516218Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[28/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339555957Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(28), op: ExtentFlush { repair_id: ReconciliationId(28), extent_id: 0, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339583775Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339604958Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339625511Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(28), op: ExtentFlush { repair_id: ReconciliationId(28), extent_id: 0, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339651201Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(28), extent_id: 0, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339675594Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339705919Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339745592Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(28), op: ExtentFlush { repair_id: ReconciliationId(28), extent_id: 0, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, New, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339796806Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(28), extent_id: 0, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339847091Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(28), op: ExtentFlush { repair_id: ReconciliationId(28), extent_id: 0, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339898403Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(28), extent_id: 0, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.339927189Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:28 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.340990022Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[29/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341024405Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(29), op: ExtentClose { repair_id: ReconciliationId(29), extent_id: 0 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341049083Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341069745Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341090077Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(29), op: ExtentClose { repair_id: ReconciliationId(29), extent_id: 0 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341114058Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(29), extent_id: 0 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341136568Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341166301Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(29), op: ExtentClose { repair_id: ReconciliationId(29), extent_id: 0 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341211838Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(29), extent_id: 0 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341259663Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341303096Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(29), op: ExtentClose { repair_id: ReconciliationId(29), extent_id: 0 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341353182Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(29), extent_id: 0 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341380799Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:29 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341537562Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[30/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341567171Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(30), op: ExtentRepair { repair_id: ReconciliationId(30), extent_id: 0, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341594654Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341615438Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341635756Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(30), op: ExtentRepair { repair_id: ReconciliationId(30), extent_id: 0, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341662691Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(30), extent_id: 0, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341687798Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(30)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341708858Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341729076Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(30), op: ExtentRepair { repair_id: ReconciliationId(30), extent_id: 0, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341755889Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(30), extent_id: 0, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341780968Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(30)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341806847Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341828625Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(30), op: ExtentRepair { repair_id: ReconciliationId(30), extent_id: 0, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341855652Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(30), extent_id: 0, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341880922Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Sending repair request ReconciliationId(30)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.341901739Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:30 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.41728858Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[31/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417327171Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(31), op: ExtentReopen { repair_id: ReconciliationId(31), extent_id: 0 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417353124Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417373924Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417394336Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(31), op: ExtentReopen { repair_id: ReconciliationId(31), extent_id: 0 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417418582Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(31), extent_id: 0 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.41744123Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417461586Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(31), op: ExtentReopen { repair_id: ReconciliationId(31), extent_id: 0 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417491495Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(31), extent_id: 0 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417571387Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417592292Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(31), op: ExtentReopen { repair_id: ReconciliationId(31), extent_id: 0 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417616562Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(31), extent_id: 0 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417638924Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:31 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417914067Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[32/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417942987Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(32), op: ExtentFlush { repair_id: ReconciliationId(32), extent_id: 6, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417968566Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.417989169Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418013166Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(32), op: ExtentFlush { repair_id: ReconciliationId(32), extent_id: 6, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.41803899Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(32), extent_id: 6, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418063248Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418083603Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418109185Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(32), op: ExtentFlush { repair_id: ReconciliationId(32), extent_id: 6, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418135728Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(32), extent_id: 6, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418159735Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(32), op: ExtentFlush { repair_id: ReconciliationId(32), extent_id: 6, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418185389Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(32), extent_id: 6, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418209072Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:32 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418423194Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[33/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418451714Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(33), op: ExtentClose { repair_id: ReconciliationId(33), extent_id: 6 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418476011Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.41849647Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418517012Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(33), op: ExtentClose { repair_id: ReconciliationId(33), extent_id: 6 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418540809Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(33), extent_id: 6 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418563128Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.41858319Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(33), op: ExtentClose { repair_id: ReconciliationId(33), extent_id: 6 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418611805Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(33), extent_id: 6 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418634752Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418668828Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(33), op: ExtentClose { repair_id: ReconciliationId(33), extent_id: 6 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418716704Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(33), extent_id: 6 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.418765265Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:33 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.41906281Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[34/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419100943Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(34), op: ExtentRepair { repair_id: ReconciliationId(34), extent_id: 6, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419129338Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419155403Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419192417Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(34), op: ExtentRepair { repair_id: ReconciliationId(34), extent_id: 6, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419240335Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(34), extent_id: 6, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.41928763Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(34)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419329338Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419364811Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(34), op: ExtentRepair { repair_id: ReconciliationId(34), extent_id: 6, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419393033Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(34), extent_id: 6, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419417926Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(34)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419438475Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419458545Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(34), op: ExtentRepair { repair_id: ReconciliationId(34), extent_id: 6, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419485218Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(34), extent_id: 6, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419509707Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(34)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.419529902Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:34 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.495757118Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[35/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.49581451Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(35), op: ExtentReopen { repair_id: ReconciliationId(35), extent_id: 6 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.495840146Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.495861239Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.495881502Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(35), op: ExtentReopen { repair_id: ReconciliationId(35), extent_id: 6 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.495921971Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(35), extent_id: 6 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.495962064Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.495987255Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(35), op: ExtentReopen { repair_id: ReconciliationId(35), extent_id: 6 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496028273Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(35), extent_id: 6 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496069797Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496110315Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(35), op: ExtentReopen { repair_id: ReconciliationId(35), extent_id: 6 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496157334Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(35), extent_id: 6 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496197106Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:35 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496424241Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[36/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496464996Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(36), op: ExtentFlush { repair_id: ReconciliationId(36), extent_id: 12, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496492414Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496513713Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496534355Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(36), op: ExtentFlush { repair_id: ReconciliationId(36), extent_id: 12, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496560058Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(36), extent_id: 12, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496650507Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496677756Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496698475Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(36), op: ExtentFlush { repair_id: ReconciliationId(36), extent_id: 12, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.49672392Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(36), extent_id: 12, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496748428Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(36), op: ExtentFlush { repair_id: ReconciliationId(36), extent_id: 12, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496774514Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(36), extent_id: 12, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496798356Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:36 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496911227Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[37/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.49694071Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(37), op: ExtentClose { repair_id: ReconciliationId(37), extent_id: 12 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496965633Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.496986201Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497006571Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(37), op: ExtentClose { repair_id: ReconciliationId(37), extent_id: 12 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497030825Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(37), extent_id: 12 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497068891Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497105815Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(37), op: ExtentClose { repair_id: ReconciliationId(37), extent_id: 12 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497132624Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(37), extent_id: 12 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497154986Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497175339Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(37), op: ExtentClose { repair_id: ReconciliationId(37), extent_id: 12 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497199382Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(37), extent_id: 12 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497221732Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:37 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497501857Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[38/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497532223Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(38), op: ExtentRepair { repair_id: ReconciliationId(38), extent_id: 12, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497575123Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497616911Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497657465Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(38), op: ExtentRepair { repair_id: ReconciliationId(38), extent_id: 12, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497707434Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(38), extent_id: 12, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497749598Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(38)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497772408Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497792751Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(38), op: ExtentRepair { repair_id: ReconciliationId(38), extent_id: 12, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497819044Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(38), extent_id: 12, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497843847Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(38)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497864662Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.4978852Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(38), op: ExtentRepair { repair_id: ReconciliationId(38), extent_id: 12, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497912179Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(38), extent_id: 12, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497937563Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(38)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.497958338Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:38 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569417853Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[39/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569454683Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(39), op: ExtentReopen { repair_id: ReconciliationId(39), extent_id: 12 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569479609Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569506279Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569533168Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(39), op: ExtentReopen { repair_id: ReconciliationId(39), extent_id: 12 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569576704Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(39), extent_id: 12 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569624185Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569668463Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(39), op: ExtentReopen { repair_id: ReconciliationId(39), extent_id: 12 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569714063Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(39), extent_id: 12 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569758249Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569800294Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(39), op: ExtentReopen { repair_id: ReconciliationId(39), extent_id: 12 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569838818Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(39), extent_id: 12 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.569862936Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:39 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.57001673Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[40/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570046487Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(40), op: ExtentFlush { repair_id: ReconciliationId(40), extent_id: 7, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570072466Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570099401Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.57012097Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(40), op: ExtentFlush { repair_id: ReconciliationId(40), extent_id: 7, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570146747Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(40), extent_id: 7, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570170804Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570191569Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(40), op: ExtentFlush { repair_id: ReconciliationId(40), extent_id: 7, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, New, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570217163Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570238532Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(40), extent_id: 7, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570261256Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(40), op: ExtentFlush { repair_id: ReconciliationId(40), extent_id: 7, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570286863Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(40), extent_id: 7, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570310864Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:40 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570485183Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[41/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570514243Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(41), op: ExtentClose { repair_id: ReconciliationId(41), extent_id: 7 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570538461Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570564128Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570585433Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(41), op: ExtentClose { repair_id: ReconciliationId(41), extent_id: 7 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570609429Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(41), extent_id: 7 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570647461Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570689225Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(41), op: ExtentClose { repair_id: ReconciliationId(41), extent_id: 7 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570738483Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(41), extent_id: 7 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570785371Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570825356Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(41), op: ExtentClose { repair_id: ReconciliationId(41), extent_id: 7 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.570861238Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(41), extent_id: 7 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.57088438Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:41 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571017957Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[42/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571047665Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(42), op: ExtentRepair { repair_id: ReconciliationId(42), extent_id: 7, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571074223Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571100951Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571122342Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(42), op: ExtentRepair { repair_id: ReconciliationId(42), extent_id: 7, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571150455Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(42), extent_id: 7, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571195775Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(42)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571240054Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571283877Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(42), op: ExtentRepair { repair_id: ReconciliationId(42), extent_id: 7, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571335593Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(42), extent_id: 7, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571386078Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(42)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571427615Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571470155Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(42), op: ExtentRepair { repair_id: ReconciliationId(42), extent_id: 7, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571517147Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(42), extent_id: 7, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.57156161Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(42)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.571605158Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:42 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.643920103Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[43/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.643959778Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(43), op: ExtentReopen { repair_id: ReconciliationId(43), extent_id: 7 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.643985795Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644006602Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644027309Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(43), op: ExtentReopen { repair_id: ReconciliationId(43), extent_id: 7 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644051584Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(43), extent_id: 7 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644074731Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644094977Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(43), op: ExtentReopen { repair_id: ReconciliationId(43), extent_id: 7 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.64411922Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(43), extent_id: 7 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644142156Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644162304Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(43), op: ExtentReopen { repair_id: ReconciliationId(43), extent_id: 7 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644186484Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(43), extent_id: 7 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644208665Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:43 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.64455801Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[44/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644587651Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(44), op: ExtentFlush { repair_id: ReconciliationId(44), extent_id: 21, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644613605Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644634206Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.64465463Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(44), op: ExtentFlush { repair_id: ReconciliationId(44), extent_id: 21, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644680183Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(44), extent_id: 21, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644704265Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644724519Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644744693Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(44), op: ExtentFlush { repair_id: ReconciliationId(44), extent_id: 21, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644769968Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(44), extent_id: 21, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644793655Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(44), op: ExtentFlush { repair_id: ReconciliationId(44), extent_id: 21, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.64481926Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(44), extent_id: 21, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.644843179Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:44 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.645875057Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[45/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.645904674Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(45), op: ExtentClose { repair_id: ReconciliationId(45), extent_id: 21 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.645929227Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.64594969Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.645970126Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(45), op: ExtentClose { repair_id: ReconciliationId(45), extent_id: 21 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646000762Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(45), extent_id: 21 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646042556Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646085606Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(45), op: ExtentClose { repair_id: ReconciliationId(45), extent_id: 21 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646134563Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(45), extent_id: 21 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646181505Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646222533Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(45), op: ExtentClose { repair_id: ReconciliationId(45), extent_id: 21 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646255083Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(45), extent_id: 21 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646277854Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:45 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646411931Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[46/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646446223Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(46), op: ExtentRepair { repair_id: ReconciliationId(46), extent_id: 21, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646474333Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646495014Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.64651546Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(46), op: ExtentRepair { repair_id: ReconciliationId(46), extent_id: 21, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646542278Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(46), extent_id: 21, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646585102Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(46)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646628514Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646672103Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(46), op: ExtentRepair { repair_id: ReconciliationId(46), extent_id: 21, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.64672513Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(46), extent_id: 21, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646775715Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(46)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.64681825Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646861066Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(46), op: ExtentRepair { repair_id: ReconciliationId(46), extent_id: 21, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646916536Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(46), extent_id: 21, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.646963238Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Sending repair request ReconciliationId(46)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.647001153Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:46 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.72225206Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[47/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.72229106Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(47), op: ExtentReopen { repair_id: ReconciliationId(47), extent_id: 21 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722316664Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722420427Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722459947Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(47), op: ExtentReopen { repair_id: ReconciliationId(47), extent_id: 21 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722505622Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(47), extent_id: 21 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722550664Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722590158Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(47), op: ExtentReopen { repair_id: ReconciliationId(47), extent_id: 21 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722618646Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(47), extent_id: 21 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722641047Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722667169Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(47), op: ExtentReopen { repair_id: ReconciliationId(47), extent_id: 21 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722692466Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(47), extent_id: 21 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.7227145Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:47 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.72285547Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[48/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722887981Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(48), op: ExtentFlush { repair_id: ReconciliationId(48), extent_id: 20, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722914152Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722934585Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722962415Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(48), op: ExtentFlush { repair_id: ReconciliationId(48), extent_id: 20, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.722988669Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(48), extent_id: 20, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723013872Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723050735Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(48), op: ExtentFlush { repair_id: ReconciliationId(48), extent_id: 20, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, New, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723082464Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(48), extent_id: 20, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723106569Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723131589Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(48), op: ExtentFlush { repair_id: ReconciliationId(48), extent_id: 20, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723158118Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(48), extent_id: 20, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723181758Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:48 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723314734Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[49/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723342812Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(49), op: ExtentClose { repair_id: ReconciliationId(49), extent_id: 20 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723366796Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723387083Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723417554Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(49), op: ExtentClose { repair_id: ReconciliationId(49), extent_id: 20 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723461945Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(49), extent_id: 20 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.72350483Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.72354551Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(49), op: ExtentClose { repair_id: ReconciliationId(49), extent_id: 20 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723591229Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(49), extent_id: 20 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723621101Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723646661Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(49), op: ExtentClose { repair_id: ReconciliationId(49), extent_id: 20 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723695935Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(49), extent_id: 20 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723737667Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:49 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723885783Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[50/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723914553Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(50), op: ExtentRepair { repair_id: ReconciliationId(50), extent_id: 20, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723941056Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723961516Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.723981878Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(50), op: ExtentRepair { repair_id: ReconciliationId(50), extent_id: 20, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.724008387Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.724029597Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(50), extent_id: 20, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.724059305Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(50)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.724097793Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(50), op: ExtentRepair { repair_id: ReconciliationId(50), extent_id: 20, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.724148259Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(50), extent_id: 20, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.724204833Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(50)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.724247074Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.724287681Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(50), op: ExtentRepair { repair_id: ReconciliationId(50), extent_id: 20, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.724337082Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(50), extent_id: 20, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.724385244Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(50)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.72442182Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:50 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.796944102Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[51/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.796984699Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(51), op: ExtentReopen { repair_id: ReconciliationId(51), extent_id: 20 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797011113Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.79703203Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797052761Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(51), op: ExtentReopen { repair_id: ReconciliationId(51), extent_id: 20 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797076978Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(51), extent_id: 20 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797116643Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797149304Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(51), op: ExtentReopen { repair_id: ReconciliationId(51), extent_id: 20 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797189955Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(51), extent_id: 20 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.79723537Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797279142Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(51), op: ExtentReopen { repair_id: ReconciliationId(51), extent_id: 20 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797328912Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(51), extent_id: 20 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.79737159Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:51 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797469539Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[52/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797566343Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(52), op: ExtentFlush { repair_id: ReconciliationId(52), extent_id: 11, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797609236Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797632035Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797652648Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(52), op: ExtentFlush { repair_id: ReconciliationId(52), extent_id: 11, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797683389Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(52), extent_id: 11, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797726974Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797769549Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(52), op: ExtentFlush { repair_id: ReconciliationId(52), extent_id: 11, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797847829Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(52), extent_id: 11, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.79788429Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797905828Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(52), op: ExtentFlush { repair_id: ReconciliationId(52), extent_id: 11, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, Done, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797940938Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(52), extent_id: 11, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.797986955Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:52 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.798948912Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[53/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.798982022Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(53), op: ExtentClose { repair_id: ReconciliationId(53), extent_id: 11 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799006472Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799027013Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.79904716Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(53), op: ExtentClose { repair_id: ReconciliationId(53), extent_id: 11 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799071158Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(53), extent_id: 11 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799093523Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799113599Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(53), op: ExtentClose { repair_id: ReconciliationId(53), extent_id: 11 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799158613Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(53), extent_id: 11 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.79920402Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799246412Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(53), op: ExtentClose { repair_id: ReconciliationId(53), extent_id: 11 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799293534Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(53), extent_id: 11 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799327016Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:53 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799526397Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[54/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799554915Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(54), op: ExtentRepair { repair_id: ReconciliationId(54), extent_id: 11, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799581718Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799602003Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799622324Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(54), op: ExtentRepair { repair_id: ReconciliationId(54), extent_id: 11, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799648896Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(54), extent_id: 11, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799674138Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(54)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799694788Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799719953Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(54), op: ExtentRepair { repair_id: ReconciliationId(54), extent_id: 11, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.79974768Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(54), extent_id: 11, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799772826Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(54)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.79979348Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799813582Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(54), op: ExtentRepair { repair_id: ReconciliationId(54), extent_id: 11, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799859042Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(54), extent_id: 11, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799886413Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Sending repair request ReconciliationId(54)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.799907417Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:54 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874245852Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[55/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874284098Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(55), op: ExtentReopen { repair_id: ReconciliationId(55), extent_id: 11 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874309967Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874330595Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874350901Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(55), op: ExtentReopen { repair_id: ReconciliationId(55), extent_id: 11 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874397798Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(55), extent_id: 11 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874444073Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874487478Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(55), op: ExtentReopen { repair_id: ReconciliationId(55), extent_id: 11 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.87453342Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(55), extent_id: 11 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874575172Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874614829Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(55), op: ExtentReopen { repair_id: ReconciliationId(55), extent_id: 11 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874655801Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(55), extent_id: 11 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.874692065Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:55 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.87496926Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[56/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.87499899Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(56), op: ExtentFlush { repair_id: ReconciliationId(56), extent_id: 10, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875165984Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875195441Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875218533Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(56), op: ExtentFlush { repair_id: ReconciliationId(56), extent_id: 10, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875244376Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(56), extent_id: 10, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875273196Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875294442Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875314267Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(56), op: ExtentFlush { repair_id: ReconciliationId(56), extent_id: 10, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875339551Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(56), extent_id: 10, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875362951Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(56), op: ExtentFlush { repair_id: ReconciliationId(56), extent_id: 10, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875388191Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(56), extent_id: 10, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875411662Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:56 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875597221Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[57/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875624882Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(57), op: ExtentClose { repair_id: ReconciliationId(57), extent_id: 10 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875648797Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875669136Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875689204Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(57), op: ExtentClose { repair_id: ReconciliationId(57), extent_id: 10 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875717428Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(57), extent_id: 10 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.87575544Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875795633Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(57), op: ExtentClose { repair_id: ReconciliationId(57), extent_id: 10 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875903272Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(57), extent_id: 10 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875937779Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875958581Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(57), op: ExtentClose { repair_id: ReconciliationId(57), extent_id: 10 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.875982471Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(57), extent_id: 10 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876004305Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:57 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876094923Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[58/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876123147Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(58), op: ExtentRepair { repair_id: ReconciliationId(58), extent_id: 10, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876149586Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876170087Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876190494Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(58), op: ExtentRepair { repair_id: ReconciliationId(58), extent_id: 10, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876216793Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(58), extent_id: 10, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876261233Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(58)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876303725Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876345978Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(58), op: ExtentRepair { repair_id: ReconciliationId(58), extent_id: 10, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876396742Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(58), extent_id: 10, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876444911Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(58)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876485702Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876526904Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(58), op: ExtentRepair { repair_id: ReconciliationId(58), extent_id: 10, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.87657378Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(58), extent_id: 10, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876617463Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(58)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.876654041Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:58 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947014728Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[59/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947053213Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(59), op: ExtentReopen { repair_id: ReconciliationId(59), extent_id: 10 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947085377Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947107649Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947128574Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(59), op: ExtentReopen { repair_id: ReconciliationId(59), extent_id: 10 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947152855Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(59), extent_id: 10 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947175529Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947195783Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(59), op: ExtentReopen { repair_id: ReconciliationId(59), extent_id: 10 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947219705Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(59), extent_id: 10 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947242102Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947262279Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(59), op: ExtentReopen { repair_id: ReconciliationId(59), extent_id: 10 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.94728642Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(59), extent_id: 10 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947308887Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:59 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947617145Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[60/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947645463Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(60), op: ExtentFlush { repair_id: ReconciliationId(60), extent_id: 14, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947671073Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.94769558Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947716919Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(60), op: ExtentFlush { repair_id: ReconciliationId(60), extent_id: 14, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947742463Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(60), extent_id: 14, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947766473Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947786765Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(60), op: ExtentFlush { repair_id: ReconciliationId(60), extent_id: 14, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.94781195Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(60), extent_id: 14, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947878027Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947903083Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(60), op: ExtentFlush { repair_id: ReconciliationId(60), extent_id: 14, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, Done, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947928747Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(60), extent_id: 14, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.947952535Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:60 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948070427Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[61/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948098423Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(61), op: ExtentClose { repair_id: ReconciliationId(61), extent_id: 14 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948126873Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948148291Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948168831Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(61), op: ExtentClose { repair_id: ReconciliationId(61), extent_id: 14 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948192978Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(61), extent_id: 14 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948215685Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.94823595Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(61), op: ExtentClose { repair_id: ReconciliationId(61), extent_id: 14 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948260191Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(61), extent_id: 14 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948282484Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948302468Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(61), op: ExtentClose { repair_id: ReconciliationId(61), extent_id: 14 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948326562Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(61), extent_id: 14 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.94834851Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:61 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948666945Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[62/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948700607Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(62), op: ExtentRepair { repair_id: ReconciliationId(62), extent_id: 14, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948727277Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948759852Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.94879909Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(62), op: ExtentRepair { repair_id: ReconciliationId(62), extent_id: 14, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948847859Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(62), extent_id: 14, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948894133Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(62)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948936048Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.948972769Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(62), op: ExtentRepair { repair_id: ReconciliationId(62), extent_id: 14, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.949020578Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(62), extent_id: 14, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.949068371Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(62)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.949106672Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.949129739Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(62), op: ExtentRepair { repair_id: ReconciliationId(62), extent_id: 14, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.949156171Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(62), extent_id: 14, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.949180871Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] No action required ReconciliationId(62)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:28.949205822Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:62 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020094829Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[63/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020133869Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(63), op: ExtentReopen { repair_id: ReconciliationId(63), extent_id: 14 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020159398Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020180087Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020200278Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(63), op: ExtentReopen { repair_id: ReconciliationId(63), extent_id: 14 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020234603Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(63), extent_id: 14 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020277488Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.02032048Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(63), op: ExtentReopen { repair_id: ReconciliationId(63), extent_id: 14 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020369085Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(63), extent_id: 14 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020410756Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020450903Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(63), op: ExtentReopen { repair_id: ReconciliationId(63), extent_id: 14 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020497562Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(63), extent_id: 14 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020539619Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:63 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020741839Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[64/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020771311Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(64), op: ExtentFlush { repair_id: ReconciliationId(64), extent_id: 1, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.02079711Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020875132Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020906729Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(64), op: ExtentFlush { repair_id: ReconciliationId(64), extent_id: 1, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020933782Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentFlush { repair_id: ReconciliationId(64), extent_id: 1, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020957978Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.020978158Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(64), op: ExtentFlush { repair_id: ReconciliationId(64), extent_id: 1, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([New, InProgress, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.021003545Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentFlush { repair_id: ReconciliationId(64), extent_id: 1, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.021027245Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.021047343Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(64), op: ExtentFlush { repair_id: ReconciliationId(64), extent_id: 1, client_id: ClientId(0), flush_number: 352, gen_number: 199 }, state: ClientData([InProgress, Done, Done]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.021072462Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentFlush { repair_id: ReconciliationId(64), extent_id: 1, client_id: ClientId(0), flush_number: 352, gen_number: 199 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.021100851Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [0] id:64 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.02228182Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[65/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022312601Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(65), op: ExtentClose { repair_id: ReconciliationId(65), extent_id: 1 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022337196Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022357703Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.02237811Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(65), op: ExtentClose { repair_id: ReconciliationId(65), extent_id: 1 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022406863Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentClose { repair_id: ReconciliationId(65), extent_id: 1 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022447022Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022489054Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(65), op: ExtentClose { repair_id: ReconciliationId(65), extent_id: 1 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022536796Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentClose { repair_id: ReconciliationId(65), extent_id: 1 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.02258176Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022621615Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(65), op: ExtentClose { repair_id: ReconciliationId(65), extent_id: 1 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022650507Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentClose { repair_id: ReconciliationId(65), extent_id: 1 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022672885Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [2] id:65 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022878851Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[66/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022910842Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(66), op: ExtentRepair { repair_id: ReconciliationId(66), extent_id: 1, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022938039Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022958583Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.022978821Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(66), op: ExtentRepair { repair_id: ReconciliationId(66), extent_id: 1, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023005159Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentRepair { repair_id: ReconciliationId(66), extent_id: 1, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023035721Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Sending repair request ReconciliationId(66)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023073329Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023114292Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(66), op: ExtentRepair { repair_id: ReconciliationId(66), extent_id: 1, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023166553Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentRepair { repair_id: ReconciliationId(66), extent_id: 1, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023216098Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] No action required ReconciliationId(66)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023258096Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023298375Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(66), op: ExtentRepair { repair_id: ReconciliationId(66), extent_id: 1, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }, state: ClientData([Done, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023355585Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentRepair { repair_id: ReconciliationId(66), extent_id: 1, source_client_id: ClientId(0), source_repair_address: 0.0.0.0:12810, dest_clients: [ClientId(1), ClientId(2)] }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023401467Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Sending repair request ReconciliationId(66)","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.023438539Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:66 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.09757406Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[67/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.097620878Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Pop front: ReconcileIO { id: ReconciliationId(67), op: ExtentReopen { repair_id: ReconciliationId(67), extent_id: 1 }, state: ClientData([New, New, New]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.097668197Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Sent repair work, now wait for resp","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.097708253Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.097749676Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] rep_in_progress: return ReconcileIO { id: ReconciliationId(67), op: ExtentReopen { repair_id: ReconciliationId(67), extent_id: 1 }, state: ClientData([New, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.097795949Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] client ExtentReopen { repair_id: ReconciliationId(67), extent_id: 1 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.09784156Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.097879448Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] rep_in_progress: return ReconcileIO { id: ReconciliationId(67), op: ExtentReopen { repair_id: ReconciliationId(67), extent_id: 1 }, state: ClientData([InProgress, New, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.09792698Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] client ExtentReopen { repair_id: ReconciliationId(67), extent_id: 1 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.09797536Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098025164Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] rep_in_progress: return ReconcileIO { id: ReconciliationId(67), op: ExtentReopen { repair_id: ReconciliationId(67), extent_id: 1 }, state: ClientData([InProgress, InProgress, InProgress]) }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098073183Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"":"downstairs","session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] client ExtentReopen { repair_id: ReconciliationId(67), extent_id: 1 }","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098120176Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Completion from [1] id:67 status:true","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098169986Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[68/68] Repair commands completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098193967Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"17 extents repaired in 1.312 ave:0.0772","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098215039Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"All required repair work is completed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098235526Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Set Downstairs and Upstairs active after repairs","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098256014Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"f62d94ba-2d01-4e79-af70-fd9fb1c8eb45 is now active with session: c9a85fe0-8af2-47fe-a897-455ed079cea0","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098278363Z","hostname":"The guest has finished waiting for activation with:199 ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"Wait for a query_work_queue command to finish before sending IO } {"msg":"Notify all downstairs, region set compare is done.","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.09830716Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"Set check for repair","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098339421Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098441768Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] All repairs completed, exit","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098495879Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[0] Starts cmd_loop","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098534927Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098573468Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] received reconcile message","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098615839Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] All repairs completed, exit","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098854042Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[1] Starts cmd_loop","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.098893976Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] All repairs completed, exit","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.099009426Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"[2] Starts cmd_loop","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:29.099038055Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} Region: es:20 ec:30 bs:4096 ts:2457600 tb:600 max_io:256 or 1048576 Loading write count information from file "/tmp/test_repair_verify.data" Read and Verify all blocks (0..600 range:true) Wrote out file "/tmp/test_repair_verify.data" Verify test completed Wrote out file "/tmp/test_repair_verify.data" CLIENT: Tests done. All submitted work has been ACK'd ---------------------------------------------------------------- Crucible gen:199 GIO:true work queues: Upstairs:0 downstairs:6 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY 1 Acked 1000 Read 100 Done Done Done false 2 Acked 1001 Read 100 Done Done Done false 3 Acked 1002 Read 100 Done Done Done false 4 Acked 1003 Read 100 Done Done Done false 5 Acked 1004 Read 100 Done Done Done false 6 Acked 1005 Read 100 Done Sent Done false STATES DS:0 DS:1 DS:2 TOTAL New 0 0 0 0 Sent 0 1 0 1 Done 6 5 6 17 Skipped 0 0 0 0 Error 0 0 0 0 Last Flush: 0 0 0 Downstairs last five completed: Upstairs last five completed: 6 5 4 3 2 1 CLIENT: Up:0 ds:6 act:3 {"msg":"Checking if live repair is needed","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:30.099610717Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} {"msg":"No Live Repair required at this time","v":0,"name":"crucible","level":30,"time":"2023-12-10T18:59:30.09967696Z","hostname":"ip-10-150-1-104.us-west-2.compute.internal","pid":1958,"session_id":"c9a85fe0-8af2-47fe-a897-455ed079cea0"} ---------------------------------------------------------------- Crucible gen:199 GIO:true work queues: Upstairs:0 downstairs:0 Downstairs last five completed: 1006 1005 1004 1003 1002 1001 Upstairs last five completed: 7 6 5 4 3 2 CLIENT: Up:0 ds:0 act:3 CLIENT: All crucible jobs finished, exiting program