Step 1. Corpus of incorrect response Q&A: AI creates a large corpus of short questions and answers from actual prompts and responses, and from training data, targeting questions that were answered with RAG and which would be considered wrong without the RAG information.
Note that step 1 is a long
process that includes creating a categorizing model for detecting the fitting q&a pairs, and human evaluation of the results.
Many of the actual prompts and responses may stem from long questions or answers, and may contain spelling mistakes, or missing information, this is fixed in the corpus.
Step 2. AI determines which categorization the layer sections were doing. It is similar to detecting and understanding brain layers and regions.
Step 3. AI points to weights that may be changed according to new information.
Step 4. Test the corrections.
Step 5. Finalize or fall back if the change does not work.
Step 6. Keep records for fallback to stable versions.