Vector Storage Based Long-term Memory Research on LLM

The memory storage component caches task-related context information during the execution of task instructions by the LLM, which is categorized into three different types of memory banks: knowledge memory, dialog memory, and task-related memory. As shown in Figure 2, the knowledge memory mainly captures and stores task-related data and documents that need to be not trained or outdated by external search engines or LLM pre-training; the dialog memory records the query and response pairs in each round of dialogs between the LLM and the user to ensure the context coherence and consistency of the subsequent dialogs, and the task memory records the task decision-making process of the LLM. After each task instruction is given to the LLM, the LLM performs strategic planning through task planning, tool selection, and observations obtained from the tool, and task-related information is systematically stored in memory for effective utilization in subsequent dialogues. In order to improve the validity of historical information during the process of storing LLM interaction information in the memory bank, an evaluation strategy is introduced to ensure that all historical experiences memorized by the LLM are valid, and at the same time, Prompt is pre-set to activate the ability of the intelligent body to recognize positive and negative sample experiences, to ensure that the information memorized by the intelligent body is correct as much as possible. Specifically, when memorizing conversational experiences, positive samples refer to real multi-round conversations in which each round is logically interrelated. On the contrary, negative samples refer to pseudo-multi-round conversations, in which there is no logical correlation between the conversation history and the latest task query, and the intelligentsia does not need to refer to previous conversational experiences when answering the query. When constructing knowledge-based experiences, knowledge experiences need to synthesize possible conflicting, irrelevant and relevant information in the knowledge. Therefore, retrieved knowledge experiences are defined into three categories: relevant, irrelevant and conflicting information. Relevant knowledge refers to experiences from which the answer to a query can be found directly. For example, if a query is made about someone’s age and the retrieved knowledge experience contains personal information about the person and his/her age, this part of the knowledge is relevant knowledge. Irrelevant knowledge experiences are those that are related to the task at hand but do not provide a direct answer. Conflicting type of knowledge refers to the existence of two contradictory historical experiences about the same task information in the knowledge experience base, for example, the retrieved knowledge contains two different ages of the same person, which the intelligent should be able to distinguish. The memory of task types is similar to external knowledge memory, which is also likely to have three types of memory: relevant, irrelevant, and conflicting, and the LLM’s ability to utilize complex historical experience information to better adapt to different tasks is enhanced by this memory recognition method.

Figure 2.

Different types of memory

For each type of memorized information text, the text is divided into fixed-length segments. These segments are converted into vector representations on the basis of a vector database for subsequent efficient vector-based retrieval using FAISS. The specific formalization is:

First a piece of text is sliced and divided into n segments T = {S₁, S₂,…, S_N} each of which is of length Each segment is of length n. For each text fragment, this research uses CoIBERT [10] as the encoder model to pre-code it into a vector representation, and its vector representation can be expressed in Equation 1: 1vi=E(Si)={ vi0,vi1,…,vim },vi∈ℝd

Where E is the function used to convert text segment into vectors, and v_i is the first i vector representation of the text fragment, i ∈ (1, m). Store all the fragment vectors in a shared vector database for subsequent retrieval.

The vector database as the basis of the whole VIMBank is mapped to the vectorized representation of each text using vector indexing, which is able to capture the semantic information of the text by mapping the text to a high-dimensional vector space, but the vector indexing approach is not able to support fast keyword matching, and when the LLM recalls the long-term memory information, on the one hand, the semantic information is very important, and on the other hand, the precision against the keyword matching to the historical memory information should also be captured. As a result, VIMBank needs to use Elasticsearch technology to map text segments and store them as sparse vectors, create an index for each keyword, and record all documents containing the keyword to realize fast processing and precise matching of large amounts of text data.

By mixing the two approaches, vector databases map a large number of text segments as dense vectors, and Elasticsearch stores text maps as sparse vectors, and in the subsequent retrieval phase, vector indexes are good at semantic understanding, while Elasticsearch-based backward indexes are good at fast and accurate keyword matching, while utilizing the advantages of both to obtain more comprehensive and accurate retrieval results.

The quality of the relevance of information retrieved from long-term memory can have a critical impact on LLM reasoning and decisionmaking in the process of LLM executing reasoning and decision-making, so the retrieval mechanism is a crucial stage in long-term memory.

Based on text memory vectorization and keyword storage, VIMBank adopts FAISS [11] retrieval method for densely embedded vectors and BM25 retrieval model for sparse vectors [12] for these two storage methods respectively. The core idea of FAISS is to generalize, partition, or quantize high-dimensional vectors so as to reduce the search space and improve the retrieval speed, which can be achieved through a combination of various indexing techniques such as inverted files, product quantization, HNSW, and GPU acceleration to achieve efficient retrieval on large-scale datasets. And keyword storage uses the Elasticsearch search engine to represent these text segments as sparse vectors through inverted indexing, of which the BM25 retrieval model is the most commonly used inverted indexing-based retrieval model, which not only considers word frequency and inverse document frequency, but also introduces factors such as lexical item saturation, which better balances the impact of high-frequency words. VIMBank improves the information retrieval speed by integrating the vector retrieval and the keyword retrieval results to improve the overall effectiveness of information retrieval. This process involves merging and de-duplicating the results from the two retrieval strategies. With these two retrieval techniques, LLM is able to achieve more comprehensive and accurate information recall.

On the other hand, in order to ensure the recall of information related to LLM retrieval and the current task, VIMBank designs different retrieval strategies based on three different types of memory information, namely external knowledge, dialog, and task. Each memory type has its own adapted retrieval mechanism during each round of interaction in LLM.

For knowledge experience, this research wants to increase the number of recall results to access more relevant information. In a vector database, documents and instructions are represented as highdimensional vectors. Vector databases enable semantic similarity search by storing and retrieving these high-level vectors. The vector similarity of different texts can be defined by approximate nearest search method using cosine similarity distance function, which is implemented as: 2dist(q,d)=q⋅d∥q∥∥d∥

In Equation 2, q is the query vector, d is the document vector, || q || is the parameter of vector q. and || d || is the parameter of vector d. Suppose that q – [q₁,q₂,…,q_n] and d = [d₁,d₂,…,d_n], then the distance function of cosine similarity is specified as: 3dist(q,d)=∑i=1nqidi∑i=1nqi2·∑i=1ndi2

Cosine similarity can measure whether two vectors are in the same direction or not without considering the size of the two vectors, this method focuses more on the relative importance of the words and can maximize the retrieval of relevant document information for similar words. In contrast, knowledge-experience ES retrieval can increase the number of returned results by tuning the retrieval parameters.

For conversational experience, ensure recall of conversation rounds that are relevant to the query context. The main performance is semantically identical and logically coherent. Therefore, for conversational memory retrieval focuses more on pre-temporal weighting and content semantic similarity. ES retrieval introduces temporal weighting, while semantics can be realized directly based on vectorized retrieval, and the quality of retrieval recall for conversational information needs to consider both kinds of retrieval results comprehensively: 4Scall =ω⋅α+(1−ω)⋅s

In Equation 4, α and s are ES scores and similarity scores, ω is the time weights, assuming that the current time is T and the timestamp of the dialog round is t, and the timestamp of the conversation round as: 5ω=e−λ(T−t)

For task memory, the focus is on recalling recent task-relevant information with increased temporal weighting. The retrieval strategy is similar to conversational memorization. For example, for the query "What year was Xi'an University of Technology founded?", the information retrieved from the memory bank with different memory types is shown in Figure 3. The background color marked in gray indicates the part of the retrieval process related to the query, while the text marked in red indicates the information related to the text and the answer to the query. As the dialog progresses, the session memory and task memory are continuously updated to ensure that the LLM can effectively handle dynamically changing dialog situations and task requirements. This design not only enhances the flexibility of the LLM, but also improves the efficiency and accuracy of its application in practical tasks.

Figure 3.

Retrieval memory

Through the memory storage and retrieval mechanism, LLM can break through the limitations of the context window and obtain contextually coherent or relevant task information from longterm memory, and the LLM memory capacity is greatly enhanced. However, as time goes by, there may be a large amount of redundant information in the LLM long-term memory, resulting in too much data in the long-term memory, which will greatly affect the retrieval efficiency of the entire VIMBank mechanism, and may even cause LLM decision errors due to outdated and inaccurate information. Therefore, in order to ensure the correctness and efficiency of information in longterm memory, a memory update mechanism needs to be designed.

To address the above two problems, this research was inspired by the forgetting curve theory proposed by Ebbinghaus, and designed the human brain’s law of forgetting new things in the updating strategy of LLM long-term memory. According to the forgetting curve theory, clearing those secondary memory segments that occurred long ago and have not been frequently recalled can avoid the LLM memory module from occupying too much memory, and also avoid the influence of old and outdated information on the LLM decisionmaking. The LLM long-term memory updating strategy is mainly guided by the following principles: (1) memory forgetting; (2) speed of forgetting; and (3) review effect. Memory forgetting aims to simulate the decline of human memory over time, forgetting speed aims to reflect the rate of forgetting of different frequency and importance of information over time, and the review effect aims to express that the steepness of the forgetting curve can be effectively slowed down when the learning content is regularly reviewed or memorized over and over again to enhance the durability of memory.

The Ebbinghaus forgetting curve can be described by an exponential decay model: 6R(t)=R0⋅e−tλ

In Equation 6, R(t) denotes the memory retention rate at time t, that is, the proportion of information retained, t denotes the time that has elapsed since the information was learned, e is approximately equal to 2.71828. λ denotes the parameter of forgetting rate, which controls the speed of memory decline and is affected by factors such as learning depth and number of repetitions. In order to simplify the process of memory updating. The » modeled as a discrete value and initialized to 1 at the time of the first memory. when a memory segment is retrieved by the LLM decision-making process will increase the forgetting rate parameter of the segment pair by 1, and will reset to 0, which makes the segment less likely to be forgotten in the future and thus retained in memory for a longer period of time.

In summary, VIMBank builds a more comprehensive LLM long-term memory mechanism through the entirety of these key components. This mechanism improves the decision quality of LLM while reducing the reasoning cost, providing new possibilities for LLM applications.

The experimental environment is shown in Table 1.

In order to verify the effectiveness of the longterm memory mechanism of VIMBank, this research conducted experiments on three different task sets, ALFWorld, HotpotQA, and KAgentBench.

The ALFWorld dataset combines task execution in virtual environments and natural language instruction comprehension, and contains about 8,000 test tasks covering a wide range of scenarios and complex tasks. HotpotQA is a dataset for studying and evaluating complex questionanswering tasks, especially multi-step reasoning tasks, and contains 7405 test samples and is commonly used as a benchmarking dataset for testing models on complex problems. KAgentBench is a benchmarking tool for evaluating the capabilities of knowledge-based intelligences, which contains multiple tasks that cover different knowledge-based reasoning capabilities, decision-making capabilities, as well as natural language understanding and generation capabilities. The above three textual task sets are shown in Figure 4:

Figure 4.

Examples from different task sets

In order to assess the effectiveness of VIMBank, the evaluation follows the principle of "Unity of knowledge and action", which integrates planning decisions and corresponding actions at each step.7Splan =1M∑j=1MmaxEM(Tn,i,Tn,j′)⋅Γ(Th,i,Th,j′)(1≤i≤N) 8Saction =1M∑j=1MmaxEM(Tn,i,Tn,j′)·∑k=1KiEM(ak,i,ak,j′)⋅Γ(vk,i,vk,j′)

In Equation 7 and Equation 8, M is the number of complex tasks decomposed into subtasks by the LLM and N is the number of real decisions made by the LLM. T_n,i is the number of i true decision, and T′n,j is the prediction result of the j-th subtask. Γ is the ROUGE-L evaluation metric function to measure the similarity between the decision results and the true results. Specifically, ROUGE-L is based on the principle of the longest common subsequence, and measures the recall and precision of the decision results through F1-Score comprehensively.9R=LCS(Tn,i,Tn,j′)| Tn,i | 10P=LCS(Tn,i,Tn,j′)| Tn,j′ | 11F1=2×R×PR+P

The calculation based on the longest common subsequence can well capture the order information between sequences and is suitable for measuring the similarity between the generated sequences and the desired results in the decision-making process.

EM (Exact Match) is another evaluation metric to assess the exact match between the generated text and the reference text. When evaluated using EM, it tends to 1 if the generated text (R) and the reference text (G) are basically the same, and tends to 0 if most of the text is different from the reference text, which can be expressed as: 12EM(R,G)={ 1, if R≈G0, if R≠G

In order to comprehensively assess the performance of LLM in decision making and execution, a penalty factor is introduced p ∈ {0, 1}, the comprehensive assessment formula is as follows: 13Stotal =(1−p)⋅(0.7⋅Splan +0.3⋅Saction )

In order to evaluate the effectiveness of VIMBank in improving the decision quality and reducing the reasoning cost on multi-tasks, this research replicates the performance of some intelligences on task datasets based on some open source LLMs and compares VIMBank with the replication results. The performance of various LLMs on task reasoning decisions is shown in Table 2.

The experimental results show that compared to ReAct and InterAct, which incorporate short-term memory, the LLM with the introduction of VIMBank's long-term memory mechanism improves the quality of decision making on different tasks, which is still a gap compared to the original paper that uses a closed-source LLM such as ChatGPT, which may be due to the overall poorer capability of the open-source models compared to ChatGPT (Qwen2-7b and ChatGLM) are less capable overall. In order to analyze the effectiveness of long-term memory in enhancing LLM decision-making, experiment observed the ALFWorld task environment when LLM performs the pick 2 task during which LLM needs to find two identical items, such as "find two books and put them in bookshelf. "Since only short-term memory can forget the previous position due to the limitation of the context window, which leads to task failure, especially in the case of performing the same task in the same environment, it is more likely to cause inefficiency in task execution, and the advantage of long-term memory mechanism in the similar task environment comes to the forefront. token test on different LLMs, with NoAgent as the benchmark, this research recorded the token consumption of LLMs with the process of multiple rounds of tasks, which effectively proved the effectiveness of VIMBank in reducing the cost of LLMs' reasoning for multiple rounds of tasks, as shown in Table 3.

In addition, this research observes the reasoning trajectories of different intelligences in the ALFWorld environment as shown in Fig. 3, which is analyzed to show that the introduction of the long-term memory mechanism can guide the new reasoning steps through previous experiences, thus avoiding repeated reasoning and improving the retrieval efficiency to a large extent.