If you go to Settings -> Personalisation -> Memory, you have two separate toggles for "Reference saved memories" and "Reference chat history".

The first one refers to the "memory updated" pop-up and its bespoke list of memories; the second one likely refers to some RAG systems for ChatGPT to get relevant snippets of previous conversations.