Following the Flow of Memories