A topic display did not help people resume short AI voice chats

Souffleur reduced an AI voice conversation to a few topic labels, decisions and open questions. Then 48 people tried to use it after an interruption.

They did not resume faster. They did not remember more. They did not report that the conversation was easier to follow than it was with an ordinary chat transcript.

That sounds like a failed interface. It is a more useful result than that.

A conversation, interrupted on purpose

Souffleur is a research prototype for AI voice chats on a phone. Instead of stacking every spoken turn into message bubbles, it turns the conversation into a compact topic display. The current subject gets the largest card. Earlier topics shrink into smaller items. Each one can show what was decided, whether something remains open and whether the idea came mostly from the person or the AI.

The premise is easy to recognize. A voice conversation is convenient while walking, cooking or doing anything that has already claimed your hands. The transcript waiting on the screen is less convenient: spoken language produces fragments, corrections and enough text to make “just glance at it” an ambitious reading strategy.

The researchers compared Souffleur with a conventional transcript. Each participant held one short planning or reflection conversation with the same voice model on their own phone. Between 2 minutes 15 seconds and 3 minutes 30 seconds into the conversation, a tic-tac-toe game took over the screen for 60 to 74 seconds. The microphone muted. The agent paused. When the game vanished, the participant had to continue.

This was not a vague usability impression. The study timed the first words after the interruption, checked whether those words continued the suspended topic, asked participants how easy the return felt, and scored what they could recall afterward.

The fancy screen got a draw

For the 39 participants included in the resumption analysis, the median delay was 2.9 seconds with the transcript and 3.3 seconds with Souffleur. That difference was not reliable. A blind model judge reading the transcripts found that 18 of 21 people in the chat condition and 15 of 18 with Souffleur continued the topic they had left. Nobody asked the agent to remind them where they were.

The rest of the measures were similarly uneventful. People rated both interfaces as easy to follow. Souffleur did not reduce reported effort or produce better recall. The same model judge scored summaries as capturing 27 percent of the conversation’s key points with the transcript and 22 percent with Souffleur, another difference the study could not distinguish from noise.

The compact display did accomplish something: it represented the conversation in far fewer words. Participants could understand it without training, and several specifically liked the topic boxes. Compression, however, is not the same as assistance. A tidier screen earns no points merely for looking as if it has already organized the meeting.

The test may have made memory too easy

The authors are unusually clear about what their null result cannot establish. With 24 people assigned to each interface, the experiment had enough statistical power to detect only large effects. “No reliable difference” does not mean the interfaces are equivalent.

More important, the study removed much of the difficulty that Souffleur was designed to solve. The conversations lasted only a few minutes and usually covered two topics. The interruption lasted about a minute. Participants had no second task competing for their attention before the game arrived, and they reported looking at the screen fairly often.

In other words, the suspended thought was still warm. Almost everyone resumed it regardless of the interface. The ordinary transcript was also a strong opponent because its last visible lines contained exactly what had been said before the interruption. Six transcript users said they reread those lines to get going again.

A compact topic view should have a better chance after a conversation has sprawled across ten subjects, or after lunch, a phone call and the discovery that somebody has used the last clean mug. None of those conditions was tested. The paper proposes longer conversations, harder interruptions and returns after hours or days as the next useful trial.

Make the feature face its actual problem

The tempting response to a null result is to enlarge the sample and run the same test again. That might reveal a small effect, but it would not answer the more interesting design question. If a feature exists to rescue a person when memory is struggling, the evaluation needs to create a situation in which memory might struggle.

Souffleur did not prove that topic displays are useless. It showed that a familiar transcript is already good enough for resuming a short, recent conversation after one minute of tic-tac-toe. That is a narrower conclusion, and a better one.

Good experiments do not owe their prototypes a victory. Sometimes the useful finding is that the problem has not become difficult yet.