Ted Chai
San Francisco, CA
Augmenting Memory
May 2024

Almost all important work requires a lot of human interaction. Building a great product requires talking to a lot of customers. Closing a big sale requires a lot of pitching and negotiation. Important work is shaped by important conversations. 

Given that important conversations contribute to important work, important conversations should be remembered. Professions dealing with important conversations have always had some sort of system to ensure the information is stored in a more robust system than human memory. Doctor’s appointments are recorded by a medical scribe, and organized in an EMR (Electronic Medical Record). Salespeople log their communication, which is organized in a CRM (Customer Relationship Management). Court proceedings are recorded by a stenographer, and organized in a CMS (Case Management System).

The trend here is that communication is first documented, then organized. Documentation can be completely unfiltered, like the court stenographer transcribing every word, or more processed, like the medical scribe jotting down only relevant symptoms and prescriptions. Over time, the quantity of conversations documented becomes inevitably large, so an organization system needs to be implemented to make the information more digestible. All of this helps the doctor, the salesperson, and the lawyer do better work.

However, the average professional does not have a scribe writing down each word they say. Our important conversations are remembered through scribbled jot dots and scattered across emails and a bunch of Google Docs. We try our best, given our limited bandwidth, to remember important conversations so we can do good work.

The literal function of a conversation intelligence product is simple: document and organize communication. In doing so, we can give people a perfect memory of all their most important conversations. Even more, we can give their coworkers a perfect memory of conversations they may have not even been a part of. If our organization system can make recollection feel effortless, then teams will be able to make more informed decisions and do better work.

Documentation
Most modern communication is automatically documented. Emails, documents, and messages can be revisited at any time. Conversation Intelligence products are documenting the last undocumented medium of communication: conversations. 

The form of conversation documentation that people are most familiar with is note taking. People have a conversation, and either a human or an AI jots down the important points. The fundamental challenge that the note taker is tasked with is managing the balance between fidelity and brevity. If the notes are too long, you might as well just read the transcript. If they’re too short, you miss out on important information. Good notes should enable the reader to understand the high-level quickly, but also dive deeper into the details if needed. Plaintext notes often attempt to accomplish this by using a nested jot dot structure, where:

• The primary jot dot contains the most important information and is read first.
     ○ The secondary jot dot provides more information, but can easily be skipped over if not considered important
          ⁃ The tertiary jot dot elaborates even more and is not read unless the reader really cares about this point. 

This is a decent solution, but the plaintext format will always be more limited than a software interface where the reader can skim or investigate at their discretion. With software, the reader can click on the part of the summary they want to know more about and be taken to the exact section in the transcript. If they want to get a sense of the tone of voice, they can press play and hear the audio recording of that sentence. If they have a question, they can get the answer from an AI chatbot. Note taking software will not just automate manual note taking, it will be vastly more digestible and useful. 

Yet, anyone who has used an AI note taker knows that more often than not, it will produce worse notes than a well informed human. Model performance is a factor, but primarily this is because the AI lacks the context that the human note taker would have. The AI doesn’t understand if your meeting is a customer interview, an earnings call, or a performance review. It doesn’t know what you’re talking about when you use the phrase “the proposal” even though you do. Once AI is able to ingest all the context across your communication history and workspace, it will outperform even the most experienced note taker. 

Organization 
The average American professional has 62 meetings a month, for a total of 744 per year. Without a good organization mechanism, even the best documentation can not save important conversations from being forgotten under a mountain of newer communication. In effect, conversation data has an expiration date. For conversation data to be durably useful, we need to equip the user with the tools to surface relevant information out of the noise.

If you’re looking back at a documented conversation, you’re likely either looking for an answer to a specific question (what deadline did we agree on?) or looking to broadly get up to speed (what did we talk about again?). For the former, there’s no reason why you should sift through conversation data manually, instead of having an LLM do it for you. For the latter, AI enables you to get caught up on an hour-long call in 2 minutes. 

But, what happens when it’s been a while since the call and your memory gets a bit more fuzzy? When we think back to our work from a few months back, we don’t remember specific meetings, but we remember the progression of the entire project. Individual conversations are only relevant as data points for the overall context—the deal, the project, the human relationship. The salesperson does not remember every pitch but likely has a good mental model for what types of pitches work well and which don’t. The product manager doesn’t remember every customer interview, but likely has a good mental model for what features the customers cared most about. 

Unfortunately, this mental model not only has an expiration date, it’s also not transferable. Unless your colleagues sat in on every call you participated in, they’ll just have to take your word on it when you fill them in on your mental model. Their range of understanding will always be a subset of yours. 

Just like note taking, this is a challenge of balancing brevity and fidelity. Your colleague could read the notes of all your communication over the past year, but that would take days. Alternatively you could summarize all the key points into a one page primer, but that would leave out a lot of relevant information. Again, we must give the colleague the ability to skim or investigate at their discretion, to meander through the data that makes up your mental model. 

The context that our colleague would need to sort through is unfortunately all over the place. Communication encompasses not only meetings, but also emails, Slack messages, proposals, and written notes, all scattered across different platforms. As a result, meandering through the data is clumsy and impractical. 

Sales teams have solved this problem well, with the CRM serving as the single source of truth for all relevant communication. However, it takes an engineering team to set up a robust CRM and a RevOps team to maintain it. This cost can be justified for sales teams because good organization has a direct impact on revenue, but teams with lower impact, albeit important conversations, are out of luck. 

Conversation Intelligence will centralize the communication across these scattered mediums into an organized, digestible workspace. With this, your colleagues can easily know what you know. Even more, every team member can know what the entire team knows. Aggregated conversation data becomes a single source of truth. We need not use our imperfect and opinionated mental models to make decisions when we have access to objective truth. What features are customers asking for the most? Just ask your AI. 

Knowledge management is just the beginning. The source of truth can be used to automate existing workflows, like task management or follow up emails. Managers can leverage the data for reporting and coaching. Individuals can use it as a personal CRM. Conversation Intelligence will unlock better memory, better decision making, better collaboration, and better work.

Linkedin
ted@recall.ai