Meena chatbot online11/21/2023 The following figure shows that correlation for different chatbots(blue dots). The actual mathematical formulation of the SSA metric is pretty sophisticated but the initial experiments conducted by Google showed a strong correlation with the human-likeness of a chatbot. However, if B responds, “Me too, I can’t get enough of Roger Federer!” then it is marked as “specific”, since it relates closely to what is being discussed. That reply could be used in dozens of different contexts. For instance, A says, “I love tennis,” and B responds, “That’s nice,” then the utterance should be marked, “not specific”. Specificity is the second metric that can help quantify human-likeness of a conversational interaction. Such responses are frequently generated by bots that are evaluated according to metrics like sensibleness alone. A generic response (ex: I don’t know) can be sensible, but it is also boring and unspecific. Sensibleness also captures other important aspects of a chatbot, such as consistency. Sensibleness arguably covers some of the most basic aspects of conversational human-likeness, such as common sense and logical coherence. Specifically, SSA tries to quantify two key aspects of human-conversations: Sensibleness and Specificity Average(SSA) is a new metric for open-domain chatbots that h captures basic, but important attributes for human conversation. How can we measure the human-likeness of a dialog? To address that challenge, Google started by introducing a new metric as the cornerstone of the Meena chatbot. That idea seems intuitive but also incredibly subjective. A key criterion to evaluate the quality of an open-domain chatbot is the fact that its dialogs feel natural to human. With Meena, Google ventures tries to address some of these challenges by building an open-domain chatbot that can chat about almost anything.īefore building Meena, Google had to solve a non-trivial challenge that is often ignored in open-domain chatbot systems. If effective, open-domain chatbots might be a key piece in the journey to humanize computer interactions.ĭespite the excitement around open-domain chatbots, the current implementation attempts still have weaknesses that prevent them from being generally useful: they often respond to open-ended input in ways that do not make sense, or with replies that are vague and generic. The alternative is an emerging area of research known as open-domain chatbots that focuses on building conversational agents that chat about virtually anything a user wants. In NLU theory, those specialized conversational agents are known as closed-domain chatbots. However, despite all the progress, most conversational systems remain highly constrained to a specific domain which contrasts with our ability as humans to naturally converse about different topics. ![]() NLU has been one of the most active areas of research of the last few years and have produced some of the most widely adopted AI systems to date. ![]() Just a few weeks into 2020, Google Research published a new paper introducing Meena, a new deep learning model that can power chatbots that can engage in conversations about any domain. Last year, the BERT model definitely stole the headlines of the NLU research space. ![]() It seems that every year Google plans to shock the artificial intelligence(AI) world with new astonishing progress in natural language understanding(NLU) systems.
0 Comments
Leave a Reply.AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |