Sharp's AI tech enhances UX through conversations with household devices
Discover CE-LLM: Sharp's AI tech for enhancing user experience through natural conversations with household devices.
When we talk about AI (Artificial Intelligence), many of you may be reminded of ChatGPT released by OpenAI, which became a worldwide sensation last year. Now it is possible to obtain highly accurate answers in a natural conversational style, summarise sentences and take minutes, and easily produce output that is on a different level from what was previously possible. We believe that more and more people are using ChatGPT in their work.
Sharp has been promoting AI development focusing on the proprietary edge AI technology, CE-LLM. Showcasing the AI Partner and AI Avatar, both powered by CE-LLM, at the highly acclaimed SHARP Tech-Day event last November. The AI Avatar also made a notable appearance at CES 2024, a renowned technology trade show. These innovative AI solutions garnered attention for their ability to facilitate seamless and authentic conversations.
＊CE-LLM is a registered trademark of Sharp Corporation.
In this blog, Sharp Japan interviewed Mr. Goto (pictured below), in charge of development, to explore the lifestyle enhancements and innovative technology that CE-LLM brings to the table.
What can we do with the CE-LLM edge AI technology?
When you ask a question to a terminal equipped with CE-LLM, the pause during the conversation is reduced, and answers in a smooth, fast, and natural way. Furthermore, problem solving support is also available. I will explain this by using the AI Partner and AI Avatar exhibited at SHARP Tech-Day as examples.
For the AI Partner, I will use Sharp’s wearable neck speaker “Sound Partner” with CE-LLM that was exhibited as an example. By registering family members’ schedules in advance, when asking the Sound Partner “I want to make a reservation at a restaurant for the weekend, so tell me everyone’s schedule,” the Sound Partner promptly replies when everyone’s time is free. Through such conversations, you can coordinate schedules, and restaurant reservations can also be supported by linking it with smartphones and other devices.
In other cases, home appliances can be controlled by consulting menu and downloading the suggested menus using seasonal vegetables for example to “Healsio Hotcook” Anhydrous Automatic Cooking Pot.
Also, after consulting shopping, after the conversation ends, by linking PCs, smartphones and tablets, the speaker can be guided to an E-commerce site that offers recommended products.
In this way, it is possible to respond to various services hands-free based on conversation, and even if you do not know how to use a home appliance, you can solve the problem in a short period of time via AI.
The other AI Avatar is a combination of a computer graphics (CG) model and a see-through display. A virtual explainer named “Sara” answered various questions with voice and 3D (stereoscopic) animations of body gestures. At SHARP Tech-Day and CES2024, Sara served as a guide to the highlights of the exhibition booths and other events.
It can even support conversational responses and problem solving, just like talking to a person. Please explain about edge AI technology CE-LLM. First of all, what does “LLM” mean?
LLM stands for Large Language Models, which is an AI built by training large amount of text data to generate textual answers to users’ questions. Famous examples include ChatGPT and recently Google Gemini. LLM itself generates text, but with extensions, it can also generate images.
So Chat GPT is also LLM. What is “edge AI”?
ChatGPT, for example, is used via cloud, but “edge AI” is AI that runs on edge devices (PCs, smartphones, etc.) operated by the users. In other words, it runs within the device at hand, and not via cloud. Since it does not require access to the Internet, it can respond more quickly than ChatGPT, which requires communication to the cloud, and it also has the advantage in security wise, with no risk of personal information leakage.
What is edge AI technology CE-LLM ?
Simply put, AI determines whether the user’s question is simple or not, and if the question is simple, the local LLM inside the edge device quickly answers it. If it is a difficult question, it hands it to cloud AI such as ChatGPT to provide complex answers.
Thus, the CE-LLM edge AI technology is a proprietary technology that provides optimal, smooth, and natural conversational interactions by immediately determining and switching between cloud AI such as ChatGPT or local LLM edge AI to process questions from users.
As explained earlier, while cloud AI has a large amount of data and high accuracy, it has response time and security issues. Edge AI, on the other hand, can respond immediately to a simple conversation. Nowadays, edge AI responds in less than one second, whereas it may take five to six seconds via cloud. This is not a smooth conversation. The concept is to use either the cloud or the local system, depending on the content of the question, and to take the best of both worlds. The technology to instantly determine this is the core of CE-LLM.
Switching between edge and cloud is interesting. Where did the idea come from?
In fact, we are referring to the brain mechanism of living organisms. The brain has two parts: the cerebrum and the limbic system. The cerebrum is the social brain that controls reasoning. The limbic system is the part of the brain that controls emotions and reflexes, and instincts related to survival (i.e., the survival brain). By using reasoning and instincts differently, humans are able to reflect (act) quickly when their lives are in danger, and in other times, we can think carefully. I thought that if we replaced the cerebrum with cloud AI and the limbic system with edge AI, and used them differently depending on the question, we could respond fast and smart.
Why did you decide to proceed with the development of CE-LLM?
Various AI-based services had already been developed, but they were basically based on the use of cloud AI, and we thought with edge AI, we could provide products and services that had never existed before through home appliances, which is Sharp’s specialty, and began development of CE-LLM.
With this Sharp proprietary CE-LLM at its core, we hope to realize “more user-friendly, people-oriented AI” by highly combining various devices in the home, including home appliances, and various internet services. Currently, we are in the process of developing this system for practical use, and in the future, we would like to incorporate CE-LLM not only in PCs and smartphones, but also in home appliances themselves.
SHARP Tech-Day attracted a lot of attention including the media. Please tell us about the insights you gained from the event.
The AI Avatar was set up with the expectation that questions about the exhibit contents would be asked. However, some visitors asked questions that we had not anticipated, such as “What’s the weather today? What time is it? Where am I?” which could not be answered very well.
A conversation using LLM is made by posing the user’s question to the LLM, which then responds to the user with the information fed back from the LLM. In other words, we are only selecting statements that seem to have a high degree of certainty to the question, not interpreting the meaning in detail. It is necessary to optimize the LLM based on the information and knowledge the customer has, such as from what position the customer is asking the question and the type of answer he or she is seeking, and to further refine the assumptions, characterization, and human-like responses. At CES 2024, we designed the avatar as an explanatory staff member who is knowledgeable about TV AQUOS, but many people asked her out for dinner, which was unexpected.
What are your future plans and personal thoughts?
Characterization is also an issue, but above all, the developed product we exhibited still requires a large terminal, and further miniaturization is essential for practical use. The entire team will work to develop a compact terminal from both the software and hardware perspectives so that a full-fledged service can be deployed quickly.
Personally, I also believe that in the future, CE-LLM could eliminate the need for a remote control. I’m thinking of a world where you talk to an air conditioner equipped with CE-LLM saying “It’s hot today,” and the air conditioner will adjust to an optimum temperature. If the remote control for home appliances becomes unnecessary, there will be no need to search for it, and the absence of a remote control will reduce the amount of materials used, saving resources. I will do my best to make it happen.
*Blog originally published by Sharp Japan: https://global.sharp/blog/2024/03/28/20617/