NVIDIA's AI chatbot expands support for Google's Gemma, voice queries
NVIDIA is advancing its experimental ChatRTX chatbot by integrating a broader array of AI models for RTX GPU users. Initially designed to utilize Mistral or Llama 2 for sifting through personal documents, the chatbot now supports Google's Gemma, ChatGLM3, and OpenAI's CLIP model. This enhancement simplifies photo searches and was first announced in February as "Chat with RTX."
ChatRTX's requirements and functionality
The demo app "Chat with RTX" requires an RTX 30/40-series GPU with a minimum of 8GB VRAM to function. It operates by setting up a local chatbot server accessible via a browser. Users can input their local documents and also YouTube videos into the server, transforming it into a robust search tool that provides summaries and answers questions based on their personal data.
Google's Gemma integration
The integration of Google's Gemma model into ChatRTX is particularly significant. The app, designed to run directly on high-performance laptops or desktop PCs, simplifies the process of running these models locally. This results in a chatbot interface that offers users the flexibility, to choose between models based on the type of data they wish to analyze or search.
ChatRTX now supports a bilingual language model
Available for download from NVIDIA's website as a 36GB file, ChatRTX now also supports ChatGLM3, which is an open bilingual (English and Chinese) large language model. Furthermore, OpenAI's Contrastive Language-Image Pre-training (CLIP) has been incorporated into the system. This feature allows users to search as well as interact with local photo data, effectively training the model to recognize images.
ChatRTX is equipped with voice query support
In its most recent update, NVIDIA has enhanced ChatRTX with voice query support by integrating Whisper, an AI-based speech recognition system. This new feature allows users to search their data using their voice, adding a new level of convenience and functionality to the chatbot. The integration of Whisper signifies a significant step toward making the chatbot more user-friendly and versatile.
