Mentorclass Insights- Conversational AI (GPT) and its impact on your business

Key Highlights from the Session

How does GPT work?

• Generative Pre-trained Transformer (GPT) models are designed to process sequential data, such as text, and generate predictions about what comes next in the sequence. So basis the Nth word in a sequence, the transformer generates the N+1st word with a certain probability. It then proceeds to generate the N+2nd word from the N+1st word and so on.
• To a human, this predictive algorithm makes complete sense. However, the model itself has no semantic knowledge of what it says.
• The training process for GPT models involves exposing the model to large amounts of text data, and using the transformer architecture to learn patterns and relationships within that data. Once trained, the model can generate new text based on the patterns it has learned.
• There is a background statistical context behind each input, which is usually hidden and as a result, the output might be very different, even if the input itself is the same.
• In addition, there is a Reinforcement Learning with Human Feedback (RLHF) hybrid framework that coupled with reward training, makes GPT synthesize natural-sounding output.

Is GPT as good as Humans already? Can it replace them?

• Machines are typically not expected to fail. Hence permission for errors for machines is very less in comparison to human beings. For example, you might never sit again in an AI-driven car that had a small accident, but you might sit again in it, if the same car was driven by your friend or a cab-driver

• Humans value the journey and context behind a solution, and not just the destination. For most humans, the history and narrative behind a solution are quite important. That is why maybe a Rembrandt is much more valuable than a camera click

• Humans have a propensity to anthropomorphize things, that is attribute them to human characteristics. As a result, we look at AI from a more ‘human’ lens and thereby mimicking human-like capabilities.

• LLMs are usually much more successful where there is an ambiguity in user query and where the output can differ qualitatively based on randomness, uncertainty and if the multiple answers are ok.

• Currently GPTs are a couple of decades away from replicating a fully-functional sentient real-world model.

Can google searches be replaced by conversational AIs?

• Microsoft wasn’t very successful with its Bing search engine and it is investing in Open AI in order to divert some of Google’s search business on its conversational AI model

• Google is not far behind and in a matter of months, we should see a hybrid model where along with the conversational answer to your questions, you would see the most appropriate page links for your queries

Where would the top disruptions happen?

• Major disruptions are already happening in the fields of Creativity. User guides, whitepapers, essays, books, and copywriting have already been revolutionized
•User-end summarization and customer-facing applications like surveys, customer history, customer onboarding, case history, etc. are fields that are very ripe for disruption

• Simple coding has already been revolutionized as the GPT models were trained on millions of lines of codes

What is the future of GPT?

• It is now possible to connect outputs of Analytical models with LLMs to build conversational analytics capability

• Public data will always be available to train the GPTs on. One would never be able to compete with the immense amount of processing power that these systems will use and the deep pockets funding this processing power.

• However, where we can indeed train the GPTs into non-public and contextual sources of contemporary data, specific to certain fields and get much better outputs on questions within those fields. This is called fine-tuning the model on niche data.

• It would soon be possible to query structured data, get your query converted to SQL and then query your database to get insights and analytics

• We may soon see immense pricing wars within GPT services, Many firms may decide to start with free and premium models for their GPTs where they give ad-supported GPT content for free and you may pay for getting an ad-free experience. Similar to buying Google keywords, you could soon buy queries and place your product or services in those answers as sponsored product/services.

How much investment would be needed to create a vertical-based offering?

• Building a niche-trained model on top of existing LLM models depends on the size of the training set for niche data. It may be cost-effective to train biased or context-specific fine-tuned model for anything from $50k to millions of dollars. Many open-source models exist to deploy niche training in-house with dedicated Hardware rather than paying high API fees for OpenAI or other big-tech LLMs.

• Typically, first versions can be released in as little as 3 months, with a team of around 6 engineers. These fine-tuned models are all built on top of the existing GPTs

• Open AI is not the only one with a GPT. Multiple firms like Google’s Anthropic are in the same domain. Google has invested 400 M$ in Anthropic

• OpenAI was a marketing genius to market the platform and make it accessible. However, Google’s AI is much ahead than Open AI’s capabilities.

• Microsoft is investing 10 B$, mainly in Azure credits to train Open AI. Most of the investments in this domain are used in burning CPU processing as they are all very compute-intense processes

• Multiple firms are investing small amounts in training the GPTs in to context-specific cases for vertical-specific problems. These firms pay to GPTs basis tokens which are charged based on the number of words that they need to train the GPT on. Open Ai’s Ada, for example, charges $0.0004 per thousand tokens (i.e., 0.04 US cents per 750 words). The higher the amount of data, the higher is the cost.

The article is authored by Ankit Agrawal, CEO, Mentor Trust, IvyCap Ventures