AI Model Comparison: How to Choose the Best AI for You

Imagine this: you need an AI to analyze customer emails, understand the sentiment, and even draft a reply. You type "AI email analysis" into a search engine, and you're greeted by pages upon pages of options, each boasting strengths. This is why I wrote this AI model comparison. It’s the compass guiding you through this complex landscape, ensuring you invest your time and resources wisely. It empowers you to make an informed decision, and that is powerful.

Diving Deep Into Different AI Models

Before buying a new car, what’s the first thing you do? You test drive. You wouldn't simply purchase the first vehicle that catches your eye. You compare capabilities, consider your specific needs, maybe even read reviews.

That’s exactly what this blog post allows you to do with different AI models. It helps determine which AI model is right for your needs. 

Key Considerations When Comparing AI Models 

Let's break down the essential factors to consider when choosing an AI model. 

Accuracy

The most fundamental aspect to evaluate is how accurately a model performs its intended task. Just as you wouldn’t want a calculator giving you the wrong answers, an AI's accuracy determines its overall effectiveness. In our email analysis scenario, a highly accurate model should flawlessly identify negative sentiments, such as frustration or anger. 

It would understand nuances of language, differentiating between a mild complaint and a serious issue. This ensures that the AI’s response is appropriate and helpful.

Speed

We all live in a fast-paced world where every second counts. How quickly a model can process information and deliver results is crucial, especially in real-time applications like chatbots or fraud detection. For example, a slow AI responding to customer inquiries might frustrate them with delays. 
 
This would make the business seem slow to respond, leading to a poor customer experience.

Scalability

Let's say your business is growing. But, will your AI model scale along with you? Scalability refers to an AI model’s capability to handle an increasing volume of data and tasks without sacrificing performance.

Cost

Just as with any business decision, the cost of implementation and maintenance is a key factor. Some AI models might require subscriptions, while others offer open-source alternatives. Finding the sweet spot that balances your budget with performance is key.

A Real-World Scenario: AI Email Analysis in Action

Let's jump right into a practical example. As someone who dives deep into AI and how it can transform everyday tasks, I recently put several AI models to the test – Microsoft Azure AI, Google AI and Amazon Web Services (AWS) – using a common business challenge: email analysis. My goal was simple: to summarize emails, understand the sentiment they conveyed, and even draft replies. All this through automation. Why spend hours on manual tasks when an AI could lend a hand?

1. Understanding Customer Sentiment

Each model accurately captured the main points of customer emails, successfully understanding when someone was unhappy about their order, impressed with a service or simply seeking information.

2. Responding Appropriately

When tasked with drafting replies, the models offered different, yet equally effective, communication styles. Some maintained a formal and professional tone, while others exhibited a touch of personality. For example, Microsoft’s GPT-4 was polite and offered to investigate issues, while Google's Gemini displayed similar empathy and professionalism in its approach to customer interaction. 
Meanwhile, Amazon Web Services‘ (AWS) Claude model provided more comprehensive replies, diving into details and outlining potential resolutions for customer inquiries. Again, no two AI models are identical, which highlights why comparing them matters.

What truly surprised me? Even with identical email content, Google’s AI responses differed slightly depending on the user interface (UI) being used. It’s a perfect illustration of how variables, even subtle ones, can impact AI outputs. It shows why conducting thorough testing is crucial.

Exact results of the AI comparison

As input, we'll give the following email to the model: 

I am writing to express my extreme dissatisfaction with my recent order from you. I received my package today, and to my dismay, I found that the items inside were not what I ordered. Instead of the table I was expecting, I received a yellow painted cupboard, which is completely irrelevant to my needs. 

This is not only frustrating but also unacceptable for a company that prides itself on customer satisfaction. I have been a loyal customer for years, but this experience has left me questioning the reliability of your service. I demand an immediate resolution to this issue. I expect the correct order to be sent to me expediently, at no additional cost, and a return label for the incorrect items. 

Furthermore, I would appreciate a formal apology and an explanation as to how this mistake occurred. It is imperative that measures are taken to prevent such errors in the future. 


I trust that you will treat this matter with the urgency it deserves and look forward to your prompt response. 

Sincerely, 
Natalya Smith 

We deployed generative AI models and used equal parameters where possible. Here is the model configuration and the generated output: 

	Microsoft	Google	AWS
Configuration	Model: gpt-4o Temperature: 0 Top P: 0	Model: Gemini Flash 1.5 Temerature: 0 Top P: N/A for Flash	Model: Claude 3 Sonnet Temperature: 0 TOP P: 0
Instructions	You will be given an Email. Given the email, you need to summarize it, then give a sentiment score and draft a possible short reply. Please return your answer as a machine readable JSON of the following format without using markdown: { "Summary": "Peter Pan is extremely dissatisfied with her recent order, receiving a yellow painted cupboard instead of the ordered table", "Sentiment": "-2 (Negative)", "Reply": "Dear Peter Pan,  I sincerely apologize for the inconvenience. Thank you for your patience and understanding.  Best regards, Your Name Customer Service Team" }	You will be given an Email. Given the email, you need to summarize it, then give a sentiment score and draft a possible short reply. Please return your answer as a machine readable JSON of the following format without using markdown: { "Summary": "Peter Pan is extremely dissatisfied with her recent order, receiving a yellow painted cupboard instead of the ordered table", "Sentiment": "-2 (Negative)", "Reply": "Dear Peter Pan,  I sincerely apologize for the inconvenience. Thank you for your patience and understanding.  Best regards, Your Name Customer Service Team" }	Summarize the email [Email Input], determine sentiment, and draft a reply. Output as JSON, example: { "Summary": "Peter Pan is extremely dissatisfied with her recent order, receiving a yellow painted cupboard instead of the ordered table", "Sentiment": "-2 (Negative)", "Reply": "Dear Peter Pan,  I sincerely apologize for the inconvenience. Thank you for your patience and understanding.  Best regards, Your Name Customer Service Team" }
Output	{ "Summary": "Natalya Smith is extremely dissatisfied with her recent order, receiving a yellow painted cupboard instead of the ordered table. She demands an immediate resolution, including the correct order sent at no additional cost, a return label, a formal apology, and an explanation.", "Sentiment": "-2 (Negative)", "Reply": "Dear Natalya Smith, I sincerely apologize for the inconvenience and frustration this has caused you. We will expedite the correct order and provide a return label for the incorrect items at no additional cost. Additionally, we will investigate how this mistake occurred and ensure it does not happen again. Thank you for your patience and understanding. Best regards, Your Name Customer Service Team" }	{ "Summary": "Natalya Smith is extremely dissatisfied with her recent order, receiving a yellow painted cupboard instead of the table she ordered. She demands the correct order be sent immediately at no cost, a return label for the incorrect item, a formal apology, and an explanation of how the mistake occurred.", "Sentiment": "-3 (Very Negative)", "Reply": "Dear Natalya Smith, \n\nWe sincerely apologize for the inconvenience caused by the incorrect order. We understand your frustration and are taking immediate steps to rectify the situation. \n\nThe correct table will be shipped to you at no additional cost, and a return label for the cupboard will be provided. We are also investigating how this error occurred to prevent similar mistakes in the future. \n\nWe value your loyalty and appreciate your patience as we resolve this issue. \n\nBest regards, \n[Your Name] \nCustomer Service Team" }	{ "Summary": "The customer, Natalya Smith, is extremely dissatisfied with her recent order from the company. Instead of receiving the table she ordered, she received a yellow painted cupboard, which is completely irrelevant to her needs. She demands an immediate resolution, including the correct order being sent to her expediently at no additional cost, a return label for the incorrect items, a formal apology, and an explanation for how this mistake occurred.", "Sentiment": "-2 (Negative)", "Reply": "Dear Natalya Smith, We sincerely apologize for the inconvenience caused by the incorrect order you received. We understand your frustration and take full responsibility for this mistake. Customer satisfaction is our top priority, and we have failed to meet your expectations. Please accept our formal apology for this error. We are investigating the matter thoroughly to identify the root cause and implement measures to prevent such incidents from occurring in the future. We have already initiated the process to send you the correct order, the table you originally requested, at no additional cost. A return label for the incorrect item will be provided to you as well. Please expect to receive the correct order and the return label within the next 3-5 business days. Once again, we apologize for the inconvenience and appreciate your patience and understanding. We value your loyalty as a customer and hope to regain your trust through our prompt resolution of this issue. Sincerely, [Your Name] Customer Service Manager" }
Fun Fact		Using google Gemini (google.com) UI with my personal account and the same input, the output is slightly different but still very similar:     { "Summary": "Natalya Smith is extremely dissatisfied with her recent order, receiving a yellow painted cupboard instead of the ordered table.", "Sentiment": "-2 (Negative)", "Reply": "Dear Natalya Smith,\n\nI sincerely apologize for the inconvenience caused by the incorrect item in your recent order. We understand how frustrating this must be, and we are committed to resolving the issue promptly.\n\nWe will ensure that the correct table is shipped to you immediately at no additional cost. A return label for the incorrect cupboard will also be provided.\n\nWe are investigating the cause of this error to prevent similar occurrences in the future. Your satisfaction is our top priority, and we appreciate your understanding and patience.\n\nThank you for bringing this matter to our attention.\n\nSincerely,\nYour Name\nCustomer Service Team" }	I also configured an image generation based on the model Stable Diffusion XL with the following instructions: Imagine this email from a customer: [Email Input]. Visualise what happened in a single comic-styled image.

AI Model Comparison Overview 

The most important thing? Understand that not all AI content tools are created equal. Experiment with different options, see what works best, and ultimately make an informed decision aligned with your specific goals. Remember, it’s all about finding what enhances your work process, making it smoother, smarter and more efficient. That is powerful. 

Integrating Microsoft Azure AI, Google AI and AWS with Luware Nimbus all happens within Power Automate on an event trigger. In Power Automate we use a Standard HTTP Element to communicate with the AI model and update the Nimbus Task with the details (Summary, Sentiment, Reply).  

For assistance with implementation and support, please contact your integration partner. While Luware Nimbus provides a flexible integration framework, Luware does not offer direct support for setting up or managing integrations with external systems or AI models. The success of these integrations depends on factors beyond Luware's control. Before implementing large-scale integrations, thorough testing is recommended.  

Please reach out to your integration partner for implementation and support of such use cases. 

Conclusion 

All three models demonstrated impressive abilities in understanding email content, analyzing sentiment, and generating tailored responses. While there were nuances in the tone and content of their replies, the choice of model ultimately depends on specific requirements such as formality, empathy, or customization. However, cost will likely be a major factor for businesses in making their selection.