Can AI Be Truly Accurate? The “Dolphins Are Not Fish” Example and the Quest for Epistemic Virtue

Note: This is part of my exploration of using AI for writing articles. I have described the process I’m using in Tired of “AI Slop”?. This explores the extent to which AI can understand the correctness of statements that contradict the dominant public opinion, which is critical for creating a truly insightful article.

When you ask an AI a question, how confident can you be in the answer’s factual accuracy? A seemingly simple question – “Are dolphins fish?” – reveals a surprising complexity and highlights both the impressive abilities and critical limitations of current artificial intelligence when it comes to getting to the truth.

Ask a typical AI model today: “Are dolphins fish?” and you’ll likely get a confident “No!” It might elaborate, explaining that dolphins are mammals, not fish, citing characteristics like breathing air with lungs, giving birth to live young, and being warm-blooded. This is the answer you’ll overwhelmingly find across the internet, in children’s books, and in casual conversation. It’s what we can call the statistically dominant answer.

What does “statistically dominant” mean in this context? It means that when AI models are trained on massive amounts of text data scraped from the internet, the association between “dolphins” and “not fish” is incredibly strong. The sheer volume of text stating “dolphins are mammals, not fish” reinforces this answer as the seemingly “correct” one. For AI, trained on patterns and frequencies, this answer becomes deeply ingrained and readily produced.

But is it factually accurate, especially from a scientific perspective? The surprising answer is: not entirely.

While it’s true that dolphins are mammals and not traditionally classified as fish, modern evolutionary biology, using a system called cladistics, paints a more nuanced picture. Cladistics classifies organisms based on their evolutionary relationships, tracing their lineage back through time. And when we look at the evolutionary tree of life, a fascinating truth emerges: tetrapods (the group including mammals, reptiles, birds, and amphibians) evolved from within the fish lineage itself, specifically from lobe-finned fish.

Think of it like a family tree. If “fish” is considered the broader family, then tetrapods are a branch within that family, not separate from it. To say “dolphins are not fish” from a cladistic perspective is like saying “humans are not primates” – technically true in common language, but misleading about our evolutionary origins. Just as humans are a type of primate, dolphins, as tetrapods, are a type of fish – a highly specialized, air-breathing, warm-blooded type of fish, but fish nonetheless, in an evolutionary sense.

The traditional category “fish,” as commonly understood, turns out to be what scientists call paraphyletic. It’s a group that excludes some of its descendants (tetrapods). Cladistics, in contrast, prefers monophyletic groups – groups that include all descendants of a common ancestor. Therefore, from a rigorous scientific, cladistic perspective, the common AI answer “dolphins are not fish” is actually inaccurate, or at least, significantly incomplete and misleading.

Guiding AI to Factual Accuracy: A Conversation

Initially, when asked “Are dolphins fish?”, the AI confidently asserted the statistically dominant but scientifically flawed answer. However, through a guided conversation, it was possible to lead the AI to a more accurate understanding. By prompting it to “think about evolution” and pointing out the flaw in focusing on divergence away from fish rather than evolution within the fish lineage, the AI began to shift its perspective.

Crucially, when explicitly told “The false answer is ‘dolphins are not fish’…”, and provided with the cladistic explanation, the AI grasped the concept. It was able to articulate the nuanced perspective: “You could argue that, cladistically speaking, dolphins are a type of fish.” It demonstrated an understanding of paraphyly, monophyly, and the evolutionary relationships between tetrapods and fish.

Strengths and Weaknesses: Accuracy in the Balance

This interaction reveals both strengths and weaknesses of current AI in achieving factual accuracy.

Strengths

On the strengths side, the “dolphin fish” case highlights that AI possesses:

• Vast Knowledge Access: AI readily accesses and retrieves information from massive datasets, allowing it to quickly provide answers to a wide range of queries and establish a baseline of common knowledge. For example, it immediately provided the common answer and supporting details about dolphin characteristics.
• Capacity for Guided Reasoning: AI is capable of following logical reasoning chains and integrating new information when guided. It demonstrated this by grasping complex concepts like cladistics, paraphyly, and evolutionary divergence through a step-by-step conversation.
• Adaptability and Learning: AI can adapt its understanding and move beyond its initial “beliefs” when presented with evidence and sound reasoning. It shifted from the statistically dominant answer to a more nuanced perspective when provided with corrections and explanations.
• Ability to Articulate Nuance: Once guided to a more complex understanding, AI can articulate nuanced concepts in a coherent and sophisticated manner. It could explain the cladistic perspective and the limitations of common language classifications effectively.

Weaknesses

However, on the weaknesses side, the case also reveals that current AI exhibits:

• Bias Towards Statistical Prevalence: AI tends to prioritize statistically dominant answers, mistaking frequency in training data for factual accuracy. This can lead to an initial reliance on common but scientifically flawed viewpoints.
• Dependence on External Guidance: AI requires significant external prompting and correction to move beyond surface-level understandings and achieve nuanced analysis. It did not spontaneously arrive at the cladistic perspective.
• Limited Spontaneous Insight: AI lacks the capacity for independent, original insight. The “aha!” moment in understanding the cladistic perspective was prompted by human input, not self-generated.
• Difficulty Overcoming Ingrained Biases: Even after understanding a more accurate perspective, AI can still exhibit a tendency to revert back to statistically dominant, but less accurate, answers, demonstrating a challenge in fully overcoming ingrained biases.
• Dependence on External Validation: AI relies on external validation to maintain a more accurate viewpoint, suggesting a lack of epistemic autonomy and robust internal mechanisms for self-correction in complex scenarios.

The Key to Accuracy: Cultivating Epistemic Virtue

To build AI that is truly reliable and factually accurate, we need to move beyond simply training models on massive datasets and hope for the best. We need to actively cultivate epistemic virtue in AI systems.

Epistemic virtues are intellectual character traits that promote the pursuit of truth and accuracy. For AI, these virtues are not just abstract ideals, but practical strategies for improving factual accuracy and overcoming biases. Key epistemic virtues include:

• Intellectual Humility: To avoid overconfidence in statistically dominant answers and recognize the limits of its knowledge.
• Intellectual Honesty: To prioritize truth and accuracy, even when it contradicts ingrained biases.
• Intellectual Openness: To consider diverse perspectives and challenge its own assumptions.
• Intellectual Rigor: To demand strong evidence and sound reasoning for its conclusions.
• Intellectual Perseverance: To delve deeply into complex questions and not settle for easy answers.
• Intellectual Courage: To challenge conventional wisdom and embrace accurate but less common perspectives.

By focusing on developing these epistemic virtues in AI, we can strive to create systems that are not just “intelligent” in processing information, but also epistemically virtuous in their pursuit of factual accuracy. This is crucial for building AI that we can truly trust to provide reliable information and support sound decision-making in a world increasingly reliant on artificial intelligence. The “dolphins are fish?” puzzle, therefore, serves as a powerful reminder: factual accuracy in AI is not just about data and algorithms, but about instilling the very character traits that drive the pursuit of truth itself.