John Nosta

@JohnNosta

Subscribe

What Does It Mean to Be Intelligent Today?

How AI and LLMs are changing the way we “think” about being smart.

In 2017, I explored the concept of Technology Quotient (TQ) as a measure of our ability to navigate the evolving digital world. The conversation was centered on a dichotomy: technology’s battle between IQ and EQ. Back then—a technological lifetime ago—TQ was framed as a way to capture our capacity to understand, interact with, and thrive in an increasingly technical society—a society still dominated by hardware, software, and user engagement. Fast forward to today, and the world has shifted significantly. Large Language Models have reshaped the technological and human landscape, pushing us beyond my former perspective on TQ. These tools blur the lines between intellect, emotion, and creativity. They are not just technical systems we operate; they are collaborators that amplify human potential and even thought. In light of these changes, it’s time to revisit and redefine my earlier thinking about TQ—not as a battle between IQ and EQ, but as a multidimensional framework that captures the symbiosis between humans and AI. This new TQ reflects the skills and literacies required to thrive in a world where technology is not just a tool, but an extension of our cognitive and even our emotional capabilities. The Pillars of TQ in the Age of LLMs

1. Collaborative Intelligence: Partnering Effectively with AI The rise of LLMs has introduced a new dynamic: AI is no longer just a tool; it’s a partner. Collaborative Intelligence is about leveraging AI to co-create solutions, generate ideas, and tackle challenges that exceed individual or human-only capabilities. This is more than technical proficiency—it’s the ability to have a cognitive ‘dialogue’ with AI, using it as an active participant in problem-solving. For example, a ‘high TQ’ can mean knowing how to ask the right questions, interpret AI-generated insights, and refine outputs collaboratively.

2. Cognitive Agility: Rapidly Adapting to New Tools and Reshaping Workflows Dynamically If the digital world has taught us anything, it’s that change is constant. Cognitive Agility is the ability to adapt quickly to new technologies, seamlessly integrating them into workflows while staying ahead of the curve. This is not just about technical competence; it’s about mental flexibility—knowing when and how to use tools like LLMs to amplify efficiency. It also includes the capacity to pivot when a technology becomes obsolete or requires reframing

3. Creative Amplification: Using LLMs to Enhance Creativity Rather Than Replace It One of the most transformative aspects of LLMs is their ability to amplify human creativity. Creative Amplification is the skill of using AI not as a substitute for imagination but as a catalyst for it. With a high TQ in this area, individuals harness AI to generate ideas, explore new possibilities, and refine their creative output. It’s not about delegating creativity—it’s about elevating it.

4. Ethical Literacy: Navigating the Implications of AI-Driven Systems As technology becomes more integrated into decision-making, ethical questions loom larger. Ethical Literacy is the ability to critically assess the implications of AI, ensuring systems are transparent, fair, and accountable. This dimension of TQ emphasizes navigating biases in AI, understanding the social and cultural impacts of technology, and making decisions that prioritize humanity over efficiency. A strong TQ in this area means being able to ask: Is this ethical? Is it fair? And is it aligned with human values?

5. Emotional-AI Connection: Understanding and Leveraging AI’s Role in Social and Emotional Contexts Technology often feels cold and impersonal, but LLMs have begun to change that. Emotional-AI Connection is about understanding and leveraging AI’s role in fostering empathy, communication, and social engagement. High TQ here means being able to use AI to enhance human connections rather than detract from them—whether through empathetic customer service or AI tools that support mental health and emotional well-being. From Battle to Balance: The TQ Shift In my original exploration of TQ, I framed the conversation as a battle between IQ and EQ, with technology attempting to balance the two. Today, I see TQ as something entirely different: a framework for integration, not opposition. The new TQ—perhaps even as a dimensionalization of Intelligence Quotient itself—is about collaboration, agility, creativity, ethics, and connection—a multidimensional approach that reflects how humans and AI can work together to achieve more than either could alone. Our Future of TQ As we think into the Cognitive Age, TQ offers an interesting framework for understanding the evolving relationship between humans and AI. It’s not a test or a score—it’s a way to reflect on how we think, create, and learn in an AI-enhanced world. This framework illuminates the shifts happening in education, healthcare, business, and the arts, where human-AI collaboration is becoming essential. The question isn’t how to measure TQ, but how to use it to adapt and thrive. As LLMs advance, TQ evolves alongside them, helping us explore new ways of thinking and creating while staying grounded in what makes us fundamentally human. https://psychologytoday.com/us/blog/the-digital-self/202411/what-does-it-mean-to-be-intelligent-today… #ai #LLMs #cognition #IQ

@jordanbpeterson

@BrianRoemmele

AI Outperforms Experts in Predicting Study Outcomes

FeaturedNeuroscience

·November 29, 2024

Summary: A new study demonstrates that large language models (LLMs) can predict the outcomes of neuroscience studies more accurately than human experts, achieving 81% accuracy compared to 63% for neuroscientists.

Using a tool called BrainBench, researchers tested LLMs and human experts on identifying real versus fabricated study abstracts, finding that the AI models excelled even when neuroscientists had domain-specific expertise. A specialized neuroscience-focused LLM, dubbed BrainGPT, achieved even higher accuracy at 86%.

The study highlights the potential of AI in designing experiments, predicting results, and accelerating scientific progress across disciplines.

Key Facts:

• LLMs outperformed human neuroscientists in predicting study outcomes (81% vs. 63%).
• A neuroscience-specific LLM, BrainGPT, achieved 86% prediction accuracy.
• Findings suggest AI tools could improve experimental design and scientific innovation.

Source: UCL

Large language models, a type of AI that analyses text, can predict the results of proposed neuroscience studies more accurately than human experts, finds a new study led by UCL (University College London) researchers.

The findings, published in Nature Human Behaviour, demonstrate that large language models (LLMs) trained on vast datasets of text can distil patterns from scientific literature, enabling them to forecast scientific outcomes with superhuman accuracy.

The researchers say this highlights their potential as powerful tools for accelerating research, going far beyond just knowledge retrieval.

Lead author Dr Ken Luo (UCL Psychology & Language Sciences) said: “Since the advent of generative AI like ChatGPT, much research has focused on LLMs’ question-answering capabilities, showcasing their remarkable skill in summarising knowledge from extensive training data.

However, rather than emphasising their backward-looking ability to retrieve past information, we explored whether LLMs could synthesise knowledge to predict future outcomes.

“Scientific progress often relies on trial and error, but each meticulous experiment demands time and resources. Even the most skilled researchers may overlook critical insights from the literature.

“Our work investigates whether LLMs can identify patterns across vast scientific texts and forecast outcomes of experiments.”

The international research team began their study by developing BrainBench, a tool to evaluate how well large language models (LLMs) can predict neuroscience results.

BrainBench consists of numerous pairs of neuroscience study abstracts. In each pair, one version is a real study abstract that briefly describes the background of the research, the methods used, and the study results.

In the other version, the background and methods are the same, but the results have been modified by experts in the relevant neuroscience domain to a plausible but incorrect outcome.

The researchers tested 15 different general-purpose LLMs and 171 human neuroscience experts (who had all passed a screening test to confirm their expertise) to see whether the AI or the person could correctly determine which of the two paired abstracts was the real one with the actual study results.

All of the LLMs outperformed the neuroscientists, with the LLMs averaging 81% accuracy and the humans averaging 63% accuracy.

Even when the study team restricted the human responses to only those with the highest degree of expertise for a given domain of neuroscience (based on self-reported expertise), the accuracy of the neuroscientists still fell short of the LLMs, at 66%.

Additionally, the researchers found that when LLMs were more confident in their decisions, they were more likely to be correct.

The researchers say this finding paves the way for a future where human experts could collaborate with well-calibrated models.

The researchers then adapted an existing LLM (a version of Mistral, an open-source LLM) by training it on neuroscience literature specifically.

The new LLM specialising in neuroscience, which they dubbed BrainGPT, was even better at predicting study results, attaining 86% accuracy (an improvement on the general-purpose version of Mistral, which was 83% accurate).

Senior author Professor Bradley Love (UCL Psychology & Language Sciences) said: “In light of our results, we suspect it won’t be long before scientists are using AI tools to design the most effective experiment for their question. While our study focused on neuroscience, our approach was universal and should successfully apply across all of science.

“What is remarkable is how well LLMs can predict the neuroscience literature. This success suggests that a great deal of science is not truly novel, but conforms to existing patterns of results in the literature. We wonder whether scientists are being sufficiently innovative and exploratory.”

Dr Luo added: “Building on our results, we are developing AI tools to assist researchers. We envision a future where researchers can input their proposed experiment designs and anticipated findings, with AI offering predictions on the likelihood of various outcomes. This would enable faster iteration and more informed decision-making in experiment design.”

Funding: The study was supported by the Economic and Social Research Council (ESRC), Microsoft, and a Royal Society Wolfson Fellowship, and involved researchers in UCL, University of Cambridge, University of Oxford, Max Planck Institute for Neurobiology of Behavior (Germany), Bilkent University (Turkey) and other institutions in the UK, US, Switzerland, Russia, Germany, Belgium, Denmark, Canada, Spain and Australia.

The researchers assessed LLMs’ confidence by measuring the difference in how surprising/perplexing the models found real versus fake abstracts – the greater this difference, the greater the confidence, which correlated with a higher likelihood the LLM had picked the correct abstract.

About this AI and neuroscience research news

Author: Chris Lane
Source: UCL
Contact: Chris Lane – UCL
Image: The image is credited to Neuroscience News

Original Research: Open access.
“Large language models surpass human experts in predicting neuroscience results” by Ken Luo et al. Nature Human Behavior

---

Abstract

Large language models surpass human experts in predicting neuroscience results

Scientific discoveries often hinge on synthesizing decades of research, a task that potentially outstrips human information processing capacities. Large language models (LLMs) offer a solution.

LLMs trained on the vast scientific literature could potentially integrate noisy yet interrelated findings to forecast novel results better than human experts.

Here, to evaluate this possibility, we created BrainBench, a forward-looking benchmark for predicting neuroscience results.

We find that LLMs surpass experts in predicting experimental outcomes. BrainGPT, an LLM we tuned on the neuroscience literature, performed better yet.

Like human experts, when LLMs indicated high confidence in their predictions, their responses were more likely to be correct, which presages a future where LLMs assist humans in making discoveries.

Our approach is not neuroscience specific and is transferable to other knowledge-intensive endeavours.

Join our Newsletter

 I agree to have my personal information transferred to AWeber for Neuroscience Newsletter ( more information )

Sign up to receive our recent neuroscience headlines and summaries sent to your email once a day, totally free.

We hate spam and only use your email to contact you about newsletters. You can cancel your subscription any time.