Meta Releases TRIBE v2: A Brain Encoding Model That Predicts fMRI Responses Across Video, Audio, and Text Stimuli

2. Temporal Aggregation

3. Subject-Specific Prediction

Data and Scaling Laws

A significant hurdle in brain encoding is data scarcity. TRIBE v2 addresses this by utilizing ‘deep’ datasets for training—where a few subjects are recorded for many hours—and ‘wide’ datasets for evaluation.

• 
  
  Training:
  
  The model was trained on 451.6 hours of fMRI data from 25 subjects across four naturalistic studies (movies, podcasts, and silent videos).
  


• 
  
  Evaluation:
  
  It was evaluated across a broader collection totaling 1,117.7 hours from 720 subjects.
  

The research team observed a log-linear increase in encoding accuracy as the training data volume increased, with no evidence of a plateau. This suggests that as neuroimaging repositories expand, the predictive power of models like TRIBE v2 will continue to scale.

Results: Beating the Baselines

TRIBE v2 significantly outperforms traditional

Finite Impulse Response (FIR)

models, the long-standing gold standard for voxel-wise encoding








.

Zero-Shot and Group Performance

Fine-Tuning

When given a small amount of data (at most one hour) for a new participant, fine-tuning TRIBE v2 for just one epoch leads to a two- to four-fold improvement over linear models trained from scratch


.

In-Silico Experimentation

The research team argue that TRIBE v2 could be useful for

piloting or pre-screening neuroimaging studies

. By running virtual experiments on the

Individual Brain Charting (IBC)

dataset,

the model recovered classic functional landmarks:

• 
  
  Vision:
  
  It accurately localized the fusiform face area (
  
  FFA
  
  ) and parahippocampal place area (
  
  PPA
  
  ).
  


• 
  
  Language:
  
  It successfully recovered the temporo-parietal junction (
  
  TPJ
  
  ) for emotional processing and
  
  Broca’s area
  
  for syntax.
  

Furthermore, applying

Independent Component Analysis (ICA)

to the model’s final layer revealed that TRIBE v2 naturally learns five well-known functional networks: primary auditory, language, motion, default mode, and visual


.

Key Takeaway

• 
  
  A Powerhouse Tri-modal Architecture
  
  : TRIBE v2 is a foundation model that integrates
  
  video, audio, and language
  
  by leveraging state-of-the-art encoders like
  
  LLaMA 3.2
  
  for text,
  
  V-JEPA2
  
  for video, and
  
  Wav2Vec-BERT
  
  for audio.
  


• 
  
  Log-Linear Scaling Laws
  
  : Much like the Large Language Models we use every day, TRIBE v2 follows a
  
  log-linear scaling law
  
  ; its ability to accurately predict brain activity increases steadily as it is fed more fMRI data, with no performance plateau currently in sight.
  


• 
  
  Superior Zero-Shot Generalization
  
  : The model can predict the brain responses of
  
  unseen subjects
  
  in new experimental conditions without any additional training. Remarkably, its zero-shot predictions are often more accurate at estimating group-averaged brain responses than the recordings of individual human subjects themselves.
  


• 
  
  The Dawn of In-Silico Neuroscience
  
  : TRIBE v2 enables ‘in-silico’ experimentation, allowing researchers to run virtual neuroscientific tests on a computer. It successfully replicated decades of empirical research by identifying specialized areas like the
  
  fusiform face area (FFA)
  
  and
  
  Broca’s area
  
  purely through digital simulation.
  


• 
  
  Emergent Biological Interpretability
  
  : Even though it’s a deep learning ‘black box,’ the model’s internal representations naturally organized themselves into five well-known functional networks:
  
  primary auditory, language, motion, default mode, and visual
  
  .
  

Check out the


Code


,


Weights


and


Demo

.

Also, feel free to follow us on



Twitter



and don’t forget to join our


120k+ ML SubReddit


and Subscribe to


our Newsletter


. Wait! are you on telegram?


now you can join us on telegram as well


.