science.sciencemag.org/content/365/6454/eaax1030/suppl/DC1

Supplementary Materials for

Hippocampal sharp-wave ripples linked to visual episodic

recollection in humans

Yitzhak Norman, Erin M. Yeagle, Simon Khuvis, Michal Harel, Ashesh D. Mehta, Rafael Malach*

*Corresponding author. Email: rafi.malach@gmail.com

Published 16 August 2019, Science 365, eaax1030 (2019) DOI: 10.1126/science.aax1030

This PDF file includes: Figs. S1 to S9 Table S1 Caption for movie S1 References and Notes Other supplementary material for this manuscript includes the following: Movie S1

2

Supplementary Figures and Tables

3

Fig. S1. Hippocampal electrodes in individual patients

Location of hippocampal electrodes in each patient presented on a pre-operative MRI (coronal

slice) alongside a 3D reconstruction of the hippocampus. Overlaid colors show hippocampal

subfield parcellation implemented in FreeSurfer (77). Insets show magnification of the

CA1\subicular recording site used for ripple detection (white arrows).

4

Fig. S2. Ripples' rate and spectral properties across the different experimental conditions

(A) Examples of individual SWR events, recorded during the different phases of the task in one

representative patient: resting-state (left), picture viewing (middle) and free-recall (right). SWR

peak is marked by a red triangle.

5

(B) Mean peri-ripple field potential and wavelet spectrograms, plotted separately for each

condition: resting-state ("rest"), picture viewing ("viewing"), verbal recall periods ("recall"), and

memory search periods ("search").

(C) The same data as in panel B – showing the four conditions overlaid one on top of the other for

comparison. Shaded areas represent SEM across patients.

(D) Normalized spectra at ripples' peak, showing similar spectral signature across conditions.

(E-F) Ripples’ peak frequency and amplitude showed no significant differences between

conditions (non-parametric Friedman test; n=15 patients).

(G) Mean (basal-) ripple rate in each experimental condition. A non-parametric Friedman test (for

repeated measures design with two replicates) indicated a significant difference between

conditions (χ2(3)=18.04, P=0.0004, n=15 patients), and post-hoc comparisons using pairwise

Friedman tests with FDR correction indicated that ripple rate was significantly lower during recall

and memory search (i.e. inter-recall intervals), as compared to picture viewing and rest conditions.

Grand average ripple rate was 0.41 events/sec (SD=0.05). Boxplots represent Q1-Q3, horizontal

line is median, whiskers show range up to 1.5 time the interquartile range, and circles above/below

whiskers show outliers. Filled dots represent individual patients. *P<0.05; **P<0.01

6

Fig. S3. Examining the consistency in ripple rate responses during picture viewing

(A) Raster plot and PSTH of SWR events time-locked to the onset of picture presentation, showing

SWR-rate response separately for each presentation cycle. Each dot in the raster plot represents an

individual SWR event. Shaded areas represent bootstrap SE computed over patients (n=15).

(B) Ripple rates across presentation cycles in individual patients. Ripple rate responses were

averaged over a time window of 500 ms centered on the peak of the grand-average ripple rate

response (latency: 725 ms post-stimulus; computed across all presentations and all patients).

Comparing ripple rates across the four presentations using a non-parametric repeated-measures

Friedman test indicated that ripple rate significantly differed between presentation cycles

(2(3)=14.27, P<0.003; n=15; left panel). Comparing ripple rate during the first presentation to the

average ripple rate during repeated presentations showed a significant difference (P<0.005, n=15,

Wilcoxon signed-rank test), which was also evident in individual patients (each line represents one

patient).

(C) Examining the consistency in ripple rates to the same picture. To test the consistency of ripple-

rate responses during repeated presentations of the same images, we conducted a two-way

ANOVA (item × presentation cycle) individually in each subject, which quantified the within- and

between- item variability of ripple rate responses (using a time window of 50-1500 ms post-

stimulus). We then computed the ratio of 'between items' to 'within items' mean squares

(BMS/WMS) and found that it was significantly greater than one across patients (P<0.05, n=15

patients, Wilcoxon signed-rank test). This result points to a consistent item-specific response

during picture viewing.

(D) Examining the consistency in ripple rates to different pictures. To examine if there were any

consistent differences in ripple rates in response to the presentation of different pictures, we

computed the mean ripple rate during the presentation of each picture (averaged over the entire

stimulus duration and across all presentations, separately in each patient) and compared the

responses across all patients using a non-parametric repeated measures Friedman test. There were

no significant ripple rate differences between the different pictures used in our study

7

(2(27)=16.94, P>0.93, n=15; left panel). Comparing ripple rates between the two categories

likewise indicated no significant difference (P>0.75, n=15, Wilcoxon signed-rank test; right

panel). Each line represents one patient.

8

Fig. S4. Content selective ripple rate modulation during free recall

(A-B) Binned scatter plots showing the correlation between SWR rate during viewing and during

free-recall of the same items (n=252 successfully recalled items, pooled across 15 patients;

repeated recollections were averaged together). A significant correlation was found only for

repeated presentations (Spearman's ρ=0.18, P=0.005).

(C) Raster plot and PSTH of SWR events time-locked to the onset of verbal report of recall,

showing mean ripple rate response separately for low-RR and high-RR images (same notations as

in Fig. 2D). Each dot in the raster plot represents an individual SWR elicited during recollection

of high-RR images. Shaded areas represent bootstrap SE computed over trials. See Movie S1 for

representative examples of spontaneous recall events.

(D) Results obtained from a nonparametric cluster-based permutation test comparing mean ripple

rate in the actual data (computed across all individual recall events), to 2000 shuffled PSTHs

produced by circularly jittering ripple timing by a random amount, independently in each trial. A

threshold of P=0.05 was first applied to the data to detect temporal-clusters of SWR rate

increase/decrease. The actual data showed an increase in SWR rate prior to the onset of verbal

recall, that was significantly greater than clusters expected by chance (P=0.0035, FWE-corrected).

(E) Fitting a negative binomial distribution separately to recall events of low-RR and high-RR

images indicated a greater probability to observe higher number of SWRs when recalling 'high-

RR' images (supplementing the analysis described in Fig 2D).

(F) Individual patients' data, showing a significantly higher ripple rate during recall of high-RR

images as compared to low-RR images (P<0.05, Wilcoxon signed-rank test, n=14 patients; one

patient was excluded from this analysis due to insufficient number of trials in each condition). To

perform the analysis in E and F, we used a time-window of 3 sec centered on the peak of the grand-

average SWR rate computed across all recall events (Fig. 2C).

9

Fig S5. Examining the difference in ripple rate between high-RR and low-RR images during

viewing

(A-B) Raster plot and PSTH of SWR events, time-locked to the presentation of high-RR and low-

RR images, showing the average ripple rate separately for novel and repeated presentations. Each

dot in the raster plot represents an individual SWR event. Shaded areas represent bootstrap SE

across patients (n=15).

(C) To better understand the observed difference in ripple rate between the first presentation and

repeated presentations (Fig 2A), and how this difference is related to low-RR and high-RR images

(Fig 2B), we averaged the ripple rate responses over a time window of 0.05 to 1.5 sec from picture

onset and carried out a two-way repeated-measures ANOVA with presentation cycle as one factor,

and image group (low-RR/high-RR) as a second factor. Consistent with the results in Fig 2A, there

was a significant effect of presentation cycle (F(3,42)=4.07, P<0.01), and a significant interaction

between presentation cycle and image group (F(3,42)=16.78, P<0.001). Additional signed-rank

tests comparing the first presentation versus the average of repeated presentations, indicated that

the above interaction arose mainly from a reduction in ripple rate associated with low-RR images

(low-RR: PFDR<0.001; high-RR: PFDR=0.08 (n.s.); n=15 subjects). There was no difference in

ripple rate between novel presentations of high-RR and low-RR images (P>0.6, Wilcoxon signed-

rank test).

10

Fig S6. SWRs during free recall were not coupled to abrupt vocalizations

(A) To confirm that SWR detection was not influenced by possible artifacts related to vocalization,

we aligned the patients' voice amplitude to SWR events detected during the free recall period (10

min in total in each patient). To perform this analysis, voice amplitude time-series were smoothed

by a 400-ms triangular window, down-sampled to 500 Hz and transformed into z-score after

clipping extreme values to the 99 percentile. There was no significant change in voice amplitude

relative to SWR peak (PFDR=NS, Wilcoxon signed-rank test comparing each time point versus the

median, n=15 patients). Shaded areas represent SEM across patients.

(B-C) To examine the potential coupling between SWRs and abrupt vocalizations, we detected

transient peaks in voice amplitude (>2.5 SD, see example in B), and computed a peri-stimulus time

histogram (PSTH) of transient vocalizations, aligned to the SWR events. A non-parametric

permutation test (similar to the one reported in Fig 2C), comparing the actual PSTH to 2,000

shuffled PSTHs produced by circularly jittering vocalization peak by a random amount, indicated

that there was no significant coupling between the events. Each dot in the raster plot represents a

single event of abrupt vocalization.

11

Fig S7. Individual patients' data supplementing fig. 3

To examine whether ripple rate during encoding was indicative of subsequent recall performance,

we computed the normalized ripple rate difference between remembered and forgotten items:

(𝑅𝐸𝑀 − 𝐹𝑂𝑅) (𝑅𝐸𝑀 + 𝐹𝑂𝑅)⁄ . This index can vary from −1.0 to 1.0, with positive values

indicating a higher ripple rate for remembered items, and negative values indicating a higher ripple

rate for non-recalled items. The left and right panels show, respectively, the average differential

SWR activity during the stimulus presentation period (50 to 1500 ms after picture onset), and

during the post-stimulus interval (750 ms from picture offset until the onset of the next image),

separately for first and repeated presentations. As described in fig. 3, remembered pictures were

significantly linked to a higher ripple rate in the post-stimulus interval during the first presentation

cycle (P<0.05, Wilcoxon signed-rank against zero, FDR corrected). Each dot represents an

individual patient.

12

Fig. S8. Peri-ripple activity in category selective electrodes

(A) Peri-ripple spectrograms showing activity in low frequencies (<30 Hz) during recall of

preferred versus non-preferred images. There was no significant difference between preferred and

non-preferred items in this frequency range (P=NS, nonparametric cluster-based permutation test

for time-frequency data (90), shuffling preferred/non-preferred labels 2000 times over electrodes).

(B) HFB power during presentation of pictures from the preferred versus the non-preferred

category (P<0.05, cluster-based permutation test; significant time-bins are marked in orange).

(C-D) Peri-ripple HFB activity during memory search (i.e. inter-recall intervals), when patients

attempted to recall the preferred versus the non-preferred category (P=NS, one-sided cluster-based

permutation test, shuffling preferred/non-preferred labels 2000 times over electrodes).

(E) Peri-ripple spectrogram averaged across category selective recording sites during the resting

state period recorded in the beginning of each run.

(F) A one-sided cluster-based permutation test indicated that peri-ripple HFB activity during recall

of preferred images (red line) (same as in Fig. 5B) was significantly higher than peri-ripple activity

during resting-state (blue line; significant time-bins are marked by a blue horizontal bar), and

during memory search periods (turquoise line; significant time-bins are marked by a turquoise

13

horizontal bar). Exact p-values: Memory search (attempting to recall the preferred category):

P=0.003, resting-state: P=0.02.

All above analyses included n=57 category-selective bipolar electrode pairs from 13 patients, after

excluding recording sites with less than five peri-ripple responses in each condition (same as in

fig. 5B).

14

Fig. S9. Anatomical locations of bipolar electrode-pairs participating in the multivariate pattern

analysis depicted in fig. 6 (n=78, 14 patients).

15

Patient # Sex Age Lang # of

Implants

Total analyzed

recording sites

Visually-

responsive

sites

Category-selective visual sites

Faces Places

1 M 28 Eng 1 68 14 6 0

2 M 50 Spa 1 55 15 0 0

3 M 38 Eng 1 51 10 1 3

4 F 44 Eng 1 77 10 1 0

5 F 29 Eng 1 57 20 3 0

6 F 22 Eng 1 118 7 0 0

7 F 34 Eng 1 68 19 2 2

8 F 30 Spa 1 95 15 3 3

9 F 29 Eng 1 123 27 1 6

10 F 32 Eng 1 104 7 2 1

11 M 41 Spa 1 37 2 0 0

12 F 55 Eng 1 122 6 1 1

13 F 27 Eng 2* 154 40 11 3

14 F 33 Eng 1 93 11 2 0

15 M 57 Eng 1 86 14 5 1

Total

1308 217 38 20

Table S1. Patient information and distribution of visually responsive electrodes across individuals.

* Patient #13 had two separate implantations; in this specific case, we performed the experiment

twice using two different sets of stimuli and collapsed together the results prior to the group

analysis.

16

Movie S1.

Examples of hippocampal ripples elicited during free recall in three patients. In the video you can

hear the verbal report of the patients during spontaneous recall events, and see the simultaneous

changes in ripple band amplitude (70-180 Hz) along with the detected SWR events (marked by

yellow triangles and accompanied by a brief sound). These examples demonstrate the rich visual

content emerging during recall, and the transient increase in ripple rate that precedes the verbal

reports of the freely recalled images (accompanying fig. 2).

Some of the images shown to patients are protected by copyright. In the video we replaced these

images by similar substitutes or illustrations. Photo of the White House by Chuck Kennedy (an

official White House photo); photo of the Golden Gate Bridge courtesy of Nicolas Raymond;

photo of the Eiffel Tower courtesy of Joe Parks. These pictures are all published under a Creative

Commons license. The illustrations are based on screenshots from The Wolf of Wall Street,

Paramount Pictures (2013); Pulp Fiction, Miramax Films (1994); and the TV show Friends, CBS

Interactive Inc. (1994-2004).

1

References and Notes

1. G. Buzsáki, L.-W. S. Leung, C. H. Vanderwolf, Cellular bases of hippocampal EEG in the behaving rat. Brain Res. Rev. 287, 139–171 (1983). doi:10.1016/0165-0173(83)90037-1 Medline

2. G. Buzsáki, Z. Horváth, R. Urioste, J. Hetke, K. Wise, High-frequency network oscillation in the hippocampus. Science 256, 1025–1027 (1992). doi:10.1126/science.1589772 Medline

3. A. Ylinen, A. Bragin, Z. Nádasdy, G. Jandó, I. Szabó, A. Sik, G. Buzsáki, Sharp wave-associated high-frequency oscillation (200 Hz) in the intact hippocampus: Network and intracellular mechanisms. J. Neurosci. 15, 30–46 (1995). doi:10.1523/JNEUROSCI.15-01-00030.1995 Medline

4. D. Sullivan, J. Csicsvari, K. Mizuseki, S. Montgomery, K. Diba, G. Buzsáki, Relationships between hippocampal sharp waves, ripples, and fast gamma oscillation: Influence of dentate and entorhinal cortical activity. J. Neurosci. 31, 8605–8616 (2011). doi:10.1523/JNEUROSCI.0294-11.2011 Medline

5. B. K. Hulse, L. C. Moreaux, E. V. Lubenov, A. G. Siapas, Membrane Potential Dynamics of CA1 Pyramidal Neurons during Hippocampal Ripples in Awake Mice. Neuron 89, 800–813 (2016). doi:10.1016/j.neuron.2016.01.014 Medline

6. G. Buzsáki, D. L. Buhl, K. D. Harris, J. Csicsvari, B. Czéh, A. Morozov, Hippocampal network patterns of activity in the mouse. Neuroscience 116, 201–211 (2003). doi:10.1016/S0306-4522(02)00669-3 Medline

7. A. Bragin, J. Engel Jr., C. L. Wilson, I. Fried, G. Buzsáki, High-frequency oscillations in human brain. Hippocampus 9, 137–142 (1999). doi:10.1002/(SICI)1098-1063(1999)9:2<137:AID-HIPO5>3.0.CO;2-0 Medline

8. M. Le Van Quyen, A. Bragin, R. Staba, B. Crépon, C. L. Wilson, J. Engel Jr., Cell Type-Specific Firing during Ripple Oscillations in the Hippocampal Formation of Humans. J. Neurosci. 28, 6104–6110 (2008). doi:10.1523/JNEUROSCI.0437-08.2008

9. B. P. Staresina, T. O. Bergmann, M. Bonnefond, R. van der Meij, O. Jensen, L. Deuker, C. E. Elger, N. Axmacher, J. Fell, Hierarchical nesting of slow oscillations, spindles and ripples in the human hippocampus during sleep. Nat. Neurosci. 18, 1679–1686 (2015). doi:10.1038/nn.4119 Medline

10. N. K. Logothetis, O. Eschenko, Y. Murayama, M. Augath, T. Steudel, H. C. Evrard, M. Besserve, A. Oeltermann, Hippocampal-cortical interaction during periods of subcortical silence. Nature 491, 547–553 (2012). doi:10.1038/nature11618 Medline

11. Y. Nir, R. J. Staba, T. Andrillon, V. V. Vyazovskiy, C. Cirelli, I. Fried, G. Tononi, Regional slow waves and spindles in human sleep. Neuron 70, 153–169 (2011). doi:10.1016/j.neuron.2011.02.043 Medline

2

12. J. J. Chrobak, G. Buzsáki, High-frequency oscillations in the output networks of the hippocampal-entorhinal axis of the freely behaving rat. J. Neurosci. 16, 3056–3066 (1996). doi:10.1523/JNEUROSCI.16-09-03056.1996 Medline

13. K. Mizuseki, G. Buzsáki, Preconfigured, skewed distribution of firing rates in the hippocampus and entorhinal cortex. Cell Rep. 4, 1010–1021 (2013). doi:10.1016/j.celrep.2013.07.039 Medline

14. H. R. Joo, L. M. Frank, The hippocampal sharp wave-ripple in memory retrieval for immediate use and consolidation. Nat. Rev. Neurosci. 19, 744–757 (2018). doi:10.1038/s41583-018-0077-1 Medline

15. G. Buzsáki, Hippocampal sharp waves: Their origin and significance. Brain Res. 398, 242–252 (1986). doi:10.1016/0006-8993(86)91483-6 Medline

16. G. Buzsáki, Hippocampal sharp wave-ripple: A cognitive biomarker for episodic memory and planning. Hippocampus 25, 1073–1188 (2015). doi:10.1002/hipo.22488 Medline

17. T. K. Leonard, K. L. Hoffman, Sharp-Wave Ripples in Primates Are Enhanced near Remembered Visual Objects. Curr. Biol. 27, 257–262 (2017). doi:10.1016/j.cub.2016.11.027 Medline

18. T. K. Leonard, J. M. Mikkila, E. N. Eskandar, J. L. Gerrard, D. Kaping, S. R. Patel, T. Womelsdorf, K. L. Hoffman, Sharp Wave Ripples during Visual Exploration in the Primate Hippocampus. J. Neurosci. 35, 14771–14782 (2015). doi:10.1523/JNEUROSCI.0864-15.2015 Medline

19. D. Khodagholy, J. N. Gelinas, G. Buzsáki, Learning-enhanced coupling between ripple oscillations in association cortices and hippocampus. Science 358, 369–372 (2017). doi:10.1126/science.aan6203 Medline

20. N. Maingret, G. Girardeau, R. Todorova, M. Goutierre, M. Zugaro, Hippocampo-cortical coupling mediates memory consolidation during sleep. Nat. Neurosci. 19, 959–964 (2016). doi:10.1038/nn.4304 Medline

21. A. G. Siapas, M. A. Wilson, Coordinated interactions between hippocampal ripples and cortical spindles during slow-wave sleep. Neuron 21, 1123–1128 (1998). doi:10.1016/S0896-6273(00)80629-7 Medline

22. Z. Clemens, M. Mölle, L. Eross, P. Barsi, P. Halász, J. Born, Temporal coupling of parahippocampal ripples, sleep spindles and slow oscillations in humans. Brain 130, 2868–2878 (2007). doi:10.1093/brain/awm146 Medline

23. A. Peyrache, M. Khamassi, K. Benchenane, S. I. Wiener, F. P. Battaglia, Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat. Neurosci. 12, 919–926 (2009). doi:10.1038/nn.2337 Medline

3

24. D. Ji, M. A. Wilson, Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat. Neurosci. 10, 100–107 (2007). doi:10.1038/nn1825 Medline

25. G. Girardeau, I. Inema, G. Buzsáki, Reactivations of emotional memory in the hippocampus-amygdala system during sleep. Nat. Neurosci. 20, 1634–1642 (2017). doi:10.1038/nn.4637 Medline

26. S. P. Jadhav, G. Rothschild, D. K. Roumis, L. M. Frank, Coordinated Excitation and Inhibition of Prefrontal Ensembles during Awake Hippocampal Sharp-Wave Ripple Events. Neuron 90, 113–127 (2016). 10.1016/j.neuron.2016.02.010 Medline

27. G. Rothschild, E. Eban, L. M. Frank, A cortical-hippocampal-cortical loop of information processing during memory consolidation. Nat. Neurosci. 20, 251–259 (2017). doi:10.1038/nn.4457 Medline

28. S. N. Gomperts, F. Kloosterman, M. A. Wilson, VTA neurons coordinate with the hippocampal reactivation of spatial experience. eLife 4, e05360 (2015). doi:10.7554/eLife.05360 Medline

29. H. Zhang, J. Fell, N. Axmacher, Electrophysiological mechanisms of human memory consolidation. Nat. Commun. 9, 4103 (2018). 10.1038/s41467-018-06553-y Medline

30. I. Skelin, S. Kilianski, B. L. McNaughton, Hippocampal coupling with cortical and subcortical structures in the context of memory consolidation. Neurobiol. Learn. Mem. 160, 21–31 (2019). doi:10.1016/j.nlm.2018.04.004 Medline

31. G. Buzsáki, Two-stage model of memory trace formation: A role for “noisy” brain states. Neuroscience 31, 551–570 (1989). doi:10.1016/0306-4522(89)90423-5 Medline

32. T. J. Davidson, F. Kloosterman, M. A. Wilson, Hippocampal replay of extended experience. Neuron 63, 497–507 (2009). doi:10.1016/j.neuron.2009.07.027 Medline

33. M. P. Karlsson, L. M. Frank, Awake replay of remote experiences in the hippocampus. Nat. Neurosci. 12, 913–918 (2009). doi:10.1038/nn.2344 Medline

34. C. T. Wu, D. Haggerty, C. Kemere, D. Ji, Hippocampal awake replay in fear memory retrieval. Nat. Neurosci. 20, 571–580 (2017). doi:10.1038/nn.4507 Medline

35. A. C. Singer, M. F. Carr, M. P. Karlsson, L. M. Frank, Hippocampal SWR activity predicts correct decisions during the initial learning of an alternation task. Neuron 77, 1163–1173 (2013). doi:10.1016/j.neuron.2013.01.027 Medline

36. G. Dragoi, S. Tonegawa, Preplay of future place cell sequences by hippocampal cellular assemblies. Nature 469, 397–401 (2011). doi:10.1038/nature09633 Medline

37. K. Diba, G. Buzsáki, Forward and reverse hippocampal place-cell sequences during ripples. Nat. Neurosci. 10, 1241–1242 (2007). doi:10.1038/nn1961 Medline

4

38. D. J. Foster, M. A. Wilson, Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440, 680–683 (2006). doi:10.1038/nature04587 Medline

39. D. K. Roumis, L. M. Frank, Hippocampal sharp-wave ripples in waking and sleeping states. Curr. Opin. Neurobiol. 35, 6–12 (2015). doi:10.1016/j.conb.2015.05.001 Medline

40. M. C. Corballis, Mental time travel: A case for evolutionary continuity. Trends Cogn. Sci. 17, 5–6 (2013). doi:10.1016/j.tics.2012.10.009 Medline

41. M. F. Carr, S. P. Jadhav, L. M. Frank, Hippocampal replay in the awake state: A potential substrate for memory consolidation and retrieval. Nat. Neurosci. 14, 147–153 (2011). doi:10.1038/nn.2732 Medline

42. H. Gelbard-Sagiv, R. Mukamel, M. Harel, R. Malach, I. Fried, Internally generated reactivation of single neurons in human hippocampus during free recall. Science 322, 96–101 (2008). doi:10.1126/science.1164685 Medline

43. S. Khuvis, E. M. Yeagle, Y. Norman, S. Grossman, R. Malach, A. D. Mehta, Face-selective units in human ventral temporal cortex reactivate during free recall. bioRxiv 487686 [preprint]. 5 December 2018.

44. Y. Norman, E. M. Yeagle, M. Harel, A. D. Mehta, R. Malach, Neuronal baseline shifts underlying boundary setting during free recall. Nat. Commun. 8, 1301 (2017). doi:10.1038/s41467-017-01184-1 Medline

45. R. Mukamel, H. Gelbard, A. Arieli, U. Hasson, I. Fried, R. Malach, Coupling between neuronal firing, field potentials, and FMRI in human auditory cortex. Science 309, 951–954 (2005). doi:10.1126/science.1110913 Medline

46. J. Parvizi, S. Kastner, Promises and limitations of human intracranial electroencephalography. Nat. Neurosci. 21, 474–483 (2018). doi:10.1038/s41593-018-0108-2 Medline

47. B. O. Watson, M. Ding, G. Buzsáki, Temporal coupling of field potentials and action potentials in the neocortex. Eur. J. Neurosci. 48, 2482–2497 (2018). Medline

48. A. Oliva, A. Fernández-Ruiz, G. Buzsáki, A. Berényi, Role of Hippocampal CA2 Region in Triggering Sharp-Wave Ripples. Neuron 91, 1342–1355 (2016). doi:10.1016/j.neuron.2016.08.008 Medline

49. N. Axmacher, C. E. Elger, J. Fell, Ripples in the medial temporal lobe are relevant for human memory consolidation. Brain 131, 1806–1817 (2008). doi:10.1093/brain/awn103 Medline

50. R. J. Staba, C. L. Wilson, A. Bragin, I. Fried, J. Engel Jr., Quantitative analysis of high-frequency oscillations (80-500 Hz) recorded in human epileptic hippocampus and

5

entorhinal cortex. J. Neurophysiol. 88, 1743–1752 (2002). doi:10.1152/jn.2002.88.4.1743 Medline

51. Y. Nir, L. Fisch, R. Mukamel, H. Gelbard-Sagiv, A. Arieli, I. Fried, R. Malach, Coupling between neuronal firing rate, gamma LFP, and BOLD fMRI is related to interneuronal correlations. Curr. Biol. 17, 1275–1285 (2007). doi:10.1016/j.cub.2007.06.066 Medline

52. H. Hentschke, M. C. Stüttgen, Computation of measures of effect size for neuroscience data sets. Eur. J. Neurosci. 34, 1887–1894 (2011). doi:10.1111/j.1460-9568.2011.07902.x Medline

53. L. Wang, R. E. B. B. Mruczek, M. J. Arcaro, S. Kastner, Probabilistic maps of visual topography in human cortex. Cereb. Cortex 25, 3911–3931 (2015). doi:10.1093/cercor/bhu277 Medline

54. R. S. Desikan, F. Ségonne, B. Fischl, B. T. Quinn, B. C. Dickerson, D. Blacker, R. L. Buckner, A. M. Dale, R. P. Maguire, B. T. Hyman, M. S. Albert, R. J. Killiany, An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31, 968–980 (2006). doi:10.1016/j.neuroimage.2006.01.021 Medline

55. B. Fischl, N. Rajendran, E. Busa, J. Augustinack, O. Hinds, B. T. T. Yeo, H. Mohlberg, K. Amunts, K. Zilles, Cortical folding patterns and predicting cytoarchitecture. Cereb. Cortex 18, 1973–1980 (2008). doi:10.1093/cercor/bhm225 Medline

56. These six regions are the early and intermediate visual areas (including V1/V2/V3/V4 and other retinotopic areas along the dorsal stream), lateral occipital cortex, inferior temporal gyrus, lingual/parahippocampal gyrus, fusiform gyrus, and entorhinal cortex.

57. S. Takahashi, Episodic-like memory trace in awake replay of hippocampal place cell activity sequences. eLife 4, e08105 (2015). doi:10.7554/eLife.08105 Medline

58. M. Moscovitch, R. Cabeza, G. Winocur, L. Nadel, Episodic Memory and Beyond: The Hippocampus and Neocortex in Transformation. Annu. Rev. Psychol. 67, 105–134 (2016). doi:10.1146/annurev-psych-113011-143733 Medline

59. J. O’Neill, B. Pleydell-Bouverie, D. Dupret, J. Csicsvari, Play it again: Reactivation of waking experience and memory. Trends Neurosci. 33, 220–229 (2010). doi:10.1016/j.tins.2010.01.006

60. A. Ben-Yakov, Y. Dudai, Constructing realistic engrams: Poststimulus activity of hippocampus and dorsal striatum predicts subsequent episodic memory. J. Neurosci. 31, 9032–9042 (2011). doi:10.1523/JNEUROSCI.0702-11.2011 Medline

61. P. Rajasethupathy, S. Sankaran, J. H. Marshel, C. K. Kim, E. Ferenczi, S. Y. Lee, A. Berndt, C. Ramakrishnan, A. Jaffe, M. Lo, C. Liston, K. Deisseroth, Projections from neocortex

6

mediate top-down control of memory retrieval. Nature 526, 653–659 (2015). doi:10.1038/nature15389 Medline

62. S. M. Polyn, V. S. Natu, J. D. Cohen, K. A. Norman, Category-specific cortical activity precedes retrieval during memory search. Science 310, 1963–1966 (2005). doi:10.1126/science.1117645 Medline

63. A. P. Vaz, S. K. Inati, N. Brunel, K. A. Zaghloul, Coupled ripple oscillations between the medial temporal lobe and neocortex retrieve human memory. Science 363, 975–978 (2019). doi:10.1126/science.aau8956 Medline

64. S. M. Smith, Fast robust automated brain extraction. Hum. Brain Mapp. 17, 143–155 (2002). doi:10.1002/hbm.10062 Medline

65. M. Jenkinson, S. Smith, A global optimisation method for robust affine registration of brain images. Med. Image Anal. 5, 143–156 (2001). doi:10.1016/S1361-8415(01)00036-6 Medline

66. M. Jenkinson, P. Bannister, M. Brady, S. Smith, Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17, 825–841 (2002). doi:10.1006/nimg.2002.1132 Medline

67. X. Papademetris, M. P. Jackowski, N. Rajeevan, M. DiStasio, H. Okuda, R. T. Constable, L. H. Staib, BioImage Suite: An integrated medical image analysis suite: An update. Insight J. 2006, 209 (2006). Medline

68. B. Fischl, FreeSurfer. Neuroimage 62, 774–781 (2012). doi:10.1016/j.neuroimage.2012.01.021 Medline

69. T. Golan, I. Davidesco, M. Meshulam, D. M. Groppe, P. Mégevand, E. M. Yeagle, M. S. Goldfinger, M. Harel, L. Melloni, C. E. Schroeder, L. Y. Deouell, A. D. Mehta, R. Malach, Human intracranial recordings link suppressed transients rather than ‘filling-in’ to perceptual continuity across blinks. eLife 5, e17243 (2016). doi:10.7554/eLife.17243 Medline

70. B. D. Argall, Z. S. Saad, M. S. Beauchamp, Simplified intersubject averaging on the cortical surface using SUMA. Hum. Brain Mapp. 27, 14–27 (2006). doi:10.1002/hbm.20158 Medline

71. A. Delorme, S. Makeig, EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21 (2004). doi:10.1016/j.jneumeth.2003.10.009 Medline

72. H. Bokil, P. Andrews, J. E. Kulkarni, S. Mehta, P. P. Mitra, Chronux: A platform for analyzing neural signals. J. Neurosci. Methods 192, 146–151 (2010). doi:10.1016/j.jneumeth.2010.06.020 Medline

7

73. K. J. Miller, C. J. Honey, D. Hermes, R. P. N. Rao, M. denNijs, J. G. Ojemann, Broadband changes in the cortical surface potential track activation of functionally diverse neuronal populations. Neuroimage 85, 711–720 (2014). doi:10.1016/j.neuroimage.2013.08.070 Medline

74. P. P. Mitra, B. Pesaran, Analysis of dynamic brain imaging data. Biophys. J. 76, 691–708 (1999). doi:10.1016/S0006-3495(99)77236-X Medline

75. P. Mitra, Observed Brain Dynamics (Oxford Univ. Press, 2007).

76. Y. Benjamini, D. Yekutieli, The control of the false discovery rate in multiple testing under dependency. Ann. Stat. 29, 1165–1188 (2001).

77. J. E. Iglesias, J. C. Augustinack, K. Nguyen, C. M. Player, A. Player, M. Wright, N. Roy, M. P. Frosch, A. C. McKee, L. L. Wald, B. Fischl, K. Van Leemput, A computational atlas of the hippocampal formation using ex vivo, ultra-high resolution MRI: Application to adaptive segmentation of in vivo MRI. Neuroimage 115, 117–137 (2015). doi:10.1016/j.neuroimage.2015.04.042 Medline

78. E. Stark, L. Roux, R. Eichler, Y. Senzai, S. Royer, G. Buzsáki, Pyramidal cell-interneuron interactions underlie hippocampal ripple oscillations. Neuron 83, 467–480 (2014). doi:10.1016/j.neuron.2014.06.023 Medline

79. E. W. Schomburg, A. Fernández-Ruiz, K. Mizuseki, A. Berényi, C. A. Anastassiou, C. Koch, G. Buzsáki, Theta phase segregation of input-specific gamma patterns in entorhinal-hippocampal networks. Neuron 84, 470–485 (2014). doi:10.1016/j.neuron.2014.08.051 Medline

80. L. D. Fiederer, J. Lahr, J. Vorwerk, F. Lucka, A. Aertsen, C. H. Wolters, A. Schulze-Bonhage, T. Ball, Electrical Stimulation of the Human Cerebral Cortex by Extracranial Muscle Activity: Effect Quantification With Intracranial EEG and FEM Simulations. IEEE Trans. Biomed. Eng. 63, 2552–2563 (2016). doi:10.1109/TBME.2016.2570743 Medline

81. J. N. Gelinas, D. Khodagholy, T. Thesen, O. Devinsky, G. Buzsáki, Interictal epileptiform discharges induce hippocampal-cortical coupling in temporal lobe epilepsy. Nat. Med. 22, 641–648 (2016). doi:10.1038/nm.4084 Medline

82. R. J. Staba, C. L. Wilson, A. Bragin, D. Jhung, I. Fried, J. Engel Jr., High-frequency oscillations recorded in human medial temporal lobe during sleep. Ann. Neurol. 56, 108–115 (2004). doi:10.1002/ana.20164 Medline

83. D. W. Scott, On Optimal and Data-Based Histograms. Biometrika 66, 605–610 (1979). doi:10.1093/biomet/66.3.605

84. L. J. P. van der Maaten, E. O. Postma, H. J. van den Herik, Dimensionality reduction: A comparative review. J. Mach. Learn. Res. 10, 66–71 (2009).

8

85. E. Levina, P. Bickel, Maximum likelihood estimation of intrinsic dimension. Adv. Neural Inf. Process. Syst. 17, 777–784 (2004).

86. A. S. Ghuman, N. M. Brunet, Y. Li, R. O. Konecky, J. A. Pyles, S. A. Walls, V. Destefino, W. Wang, R. M. Richardson, Dynamic encoding of face information in the human fusiform gyrus. Nat. Commun. 5, 5672 (2014). doi:10.1038/ncomms6672 Medline

87. C. Jacques, N. Witthoft, K. S. Weiner, B. L. Foster, V. Rangarajan, D. Hermes, K. J. Miller, J. Parvizi, K. Grill-Spector, Corresponding ECoG and fMRI category-selective signals in human ventral temporal cortex. Neuropsychologia 83, 14–28 (2016). doi:10.1016/j.neuropsychologia.2015.07.024 Medline

88. D. M. Groppe, T. P. Urbach, M. Kutas, Mass univariate analysis of event-related brain potentials/fields I: A critical tutorial review. Psychophysiology 48, 1711–1725 (2011). doi:10.1111/j.1469-8986.2011.01273.x Medline

89. R. Oostenveld, P. Fries, E. Maris, J.-M. Schoffelen, FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell. Neurosci. 2011, 156869 (2011). doi:10.1155/2011/156869 Medline

90. E. Maris, R. Oostenveld, Nonparametric statistical testing of EEG- and MEG-data. J. Neurosci. Methods 164, 177–190 (2007). doi:10.1016/j.jneumeth.2007.03.024 Medline

91. Images shown to patients are protected by copyright. Images presented in the figure are similar substitutes. Photo of Bill Clinton courtesy of Gage Skidmore; photo of the Golden Gate Bridge courtesy of Nicolas Raymond; photo of the Leaning Tower of Pisa courtesy of Josu; photo of Barack Obama courtesy of the U.S. Government. All pictures are published under a Creative Commons license.