Participants and procedures
Simultaneous EEG-fMRI was collected in twenty-two adults (ages 23–51 years; mean age: 36.8; 50% male) recruited from the Rockland County, NY community. Participants enrolled in this study have no history of psychiatric or neurological illnesses. All imaging was collected using a 3 T Siemens TrioTim equipped with a 12-channel head coil. EEG data were collected using an MR-compatible system by Brain Products consisting of the BrainCap MR with 64 channels, two 32-channel BrainAmp MR amplifiers, and a PowerPack battery. Cortical electrodes were arranged according to the international 10–20 system. Inside the scanner, eye tracking was collected in the left eye using the EyeLink 1000 Plus.
Participants attended two sessions between 2 and 354 days between scans (time between scans, mean: 38.2 days; median: 11 days); see Table 1 for the breakdown of data acquired during sessions. The scanning protocol consisted of three recording settings. The “Outside” setting was an EEG recording collected outside the MRI scanner in a non-shielded room; the “Scanner OFF” setting consisted of EEG recordings collected inside the static field of the MRI scanner while the scanner was off; the “Scanner ON” setting consisted the simultaneous EEG and fMRI recordings. All research performed was approved by NKIs Institutional Review Board (IRB# 941632). Prior to the experiment, written informed consent was obtained from all participants. Participants also provided demographic information and behavioral data, including information on their last month of sleep (Pittsburgh Sleep Study)23, the amount of sleep they had the previous night, and their caffeine intake before the scan session.
EEG acquisition
The entire procedure of data collection takes approximately three hours. About 45 minutes are spent preparing a participant for EEG. Preparation begins with measuring the participant’s head to determine cap size. An anterior to posterior measurement is also taken across the top of the head, from the nasion to the inion. A mark is made at the center of the forehead 10% of the measured nasion-inion distance. A proper sized cap is placed on the participant’s head with the Fpz electrode centered at the mark on the forehead. Once the subject is fitted with the EEG cap, electrodes are filled with electrolyte gel. In this study, EEG is collected using a customized cap to record 61 cortical channels, two electrooculogram (EOG) channels placed above (channel 64) and below the left eye (channel 63), and one electrocardiography (ECG) channel (channel 32) placed on the back. In addition, the cap also contains a reference and ground electrode. Electrodes were filled using V19 Abralyt HiCl electrode gel. Electrode impedance was recorded before every recorded run; to ensure good data quality, electrode impedance was kept below 20kOhm. EEG was recorded using BrainVision Recorder at a sampling rate of 5 kHz. After cap preparation, participants completed a single run of the flickering checkerboard experiment in the “Outside” scan condition viewing a 19-inch LCD monitor. After the Outside scan, a 3D scan of the participant’s head is collected to digitize the position of EEG electrodes. 3D scans were collected using a portable scanner, the Occipital Structure Sensor (Occipital Inc, Boulder CO), and iPad Mini 4 (Apple, Cupertino, CA). Due to protected health information (PHI) restrictions, 3D scans will not be available in the data release; however, location files containing the positions of the electrodes will be provided with this release.
Simultaneous EEG-fMRI recording in the MRI scanner
After 3D digitization, participants enter the MRI scanner and are placed on the scanner bed in the supine position. Cushions are placed around the head to provide stabilization and minimize head motion during scans. At this stage, participants are provided with MR-safe goggles if they have any visual impairment that requires glasses. The screen is rear projected at the end of the MRI scanner at 1300 mm to the mirror mounted on the head coil. The videos are shown on a projector screen with a size of 440 × 330 mm with a 1024 × 768 resolution. This creates a horizontal and vertical viewing angle of 19.35° and 14.51° respectively, with a resolution of 0.0189 degrees/pixel for both directions. The light in the room was kept on during the imaging session. Participants were also fitted with a respiratory transducer belt to monitor breathing, which was recorded using BIOPAC MP150 (BIOPAC Systems, Inc., Goleta, CA).
Once positioned in the scanner, the participant’s head is enclosed in a 12-channel 3 T head matrix MR coil. The cable bundle (flat ribbon) from the EEG cap is routed through the front of the head coil and fixed at the top of the head coil using medical tape. The cable bundles are connected to the amplifiers and battery pack. During EEG data acquisition, the software is synchronized to the master clock of the MRI scanner. During recordings, EEG data was continuously collected along with task onset triggers, and volume triggers were recorded at the beginning of each TR. For specifics on the equipment and connections for simultaneous recordings, see Fig. 1 and Table 2.
Schematic of EEG-fMRI setup.
Eye tracking acquisition
For recordings collected inside the scanner, eye position and pupil dilation were recorded using an infrared-based eye tracker (EyeLink 1000 Plus, SR Research Ltd., Ontario, Canada; http://www.sr-research.com) at a sampling rate of 1000 Hz. Prior to release, data were down sampled to 250 Hz. The eye tracker was calibrated using a 9-point grid before recordings in the MRI scanner. Participants were asked to direct their gaze at dots presented on the grid. Calibration was followed by a validation step until the error between the two measurements was less than 1° 24.
MRI data acquisition
MRI data were acquired using a 12-channel head coil on 3.0 T Siemens TIM Trio. MPRAGE structural T1w images were acquired with the following parameters: TR = 2500 ms; TI = 1200 ms; TE = 2.5 ms; slices = 192; matrix size = 256 × 256; voxel size = 1 mm3 isotropic; flip angle = 8°; partial Fourier off; pixel bandwidth = 190 Hz/Px. All BOLD fMRI sequences were acquired with these parameters: TR = 2100 ms; TE = 24.6 ms; Flip Angle = 60°; slices = 38; matrix size = 64 × 64; voxel size = 3.469 × 3.469 × 3.330 mm. The run length for each task is listed in Table 1.
Task data/stimuli description
In this section, we describe the data generated for this study focused on collecting simultaneous EEG and fMRI. This dataset consists of a task, naturalistic stimuli, and resting state data. In addition to the simultaneously collected data, task data was collected outside the MRI scanner and inside the scanner environment with the scanner off. The collection of this data enables us to assess the impact of changes in the scanning environment on the EEG recordings. EEG-fMRI data were collected across two scan sessions; structural data was collected during the middle of the scan (see Table 1 for details). Code for presenting task stimuli and naturalistic stimuli, along with code to preprocess EEG and fMRI imaging data is available on GitHub (https://github.com/NathanKlineInstitute/NATVIEW_EEGFMRI).
Checkerboard stimulus
The use of flickering visual stimuli has been used to investigate the visual system in EEG25,26 and fMRI27,28. A high-contrast flickering checkerboard was used to stimulate the primary visual cortical regions. Participants were shown a flickering radial checkerboard at a frequency of 12 Hz in 20-second trials following a 20-second rest period across five repetitions. The checkerboard stimulus was presented outside the scanner after cap placement, inside the scanner with the scanner off, and inside the scanner during a simultaneous EEG and fMRI. These three recordings were collected to measure the impact of the MRI scanning sequence on the recorded EEG.
Rest
The participant is presented with a white fixation cross in the center of a black screen and instructed to rest with eyes open. Participants had one rest scan per session, and each scan had a duration of 10 minutes.
Inscapes
Inscapes is a computer-generated animation featuring abstract 3D shapes and moves in slow continuous transitions. The video was originally developed as a 7-minute video for children to watch during brain scans as a means to provide stimulation to keep them engaged while minimizing some cognitive processing that may be engaged19. Participants were presented with an extended version of Inscapes that was 10 minutes long. Similar to the rest scan, the Inscapes video was viewed once per scan session.
Predictive eye estimation regression (PEER) calibration
Predictive eye estimation regression (PEER) is an imaging-based calibration scan used to estimate the direction of eye gaze29 and here can be used as a complement to the optical eye tracking of the Eyelink 1000. Participants were told to direct their gaze at dots that appeared at predefined points on the screen. The PEER method estimates eye gaze from the collected scan using support vector regression (SVR). The algorithm estimates the direction of eye gaze during each repetition (TR) in the fMRI time series.
Naturalistic stimuli (Movies)
Participants viewed three movies twice in one scan session. Videos varied between 258 s and 600 s. Naturalistic stimuli included “The Present” [4m18s] (uploaded to YouTube 7 Feb 2016)30, two 10-minute clips from “Despicable Me” [clips taken from Russian Blu-Ray with exact times 1:02:09-1:12:09 (English) and 0:12:12-0:22:12 (Hungarian)]31, and three 5-minute monkey videos32. The monkey videos are part of a database with multiple videos33; for this study, Monkey 1, Monkey 2, and Monkey 5 represent the first, second, and fifth videos in the database, respectively. The videos used in this study are available to download in the GitHub repository (https://github.com/NathanKlineInstitute/NATVIEW_EEGFMRI/tree/main/stimulus).
Limitations
The fMRI imaging in this dataset was collected using a 12-channel head coil on a 3 T scanner platform. While we possess the ability to scan at 3 T with a 32-channel head coil, the head coil design does not allow for the EEG cap cable bundle to be routed perpendicularly from the top of the head. Moreover, the design of the head coil does not permit alternate routing of the cable bundle because it will block the participant’s eyes or the cable length is too short to reach the amplifiers. In this study, the cable bundle is routed through the front and then taped to the top of the head coil before it is connected to the amplifier. Using a 12-channel head coil limited the sequences that could be used to collect fMRI, including the use of multiband sequences. Another limiting factor is that imaging sequences with faster TRs could not be used while collecting data simultaneously. The main issue is a safety concern regarding radiofrequency (RF) power deposition that causes heating in the EEG leads and electrodes during a sequence34. In this study, we used a TR of 2100 ms to ensure participants were not at risk of discomfort or burns due to the heating of electrodes.
We have not been able to calculate the delay between the stimulus computer and the projector, due to lack of equipment to conduct the appropriate measurement (photodiode)35. If in the future we do purchase this equipment and are able to perform this measurement, this information will be posted on the study website.
EEG preprocessing
We developed an automated pipeline to preprocess all EEG data collected outside and inside the MRI scanner. The preprocessing methods used on the EEG data depended on where data were acquired. All EEG data were preprocessed using EEGLAB and associated plugins36. For data collected inside the scanner, data were preprocessed using the FMRIB plug-in for EEGLAB, provided by the University of Oxford Centre for Functional MRI of the Brain (FMRIB)37,38.
For data collected in the Outside setting, the following preprocessing steps were used: (i) bandpass filter using a Hamming windowed sinc FIR filter between 0.5 Hz and 70 Hz; (ii) reference electrodes using average reference, excluding the ECG channel, EOG channels, and electrodes excluded during the EEG quality control process.
For data collected in the Scanner OFF setting, the initial preprocessing steps used in the Outside setting were used. In addition, pulse artifact detection and removal were used due to the increased contamination of the signal caused by the participant’s heartbeat3: (i) QRS/heartbeat detection using the ECG channel; (ii) pulse artifact/BCG removal using template subtraction based on the median artifact; (iii) bandpass filter using a Hamming windowed sinc FIR filter between 0.5 Hz and 70 Hz; (iv) reference electrodes using average reference, excluding the ECG channel and EOG channels.
For data collected in the Scanner ON setting, the initial preprocessing steps used in the Scanner OFF setting were used. In addition, gradient artifact removal was used to remove the contamination of the EEG signal caused by the changing gradients from the fMRI pulse sequence37: (i) gradient artifact removal; (ii) QRS/heartbeat detection using the ECG channel; (iii) pulse artifact/BCG removal using template subtraction based on the median artifact; (iv) bandpass filter using a Hamming windowed sinc FIR filter between 0.5 Hz and 70 Hz; (v) reference electrodes using average reference, excluding the ECG channel and EOG channels.
Gradient artifact removal
The gradient artifact is the most significant noise source in simultaneous EEG-fMRI data, measuring more than 400 times larger than the lowest amplitude EEG events3. The FMRIB plugin uses the FASTR method to remove gradient artifacts38. The method requires recording the scanner trigger at the start of each TR. An average template is computed from the detected TRs and subtracted from the raw EEG data. Following this process, the data is corrected further using principal component analysis (PCA) to reduce residual artifacts. Residual artifacts are further reduced using adaptive noise cancellation3.
Pulse artifact/BCG removal
ECG data collected inside the MRI scanner has a pronounced T wave that increases as the field strength increases39. The FMRIB plugin identifies these QRS events using an algorithm that detects events, aligns them, and corrects for false positives and negatives40,41. A median signal is computed from the events to create an artifact template, which is subsequently subtracted from the data.
MRI preprocessing
We used the Connectome Computation System (CCS) to preprocess the MRI/fMRI data42. For the anatomic data, we performed skull stripping using a combination of Brain Extraction Toolbox (BET) and Freesurfer. Data was then segmented (Freesurfer) and registered to a template space (MNI152 2006) using FLIRT and MCFLIRT43,44. The runs for the fMRI data were all preprocessed equally. Initially, the first five volumes are discarded, then the data is despiked and slice time and motion corrected. The functional data is skull striped with 3dAutomask, refined using the structural data, and registered to the anatomical images using boundary-based registration based on N4 using Freesurfer. Nuisance correction is done using the Friston 24 motion parameters, average CSF, and WM signals, with/without global signal regression (GSR). The data is also processed with/without temporal filtering (0.01–0.1 Hz) and with/without a 6 mm FWHM spatial filter. Time series were extracted from 400 ROIS (Schaefer 400 atlas) to be further processed45.
Data quality control
EEG
To assess the quality of the EEG data in this study, we followed the quality control pipeline similar to that described in Delorme et al.46. This approach yields three metrics related to data quality: (i) percent of “good” channels; (ii) percent of “good” trials; and (iii) number of independent components (ICs) related to brain source activity. From this pipeline, “good” channels are defined as those that remain after completing the related preprocessing steps: removal of channels with more than five seconds of non-activity, with signal greater than four standard deviations due to high-frequency noise, or Pearson’s correlation coefficient less than 0.7 with nearby channels. Accordingly, “good” trials are related to the data periods that are not contaminated by artifacts such as body movement. In this study, we removed data segments with a variance higher than twenty times the variance of the calibration data. Finally, independent component analysis (ICA) was computed using the RunICA plugin for the EEGLAB toolbox to produce ICs47. Afterward, the ICLabel plugin was used to identify ICs belonging to brain source activity48. The resulting metric calculates the percentage of ICs associated with brain source activity divided by the total number of ICs found from ICA.
MRI
Temporal measures of fMRI data include median and median framewise displacement44, root mean square of the temporal change (DVARS), and temporal signal-to-noise ratio (tSNR).
Statistical analysis
Permutation testing offers a robust framework for statistical significance assessment in EEG analysis. Multiple permutation testing was performed on the flickering checkerboard task to identify differences between the two task conditions: the rest and flickering checkerboard blocks. This analysis is used to find statistical relevance that identifies differences underlying the EEG data. One advantage of multiple permutation testing is that it does not require the same number of trials for each condition. In our permutation testing, trials were shuffled into two new groups, followed by the calculation of a paired t-test. Finally, pixel-based multiple comparisons correction was applied to reduce the familywise error rate49.
Data privacy
All imaging data in this release has been de-identified, removing any personal identifying information (as defined by the Health Insurance Portability and Accountability) from data files, including facial features. Facial features from the T1w MRIs were removed using the “mri_deface” software package developed by Bischoff-Grethe et al.50. Data and code are shared under the CC BY 4.0 license.
