Reading Tobii Data (from Titta)

Reading Tobii Data (from Titta)#

Titta is a package that allows Tobii eye-trackers to work with PsychoPy experiments. Different from Tobii Pro Lab, Titta captures and saves raw data streams to a HDF5 file, including gaze samples, event messages, calibration info, etc.

Create TobiiTittaReader Object#

Similar to reading Eyelink data, the first step is to pass the necessary info to a TobiiTittaReader object. Here we use a data file provided by the Titta package as an example.

In this example, a participant viewed 3 pictures with eye movements recorded. Trial boundaries were marked in the following format: [marker]_[stimulus] (e.g., onset_im3.jpeg, offset_im3.jpeg)

import pupeyes as pe

# file name
path = './data/participant1.h5'

# event marker format, specified as a dictionary {name: data type}   
msg_format = {'marker':str, 'stimulus':str} # e.g., onset_im3.jpeg -> marker: onset, stimulus: im3.jpeg
delimiter = '_' # delimiter for messages

# start and stop notations for each trial
start_msg = 'onset'
stop_msg = 'offset'

# If you have any constant columns that you want to add
add_cols = {'subject':'participant1'}

# pass the necessary info to a `TobiiTittaReader` object
raw = pe.TobiiTittaReader(path=path, 
                          start_msg=start_msg, 
                          stop_msg=stop_msg, 
                          msg_format=msg_format, 
                          delimiter='_',
                          add_cols=add_cols)

Get Gaze Samples#

Once a TobiiTittaReader is created, getting gaze samples is easy.

samples = raw.get_samples()

# select columns of interest
use_cols = ['subject','marker','stimulus','trialtime','system_time_stamp','left_gaze_point_on_display_area_x','left_gaze_point_on_display_area_y','left_pupil_diameter','right_gaze_point_on_display_area_x','right_gaze_point_on_display_area_y','right_pupil_diameter']
samples[use_cols].head()
subject marker stimulus trialtime system_time_stamp left_gaze_point_on_display_area_x left_gaze_point_on_display_area_y left_pupil_diameter right_gaze_point_on_display_area_x right_gaze_point_on_display_area_y right_pupil_diameter
0 participant1 onset im3.jpeg 0 727746551321 0.499874 0.479949 2.643646 0.499563 0.494971 2.762421
1 participant1 onset im3.jpeg 2 727746552988 0.500004 0.482752 2.635223 0.498788 0.495276 2.764008
2 participant1 onset im3.jpeg 3 727746554655 0.500121 0.478806 2.636536 0.497905 0.501001 2.771042
3 participant1 onset im3.jpeg 5 727746556321 0.498919 0.482849 2.635376 0.498185 0.497359 2.75914
4 participant1 onset im3.jpeg 7 727746557988 0.500586 0.487291 2.639938 0.497897 0.498819 2.765289

Note

The column names are different from those in the Eyelink data. This is an intentional choice to preserve original column names defined by Titta.

Get Custom Messages#

You can also get the custom messages that define your trials to use in further analyses.

messages = raw.get_messages()
messages.head()
id system_time_stamp msg marker stimulus subject
0 0 727746550762 onset_im3.jpeg onset im3.jpeg participant1
1 0 727749532869 offset_im3.jpeg offset im3.jpeg participant1
2 1 727749549502 onset_im2.jpeg onset im2.jpeg participant1
3 1 727752532634 offset_im2.jpeg offset im2.jpeg participant1
4 2 727752549288 onset_im1.jpeg onset im1.jpeg participant1

Get Fixations#

Different from Eyelink, data streams provided by Tobii SDK do not include fixations, saccades, or blinks. As such, PupEyes cannot extract them. To get these events, users may apply open-source event-detection algorithms to the raw gaze data.

The Titta package included an example of using the I2MC algorithm to get fixations. Here is how you can do it in the context of PupEyes.

import I2MC # pip install I2MC if not installed
import pandas as pd
import numpy as np

### These configurations are copied from https://github.com/marcus-nystrom/Titta/blob/master/demo_analyses/detect_fixations.py

# =============================================================================
# NECESSARY VARIABLES
# =============================================================================
opt = {}
# General variables for eye-tracking data
opt['xres']         = 1920.0                # maximum value of horizontal resolution in pixels
opt['yres']         = 1080.0                # maximum value of vertical resolution in pixels
opt['missingx']     = -opt['xres']          # missing value for horizontal position in eye-tracking data (example data uses -xres). used throughout the algorithm as signal for data loss
opt['missingy']     = -opt['yres']          # missing value for vertical position in eye-tracking data (example data uses -yres). used throughout algorithm as signal for data loss
opt['freq']         = 600.0                 # sampling frequency of data (check that this value matches with values actually obtained from measurement!)

# Variables for the calculation of visual angle
# These values are used to calculate noise measures (RMS and BCEA) of
# fixations. The may be left as is, but don't use the noise measures then.
# If either or both are empty, the noise measures are provided in pixels
# instead of degrees.
opt['scrSz']        = [52.7, 30]    # screen size in cm
opt['disttoscreen'] = 63.0                  # distance to screen in cm.

# =============================================================================
# OPTIONAL VARIABLES
# =============================================================================
# The settings below may be used to adopt the default settings of the
# algorithm. Do this only if you know what you're doing.

# # STEFFEN INTERPOLATION
opt['windowtimeInterp']     = 0.1                           # max duration (s) of missing values for interpolation to occur
opt['edgeSampInterp']       = 2                             # amount of data (number of samples) at edges needed for interpolation
opt['maxdisp']              = opt['xres']*0.2*np.sqrt(2)    # maximum displacement during missing for interpolation to be possible

# # K-MEANS CLUSTERING
opt['windowtime']           = 0.2                           # time window (s) over which to calculate 2-means clustering (choose value so that max. 1 saccade can occur)
opt['steptime']             = 0.02                          # time window shift (s) for each iteration. Use zero for sample by sample processing
opt['maxerrors']            = 100                           # maximum number of errors allowed in k-means clustering procedure before proceeding to next file
opt['downsamples']          = [2, 5, 10]
opt['downsampFilter']       = True                         # use chebychev filter when downsampling? Its what matlab's downsampling functions do, but could cause trouble (ringing) with the hard edges in eye-movement data

# # FIXATION DETERMINATION
opt['cutoffstd']            = 2.0                           # number of standard deviations above mean k-means weights will be used as fixation cutoff
opt['onoffsetThresh']       = 3.0                           # number of MAD away from median fixation duration. Will be used to walk forward at fixation starts and backward at fixation ends to refine their placement and stop algorithm from eating into saccades
opt['maxMergeDist']         = 30.0                          # maximum Euclidean distance in pixels between fixations for merging
opt['maxMergeTime']         = 30.0                          # maximum time in ms between fixations for merging
opt['minFixDur']            = 40.0                          # minimum fixation duration after merging, fixations with shorter duration are removed from output
opt['chebyOrder']           = 8

# Change parameters according to the recommendations on Github
if opt['freq'] == 120:
    opt['downsamples']          = [2, 3, 5]
    opt['chebyOrder']           = 7

if opt['freq'] < 120:
    opt['downsamples']          = [2, 3]
    opt['downsampFilter']       = False

Once we set up the options, we reformat the data so it can be accepted by I2MC.

# Reference: https://github.com/marcus-nystrom/Titta/blob/master/demo_analyses/import_funcs.py

# # screen resolution
res = [1920, 1080]  

# create columns required by I2MC
samples['time'] = samples['trialtime']
samples['L_X']  = samples['left_gaze_point_on_display_area_x']  * res[0]
samples['L_Y']  = samples['left_gaze_point_on_display_area_y']  * res[1]
samples['R_X']  = samples['right_gaze_point_on_display_area_x'] * res[0]
samples['R_Y']  = samples['right_gaze_point_on_display_area_y'] * res[1]

# select columns of interest
I2MC_cols = ['subject','marker','stimulus','time','L_X','L_Y','R_X','R_Y']
samples_I2MC = samples[I2MC_cols]
samples_I2MC.head()
subject marker stimulus time L_X L_Y R_X R_Y
0 participant1 onset im3.jpeg 0 959.758789 518.345215 959.161865 534.568542
1 participant1 onset im3.jpeg 2 960.006836 521.372498 957.672241 534.89856
2 participant1 onset im3.jpeg 3 960.232544 517.110229 955.976929 541.08075
3 participant1 onset im3.jpeg 5 957.92511 521.477173 956.514954 537.147766
4 participant1 onset im3.jpeg 7 961.124634 526.27478 955.961487 538.724243

We create a simple wrapper for the I2MC function. Then, we take advantage of pandas .groupby() to apply I2MC to each group.

def run_I2MC(samples, options):
    fix,_,_ = I2MC.I2MC(samples.dropna(), options=options)
    return pd.DataFrame(fix)

fixations = samples_I2MC.groupby(['subject','marker','stimulus'])[I2MC_cols].apply(run_I2MC, options=opt)
fixations.reset_index(inplace=True)
fixations.head()

I2MC: Searching for valid interpolation windows
I2MC: Replace interpolation windows with Steffen interpolation
I2MC: 2-Means clustering started for left eye signal
I2MC: 2-Means clustering started for right eye signal

I2MC: Determining fixations based on clustering weight mean for averaged signal and separate eyes + 2.00*std
I2MC: Searching for valid interpolation windows
I2MC: Replace interpolation windows with Steffen interpolation
I2MC: 2-Means clustering started for left eye signal

I2MC: 2-Means clustering started for right eye signal

I2MC: Determining fixations based on clustering weight mean for averaged signal and separate eyes + 2.00*std
I2MC: Searching for valid interpolation windows
I2MC: Replace interpolation windows with Steffen interpolation
I2MC: 2-Means clustering started for left eye signal

I2MC: 2-Means clustering started for right eye signal

I2MC: Determining fixations based on clustering weight mean for averaged signal and separate eyes + 2.00*std
subject marker stimulus level_3 cutoff start end startT endT dur xpos ypos flankdataloss fracinterped RMSxy BCEA fixRangeX fixRangeY
0 participant1 onset im1.jpeg 0 0.858637 0 54 0 92 92 1071.743774 584.906555 0.0 0.0 0.103671 0.024970 0.197996 0.343122
1 participant1 onset im1.jpeg 1 0.858637 98 169 165 285 120 534.102539 246.847778 0.0 0.0 0.133866 0.115515 0.844065 0.573339
2 participant1 onset im1.jpeg 2 0.858637 196 272 328 457 129 742.064209 420.482605 0.0 0.0 0.124024 0.035410 0.223386 0.496848
3 participant1 onset im1.jpeg 3 0.858637 288 442 482 740 258 783.386963 487.159485 0.0 0.0 0.115604 0.040202 0.386121 0.475640
4 participant1 onset im1.jpeg 4 0.858637 448 522 748 873 125 789.429810 532.545044 0.0 0.0 0.088122 0.024529 0.211964 0.539856

Note

Refer to the I2MC package to see what these columns mean. There is slight discrepancy in the data for “im1.jpeg’ if you compare these data with those generated by detection_fixations.py in the Titta example. This is likely due to dropna() in the run_I2MC function dropping one row with missing values for that stimulus in the current implementaiton.

Pupil Preprocessing#

You can initiate PupilProcessor similarly to Eyelink data. A couple of caveats, however:

  • Different eye-trackers use different values to indicate missing pupil sizes. Tobii seems to use NaN. This needs to be specified by passing pd.NA or np.nan to eyetracker_missing_value.

  • time_col expects integer values in milliseconds which is why we used trialtime here.

  • Sampling rate check is not performed for Tobii eye-trackers, as they seem to sometimes produce erratic timestamps.

The user is responsible for making sure timestamps and pupil size values are in the correct format expected by PupEyes.

samples.head()[['subject','marker','stimulus','trialtime','left_pupil_diameter','L_X','L_Y']]
subject marker stimulus trialtime left_pupil_diameter L_X L_Y
0 participant1 onset im3.jpeg 0 2.643646 959.758789 518.345215
1 participant1 onset im3.jpeg 2 2.635223 960.006836 521.372498
2 participant1 onset im3.jpeg 3 2.636536 960.232544 517.110229
3 participant1 onset im3.jpeg 5 2.635376 957.92511 521.477173
4 participant1 onset im3.jpeg 7 2.639938 961.124634 526.27478 
# the one row with missing pupil size
# user needs to specify the missing value for the eye tracker
samples[samples.L_X.isnull()][['subject','marker','stimulus','trialtime','left_pupil_diameter','L_X','L_Y']]
subject marker stimulus trialtime left_pupil_diameter L_X L_Y
3652 participant1 onset im1.jpeg 122 <NA> <NA> <NA> 
p = pe.PupilProcessor(
    data=samples, # pass your data
    trial_identifier=['subject','marker','stimulus'], # column(s) that disambiguate one trial from another in your data
    x_col = 'L_X',
    y_col = 'L_Y',
    pupil_col = 'left_pupil_diameter',
    time_col = 'trialtime',
    samp_freq = 600, # Hz
    device='tobii_titta',
    eyetracker_missing_value=pd.NA # <NA> is the missing value for the eye tracker
    )

Device: tobii_titta
Eye-tracker missing value is <NA>. Replacing with 0.
Sampling frequency check skipped for tobii_titta data.
PupilProcessor initialized with 5369 samples
Pupil column: left_pupil_diameter, Time column: trialtime, X column: L_X, Y column: L_Y
Trial identifier: ['subject', 'marker', 'stimulus'], Number of trials: 3

# do preprocessing
p.deblink().artifact_rejection().smooth().check_missing().interpolate(missing_threshold=.4).baseline_correction(baseline_query='marker=="onset"', baseline_range=[0,100])

Running deblink using sampling frequency 600Hz

✓ Deblinking completed!
  → New column: 'left_pupil_diameter_db' (blinks removed)
  → Previous column 'left_pupil_diameter' preserved.
  → 0 trial(s) failed.

✓ Artifact rejection completed!
  → New column: 'left_pupil_diameter_db_ar' (artifacts removed)
  → Previous column 'left_pupil_diameter_db' preserved.
  → 0 trial(s) failed.

✓ Smoothing completed!
  → New column: 'left_pupil_diameter_db_ar_sm' (smoothed)
  → Previous column 'left_pupil_diameter_db_ar' preserved.
  → 0 trial(s) failed.

✓ Missing values checked!
  → 0 trial(s) failed.

✓ Interpolation completed!
  → New column: 'left_pupil_diameter_db_ar_sm_it' (interpolated)
  → Previous column 'left_pupil_diameter_db_ar_sm' preserved.
  → 0 trial(s) failed.

✓ Baseline correction completed!
  → New column: 'left_pupil_diameter_db_ar_sm_it_bc' (baseline corrected)
  → Previous column 'left_pupil_diameter_db_ar_sm_it' preserved.
  → 0 trial(s) failed.

<pupeyes.pupil.PupilProcessor at 0x702200c3d880>

# summary data
p.summary()
subject marker stimulus n_samples run_deblink pct_deblink run_speed pct_speed run_size pct_size run_smooth run_check_missing missing run_interpolate pct_interpolate run_baseline_correction baseline
0 participant1 onset im3.jpeg 1789 True 0.0 True 0.0 True 0 True True 0.055338 True 0.055338 True 2.625614
1 participant1 onset im2.jpeg 1790 True 0.0 True 0.003352 True 0 True True 0.218994 True 0.218994 True 2.581183
2 participant1 onset im1.jpeg 1790 True 0.004469 True 0.0 True 0 True True 0.1 True 0.1 True 2.75396 
# inspect results
viewer = pe.PupilViewer(p)

#viewer.run()
Contents