API Docs (Objective-C)

GazeTracker

@interface GazeTracker : NSObject

GazeTracker is the class that generates gaze tracking data from the video from the device's front camera.

Construction

initGazeTracker

+ (void)initGazeTrackerWithLicense:(NSString * _Nonnull)license delegate:(id <InitializationDelegate> _Nonnull)delegate option:(UserStatusOption * _Nullable)option

GazeTracker constructing process includes authentication.

The process is asynchronous.

InitializationDelegate will relay constructed object.

• parameter

  • license:

    • type: NSString *
    • description: Key generated from manage.seeso.io

  • delegate

    • type: id <InitializationDelegate>
    • description: Delegate represents initialization status

  • option

    • type: UserStatusOption * _Nullable
    • description: A class containing User status option information (optional)

• return: void

• usage:

  [GazeTracker initGazeTrackerWithLicense:@"key" delegate:self];
  
  UserStatusOption * option = [[UserStatusOption alloc] init];
  [option useAll];
  [GazeTracker initGazeTrackerWithLicense:@"key" delegate:self, option: option];

Destruction

deinitGazeTracker

+ (void)deinitGazeTrackerWithTracker:(GazeTracker * _Nullable)tracker;

After the destruction, every API will be disabled. For memory optimization, assigning nil to the destructed object is recommended.

• parameter

  • tracker

    • type: GazeTracker *
    • description: Target GazeTracker instance of destruction

• return: void

• usage:

  [GazeTracker deinitGazeTrackerWithTracker:tracker];
  tracker = nil;

Gaze Tracking

startTracking

- (void)startTracking;

This is the function that should be called before starting gaze tracking. It calls the StatusDelegate.onStarted() when succeeded.

• parameter: X

• return: void

• usage:

  [tracker startTracking];

stopTracking

- (void)StopTracking;

This function is called to stop gaze tracking. It calls the StatusDelegate.onStop() object when succeed.

• parameter: X

• return: void

• usage:

  [tracker stopTracking];

isTracking

- (BOOL) isTracking

This function represents the status of gaze tracking. It returns true when gaze tracking is working, false when gaze tracking is stopped.

• parameter: X

• return: BOOL

• usage:

  BOOL isTracking = [tracker isTracking];
  NSLog("Tracking status: %d", isTracking);

isCalibrating

- (BOOL) isCalibrating

It returns true when calibrating, false when when not calibrating.

• parameter: X

• return: BOOL

• usage:

  BOOL isCalibrating = [tracker isCalibrating];
  NSLog("Calibrating status: %d", isCalibrating);

setTrackingFPS

- (BOOL)setTrackingFPSWithFps:(NSInteger)fps

The parameter that sets the FPS of the gaze tracking source. Its value should bigger than 0 and no more than 30.

If the value is out of range, it will return false. FPS can be dropped due to device spec. The default value is 30.

If state is TrackingState.FACE_MISSING, FPS will set as 30 automatically.

• parameter

  • fps

    • type: NSInteger
    • description: Custom FPS(Frame Per Second) for gaze tracking

• return: BOOL

• usage:

  [tracker setTrackingFPSWithFps:20];

UserStatus Setting

setAttentionInterval

    - (void)setAttentionIntervalWithInterval:(NSInteger)interval;

Set time interval for the UserStatus Attention callback.

The UserStatus Attention score will be calculated for the given time interval.

The beginning and ending timestamps are passed through the onAttention callback as timestampBegin and timestampEnd.

The interval range is 10 to 60 (seconds), and the default value is 30 seconds.

• parameter

  • interval

    • type: NSInteger
    • description: Time interval for the UserStatus Attention score.

• return: void

• usage:

    [tracker setAttentionInervalWithInterval:20];

Get UserStatus Data

getAttentionScore

    - (double)getAttentionScore

Get current Attention score from the GazeTracker.

This API does not provide timestamp of the Attention score data.

Use onAttention callback, unless the Attention score is required at a specific time/location.

• parameter: X

• return: double

• usage:

    double score = [tracker getAttentionScore];

Calibration

startCalibration

- (BOOL)startCalibrationWithMode:(enum CalibrationMode)mode criteria:(enum AccuracyCriteria)criteria region:(CGRect)region

This will return true when the parameters are valid.

It also returns false when calibration region is out of screen.

The calibration process will stop when false has been returned.

• parameter

  • mode

    • type: enum CalibrationMode
    • description: Can select calibration option. Three options — DEFAULT(0), ONE_POINT(1), FIVE_POINT(2) — are available. Default is FIVE_POINT.
    • default: DEFAULT(0)

  • region

    • type: CGRect
    • description: Region that needs calibration. The unit is point(pt).
    • default: Full screen region of device.

  • criteria

    • type: enum AccuracyCriteria
    • description: Option that manage calibration process and accuracy. Three options - DEFAULT(0), LOW(1), HIGH(2) - are available. Default is DEFAULT.
    • default: DEFAULT(0)

• return: BOOL

• usage:

  BOOL startResult = [tracker startCalibrationWithMode: CalibrationModeONE_POINT AccuracyCriteriaDEFAULT region:[self.view frame]];

stopCalibration

- (void)stopCalibration;

• parameter: X

• return: void

• usage:

  [tracker stopCalibration];

startCollectSamples

- (BOOL)startCollectSamples;

This relays the coordinates of the point that should be seen when calibration is in progress at the function: CalibrationDelegate.onCalibrationNextPoint.

If startCollectionSamples calling timing went wrong, it will return false You should display coordinates on the screen and call startCollectSamples for calibration.

Also, you can re-initialize calibration progress by calling startCollectSamples during the calibration progress.

• parameter: X

• return: BOOL

• usage:

  [tracker startCollectSamples];

setCalibrationData

- (BOOL)setCalibrationDataWithCalibrationData:(NSArray<NSNumber *> * _Nonnull)calibrationData

Set existing calibration data to gazeTracker.

• parameter

  • calibrationData

    • type: NSArray<NSNumber *> * _Nonnull
    • description: Calibration Data

• return: BOOL

• usage:

  if([tracker setCalibrationDataWithCalibrationData: data]){
      NSLog(@"Loading of calibration data was successful.");
  }else {
      NSLog(@"Failed to load calibration data.");
  }

Camera Preview

setCameraPreview

- (void)setCameraPreviewWithPreview:(UIView * _Nonnull)preview;

This is a preview of the camera that GazeTracker using. You should rotate the screen if landscape mode

• parameter

  • cameraPreview

    • type: UIView
    • description: UIView for camera preview.

• return: Void

• usage:

  [tracker setCameraPreviewWithPreview: preview];

removeCameraPreview

- (void)removeCameraPreview;

This will remove the camera preview that set at GazeTracker.

• parameter: X

• return: void

• usage:

  [tracker removeCameraPreview];

Version Check

getFrameworkVersion

(NSString * _Nonnull)getFrameworkVersion

This will return SeeSo framework version.

• parameter: X

• return: NSString

• usage:

  NSString * version = [GazeTracker getFrameworkVersion];
  NSLog(@"SeeSo version: %@.", version);

Delegates assistants

setDelegates

- (void)setDelegatesWithStatusDelegate:(id <StatusDelegate> _Nullable)statusDelegate gazeDelegate:(id <GazeDelegate> _Nullable)gazeDelegate calibrationDelegate:(id <CalibrationDelegate> _Nullable)calibrationDelegate imageDelegate:(id <ImageDelegate> _Nullable)imageDelegate userStatusDelegate:(id <UserStatusDelegate> _Nullable)userStatusDelegate;

Enroll all delegates that inherited GazeTrackerDelegate to GazeTracker at once.

• parameter

  • statusDelegate

    • type: id <StatusDelegate>

  • gazeDelegate

    • type: id <GazeDelegate>

  • calibrationDelegate

    • type: id <CalibrationDelegate>

  • imageDelegate

    • type: id <ImageDelegate>

  • userStatusDelegate

    • type: id <UserStatusDelegate>

• return: void

• usage:

  // Parameters are nullable
  [tracker setDelegatesWithStatusDelegate:statusDelegate gazeDelegate:gazeDelegate calibrationDelegate:NULL imageDelegate:NULL, userStatusDelegate: userStatusDelegate];

statusDelegate

@property (nonatomic, weak) id <StatusDelegate> _Nullable statusDelegate;

Can enroll and remove StatusDelegate in GazeTracker easily.

• type: id <StatusDelegate>

• description: StatusDelegate variable uses in GazeTracker

• usage:

  tracker.statusDelegate = self // Enroll delegate
  tracker.statusDelegate = nil // Remove delegate

gazeDelegate

@property (nonatomic, weak) id <GazeDelegate> _Nullable gazeDelegate;

Can enroll and remove GazeDelegate in GazeTracker easily.

• type: id <GazeDelegate>

• description: GazeDelegate variable uses in GazeTracker

• usage:

  tracker.gazeDelegate = self // Enroll delegate
  tracker.gazeDelegate = nil // Remove delegate

calibrationDelegate

@property (nonatomic, weak) id <CalibrationDelegate> _Nullable calibrationDelegate;

Can enroll and remove CalibrationDelegate in GazeTracker easily.

• type: id <GazeDelegate>

• description:: CalibrationDelegate variable uses in GazeTracker

• usage:

  tracker.calibrationDelegate = self // Enroll delegate
  tracker.calibrationDelegate = nil // Remove delegate

imageDelegate

@property (nonatomic, weak) id <CalibrationDelegate> _Nullable calibrationDelegate;

Can enroll and remove ImageDelegate in GazeTracker easily.

• type: id <CalibrationDelegate>

• description: ImageDelegate variable uses in GazeTracker

• usage:

  tracker.imageDelegate = self // Enroll delegate
  tracker.imageDelegate = nil // Remove delegate

UserStatusDelegate

@property (nonatomic, weak) id <UserStatusDelegate> _Nullable userStatusDelegate;

Can enroll and remove UserStatusDelegate in GazeTracker easily.

• type: id <UserStatusDelegate>

• description: UserStatusDelegate variable uses in GazeTracker

• usage:

  tracker.userStatusDelegate = self // Enroll delegate
  tracker.userStatusDelegate = nil // Remove delegate

Class

UserStatusOption

@interface UserStatusOption : NSObject

The class contains User Status options information for GazeTracker

• functions:

  • isUseAttention

    • parameter: x
    • return: BOOL
    • description: Return true if Attention flag is on, otherwise return false.

  • isUseBlink

    • parameter: x
    • return: BOOL
    • description: Return true if Blink flag is on, otherwise return false.

  • isUseDrowsiness

    • parameter: x
    • return: BOOL
    • description: Return true if Drowsiness flag is on, otherwise return false.

  • useAttention

    • parameter: x
    • return: void
    • description: Set Attention flag.

  • useBlink

    • parameter: x
    • return: void
    • description: Set Blink flag.

  • useDrowsiness

    • parameter: x
    • return: void
    • description: Set Drowsiness flag.

  • useAll

    • parameter: x
    • return: void
    • description: Set All User Status flag.

GazeInfo

@interface GazeInfo : NSObject

x

@property (nonatomic, readonly) double x;

x coordinate value of gaze point. Origin is device screen. The unit is pixel(px).

y

@property (nonatomic, readonly) double y;

y coordinate value of gaze point. Origin is device screen. The unit is pixel(px).

timestamp

@property (nonatomic, readonly) double timestamp;

Timestamp of gaze point. The unit is millisecond. The time format is UTC.

trackingState

@property (nonatomic, readonly) enum TrackingState trackingState;;

SUCCESS, LOW_CONFIDENCE, UNSUPPORTED, FACE_MISSING

eyeMovementState

@property (nonatomic, readonly) enum EyeMovementState eyeMovementState;

FIXATION, SACCADE, UNKNOWN

screenState

@property (nonatomic, readonly) enum ScreenState screenState;

INSIDE_OF_SCREEN, OUTSIDE_OF_SCREEN, UNKNOWN

GazeTrackerDelegate

@protocol GazeTrackerDelegate

protocol variable uses in GazeTracker. All delegate in GazeTracker inherit GazeTrackerDelegate.

InitializationDelegate

@protocol InitializationDelegate <GazeTrackerDelegate>

onInitialized

- (void)onInitializedWithTracker:(GazeTracker * _Nullable)tracker error:(enum InitializationError)error;

The callback function that calls when GazeTracker.init function is called.

It returns a constructed object when succeed, but nil if failed.

The error will be set by its type when construction failed.

• parameter

  • tracker

    • type: GazeTracker *
    • description: Relay GazeTracker when construction succeed. Relay nil if failed.

  • error

    • type: enum InitializationError
    • description: The enum that contains error types of initGazeTracker.

• return: void

• usage

  - (void)onInitializedWithTracker:(GazeTracker *)tracker error:(enum InitializationError)error {
      if (tracker != nil){
          self->tracker = tracker;
      }else {
          if (error == InitializationErrorERROR_INIT) {
              // When initialization is failed
              NSLog(@"Initialization failed");
          } else if (error == InitializationErrorERROR_CAMERA_PERMISSION) {
              // When camera permission doesn not exists
              NSLog(@"Required permission not granted");
          } 
          
          ...
      }
  

StatusDelegate

@protocol StatusDelegate <GazeTrackerDelegate>

onStarted

- (void)onStarted;

The function that automatically calls after GazeTracker.StartTracking succeed. Actions like calibration, preview, etc. are available after it.

• parameter: X

• return: void

• usage:

  - (void)onStarted{
      [tracker startCalibrationWithMode:CalibrationModeDEFAULT region:self.view.frame];
  }

onStopped

- (void)onStoppedWithError:(enum StatusError)error;

Error value will be StatusError.ERROR_NONE if gaze tracking stopped after GazeTracker.onStopTracking called but different values for a different statuses. It works properly when startTracking explicitly called at the gaze tracker stopping process.

• parameter

  • error

    • type: StatusError
    • description: StatusErrorERROR_NONE, StatusErrorERROR_CAMERA_START, StatusErrorERROR_CAMERA_INTERRUPT are available.

• return: void

• usage:

  - (void)onStoppedWithError:(enum StatusError)error {
      NSLog(@"stopped : %ld", error);
  }

GazeDelegate

@protocol GazeDelegate <GazeTrackerDelegate>

onGaze

- (void)onGazeWithGazeInfo:(GazeInfo * _Nonnull)gazeInfo;

• parameter

  • gazeInfo

    • type : GazeInfo

• return: void

• usage:

  - (void)onGazeWithGazeInfo:(GazeInfo * _Nonnull)gazeInfo {
      if([gazeInfo trackingState] == TrackingStateSUCCESS){
  	    // Gaze tracking succeed after calibration
      }else {
  	    // Face Missing
      }
  }

CalibrationDelegate

@protocol CalibrationDelegate <GazeTrackerDelegate>

CalibrationDelegate is never called on the main queue. Please use dispatch_get_main_queue when using UI.

---

onCalibrationNextPoint

- (void)onCalibrationNextPointWithX:(double)x y:(double)y;

The x, y coordinate value of the gaze point that should be focused during the calibration process. You should call startCollectSamples to keep process the calibration.

• parameter

  • x, y

    • type: double
    • description: The x, y coordinate value of the gaze point that should be focused during the calibration process. Origin is the device screen. The unit is point(pt)

• return: void

• usage:

  - (void)onCalibrationNextPointWithX:(double)x y:(double)y {
  // Display UI at given point
      // call startCollectSamples() like below for start calibration
      [tracker startCollectSamples];
  }

onCalibrationProgress

- (void)onCalibrationProgressWithProgress:(double)progress;

The value will be between 0.0~1.0. The next point will be guided when the value reaches 1.0.

• parameter

  • progress

    • type: double
    • description: Calibration progression for each point.

• return: void

• usage:

  - (void)onCalibrationProgressWithProgress:(double)progress {
      [progressView setProgress:progress];
  }

onCalibrationFinished

- (void)onCalibrationFinishedWithCalibrationData:(NSArray<NSNumber *> * _Nonnull)calibrationData;

Callback for notify calibration ends. When this function is called, the calibration UI will be removed. After this callback, data from GazeDelegate will be calibrated gaze data.

The calibrationData passed as a parameter has already been applied to GazeTracker,

and you can save it and load the this calibration data directly into GazeTracker without new calibration by calling setCalibrationData(calibrationData) when restarting the app etc..

• parameter: X

• return: void

• usage:

  - (void)onCalibrationFinishedWithCalibrationData:(NSArray<NSNumber *> * _Nonnull)calibrationData{
      // Remove calibration UI
      [self removeCalibrationUI];
      //save calibration Data
      [self calibrationData] = calibrationData;
  }
  

ImageDelegate

@protocol ImageDelegate <GazeTrackerDelegate>

Functions

onImage

- (void)onImageWithTimestamp:(double)timestamp image:(CMSampleBufferRef _Nonnull)image;

The function that provide the image as CMSampleBuffer form.

• parameter

  • timestamp

    • type: double
    • description: The timestamp of camera image creation. The unit is millisecond. The time format is UTC.

  • image

    • type: CMSampleBufferRef
    • description: CMSampleBuffer type image from camera(kCVPixelFormatType_32BGRA, AVCaptureSession.Preset.vga640x480). The direction is 90 deg rotated as CCW from portrait direction.

• return: void

• usage:

  - (void)onImageWithTimestamp:(double)timestamp image:(CMSampleBufferRef)image {
    // SAMPLE: save image as jpeg with proper function
      if (imageSave){
          [self writeJPEGFileImage:image];
      }
  }

UserStatusDelegate

    @protocol UserStatusDelegate <GazeTrackerDelegate>

Attention: How much the user attention is focused on the screen content for interval time (0.0 ~ 1.0)

• Timestamp range of the data will be passed as timestampBegin and timestampEnd in onAttention callback.
• The default time interval is 30 seconds.
• If the user attention level is low, score in onAttention callback will be closed to 0.0.
• If the user attention level is high, score in onAttention callback will be closed to 1.0.

Drowsiness: If the user feel drowsiness (True/False)

• Timestamp of the data will be passed as timestamp in onDrowsiness callback.
• If the user feel Drowsiness, isDrowsiness in onDrowsiness callback will be true.
• Otherwise, isDrowsiness will be false.

Blink: If the user blink eyes (left eye, right eye, general(both eyes))

• Timestamp of the data will be passed as timestamp in onBlink callback.
• If the user blink left eye, isBlinkLeft in onBlink callback will be true.
• If the user blink right eye, isBlinkRight in onBlink callback will be true.
• If the user blink eyes, isBlink in onBlink callback will be true (This is a general blink condition).
• If the user's eyes are wide, eyeOpenness in onBlink callback will be closed to 1.0 (not available yet).
• If the user's eyes are narrow, eyeOpenness in onBlink callback will be closed to 0.0 (not available yet).

onAttention

    @optional 
    - (void)onAttentionWithTimestampBegin:(NSInteger)timestampBegin timestampEnd:(NSInteger)timestampEnd score:(double)score;

• parameter:

  • timestampBegin

    • type: NSInteger
    • description: Beginning Timestamp of the data.

  • timestampEnd

    • type: NSInteger
    • description: Ending Timestamp of the data.

  • score

    • type: double
    • description: User Attention rate score between the timestamps.

• return: void

• usage:

     - (void)onAttentionWithTimestampBegin:(NSInteger)timestampBegin timestampEnd:(NSInteger)timestampEnd score:(double)score {
    [attentionView setText: [NSString stringWithFormat: @"Attention : %f", ((double)((NSInteger)(score * 1000))/1000.0)]];
    }

onBlink

    @optional 
    - (void)onBlinkWithTimestamp:(NSInteger)timestamp isBlinkLeft:(BOOL)isBlinkLeft isBlinkRight:(BOOL)isBlinkRight isBlink:(BOOL)isBlink eyeOpenness:(double)eyeOpenness;

• parameter:

  • timestamp

    • type: NSInteger
    • description: Timestamp of the data.

  • isBlinkLeft

    • type: BOOL
    • description: User Left Blink flag.

  • isBlinkRight

    • type: BOOL
    • description: User Right Blink flag.

  • isBlink

    • type: BOOL
    • description: User Blink flag.

  • eyeOpenness

    • type: double
    • description: User EyeOpenness rate (not available yet).

• return: void

• usage:

    - (void)onBlinkWithTimestamp:(NSInteger)timestamp isBlinkLeft:(BOOL)isBlinkLeft isBlinkRight:(BOOL)isBlinkRight isBlink:(BOOL)isBlink eyeOpenness:(double)eyeOpenness{
        [blinkView setText:[NSString stringWithFormat:@"Blink: %d", isBlink]];
        [blinkLeftView setText:[NSString stringWithFormat:@"Blink Left: %d", isBlinkLeft]];
        [blinkRightView setText:[NSString stringWithFormat:@"Blink Right: %d", isBlinkRight]]; 
    }

onDrowsiness

    @optional 
    - (void)onDrowsinessWithTimestamp:(NSInteger)timestamp isDrowsiness:(BOOL)isDrowsiness;

• parameter:

  • timestamp

    • type: NSInteger
    • description: Timestamp of the data.

  • isDrowsiness

    • type: BOOL
    • description: User Drowsiness flag.

• return: void

• usage:

    - (void)onDrowsinessWithTimestamp:(NSInteger)timestamp isDrowsiness:(BOOL)isDrowsiness{
        [drowsinessView setText:[NSString stringWithFormat:@"Drowsiness : %d", isDrowsiness]];
    }

ENUM

InitializationError

enum InitializationError : NSInteger;

The enum that contains error types of InitializationDelegate.

Please read : Authentication for more details.

StatusError

enum StatusError : NSInteger;

The enum that contains error types of StatusDelegate

• constant:

  • StatusError ERROR_NONE

    • value: 0
    • description: GazeTracker.stopTracking call succeed without error.

  • StatusError ERROR_CAMERA_START

    • value: 1
    • description: Error code occurs when GazeTracker.startTracking is called but front camera of device is not available.

  • ERROR_CAMERA_INTERRUPT

    • value: 2
    • description: Error code occurs when camera is unavailable.

TrackingState

enum TrackingState : NSInteger;

The enum that contains state types using at GazeDelegate.

• constant:

  • SUCCESS

    • value: 0
    • description: Face alignment is in a best position (Gaze tracking success, with valid x and y).

  • LOW_CONFIDENCE

    • value: 1
    • description: Face alignment is not in the best position, should not be used for precise gaze tracking (Gaze tracking success, with less accurate x and y).

  • UNSUPPORTED

    • value: 2
    • description: Face alignment is not suitable for tracking (Gaze tracking fail, with invalid x and y).

  • FACE_MISSING

    • value: 3
    • description: Face is missing (Gaze tracking fail).

CalibrationMode

enum CalibrationMode : NSInteger;

The enum that contains mode types of GazeTracker.startCalibartion .

• constant

  • DEFAULT

    • value: 0
    • description: Default. Represent FIVE_POINT.

  • ONE_POINT

    • value: 1
    • description: One-point calibration mode.

  • FIVE_POINT

    • value: 5
    • description: Five-point calibration mode.

  • SIX_POINT

    • value: 6
    • description: Six-point calibration mode

AccuracyCriteria

 enum AccuracyCriteria : NSInteger;

The enum that contains accuracy criteria of GazeTracker.startCalibartion .

• enum

  • DEFAULT

    • value: 0
    • description: Default calibration accuracy criteria.

  • LOW

    • value: 1
    • description: Low calibration accuracy criteria.

  • HIGH

    • value: 2
    • description: High calibration accuracy criteria.

EyeMovementState

enum EyeMovementState : NSInteger;

The class that contains state types using at EyeMovementDelegate.

• constant

  • FIXATION

    • value: 0

  • SACCADE

    • value: 2

  • UNKNOWN

    • value: 3

ScreenState

enum ScreenState : NSInteger;

• constant:

  • INSIDE_OF_SCREEN

    • value : 0
    • description: Gaze tracking is success and the gaze point is inside of device screen

  • OUTSIDE_OF_SCREEN

    • value : 1
    • description: Gaze tracking is success and the gaze point is outside of device screen

  • UNKNOWN

    • value : 2
    • description: Gaze tracking is fail