Skip to main content
Version: Legacy SeeSo

API Docs (C++)

Overview

ClassDescription
GazeTrackerClass that generates gaze tracking data
IGazeCallbackGaze data listener
IFaceCallbackFace data listener
ICalibrationCallbackCalibration process listener
IUserStatusCallbackUser status data listener
CoordConverterV2Matrix-based coordinate converter
ErrorReporterError listener base class
SeeSoBooleanSubstitution for bool type
SeeSoStatusModuleOptionsEnum with flag for each module when using StatusModule
SeeSoCalibrationPointNumEnum with number of points for calibration
SeeSoCalibrationAccuracyEnum containing options that manage calibration process and accuracy
SeeSoTrackingStateEnum containing tracking state types used in OnGaze callback
SeeSoEyeMovementStateEnum containing eye movement state types used in OnGaze callback

global_init

void global_init(const char* file = "seeso_core.dll")

This function sets up the program environment that SeeSo SDK needs.
This function must be called at least once within a program before the program calls any other function in SeeSo SDK.
It's a dll loader in actual. On other platforms except Windows, this function is no-op.

Exceptions

Failed to load a dll

SeeSo SDK requires the dll files to execute(which are included in the SDK files), including seeso_core.dll and several third party library dlls.
See seeso/bin or error logs for such dll files. seeso::global_init() will throw if one or more of these files are missing.
Make sure that they're in the same directory with an executable file, or use add_custom_command in CMakeLists.txt as the following.
If add_custom_command isn't working, copy and paste the dll files to the directory(where an executable file is located) manually.

Failed to find XXX from dll

Dll files in the SeeSo SDK are broken, or wrong version is used.
Re-download the SDK from SeeSo Console and try again.

getVersionStr

std::string getVersionStr()

return SeeSo SDK version information in string.

ReturnTypeDescription
version_strstd::stringSeeSo SDK version in std::string

Example

const std::string version = seeso::getVersionStr();

getVersionInt

int32_t getVersionInt()

Return SeeSo SDK version as 9 digit integer.
Version forms XXYYZZZZZ, where XX is a major version, YY is a minor version, and ZZZZZ is a patch version.

ReturnTypeDescription
version_intint32_tSeeSo SDK version in 9 digit integer

Example

int32_t version = seeso::getVersionInt();

GazeTracker

class GazeTracker

GazeTracker is the class that generates GazeTracking data.

Summary

Constructor and Destructor
GazeTracker(const converter_type&)
~GazeTracker()
Functions
initialized
deinitialized
isTrackerInitialized
setFaceDistance
getFaceDistance
setGazeCallback
setFaceCallback
setCalibrationCallback
setUserStatusCallback
removeGazeCallback
removeFaceCallback
removeCalibrationCallback
removeUserStatusCallback
setTrackingFPS
addFrame
startCalibration
stopCalibration
startCollectSamples
setCalibrationData
setAttentionRegion
getAttentionRegion
removeAttentionRegion
setAttentionInterval
getAttentionScore
converter

GazeTracker

explicit GazeTracker(const converter_type& coord_converter = makeNoOpConverter<float>())
ParametersTypeDescription
coord_converterconverter_typeCoordinate system converter. See Custom Coordinate

Example

// Coordinate system is camera-centered millimeters coordinate(default)
GazeTracker gaze_tracker();

// Coordinate system is display pixel coordinate
const auto camera2display = seeso::makeDefaultCameraToDisplayConverter<float>(...);
GazeTracker gaze_tracker_custom(camera2display);

~GazeTracker

~GazeTracker()

Effectively calls GazeTracker::deinitialize() and releases internal object.

initialize

int GazeTracker::initialize(
const std::string& licenseKey,
const SeeSoStatusModuleOptions& status_option = {kSeeSoFalse, kSeeSoFalse, kSeeSoFalse},
ErrorReporter* error_reporter = DefaultErrorReporter())

GazeTracker construction requires authentication. This example will pass SeeSoStatusModuleOptions to activate User Status Detector in GazeTracker. After initialization, authentication result will be displayed as a log with status code.

ParametersTypeDescription
licenseKeyconst std::string&Key generated from SeeSo Console
status_optionconst SeeSoStatusModuleOption&User-status-module option
error_reporterErrorReporter*Optional ErrorReporter. GazeTracker does not own the object.
Return TypeDescription
intAuthentication code. See Authentication for more details.

Example

SeeSoStatusModuleOptions status_options = {};
userStatusOption.use_attention = kSeeSoTrue;
userStatusOption.use_blink = kSeeSoTrue;
userStatusOption.use_drowsiness = kSeeSoTrue;
const int errCode = gaze_tracker.initialize('licenseKey', status_options);
if (errCode == 0) {
// Initialization success
} else {
// Initialization failed
}

deinitialize

void GazeTracker::deinitialize()

Deinitialize internal object. Calling other member functions(except for initialize) after calling deinitialize() is undefined behavior.

isTrackerInitialized

bool GazeTracker::isTrackerInitialized() const;
Return TypeDescription
boolReturns true if initialized, false otherwise.

setFaceDistance

void GazeTracker::setFaceDistance(int cm)

Set face distance from the camera in cm. To get better GazeTracking performance, accurate face distance should be set by this function. Default valule is 60 (cm).

ParametersTypeDescription
faceDistanceintDistance between user face and camera in cm.

getFaceDistance

int getFaceDistance() const
Return TypeDescription
intDistance between user face and camera in cm.

setGazeCallback

void GazeTracker::setGazeCallback(seeso::IGazeCallback *listener)

Set Gaze callback for GazeTracking.
Note that GazeTracker does not own the instance.

ParametersTypeDescription
listenerIGazeCallback*Address of gaze listener instance

Example

seeso::IGazeCallback gaze_callback;
gaze_tracker.setGazeCallback(&gaze_callback);

setFaceCallback

void GazeTracker::setFaceCallback(seeso::IFaceCallback *listener)

Set Face callback for GazeTracking.
Note that GazeTracker does not own the instance.

ParametersTypeDescription
listenerIFaceCallback*Address of face listener instance

Example

seeso::IFaceCallback face_callback;
gaze_tracker.setFaceCallback(&face_callback);

setCalibrationCallback

void GazeTracker::setCalibrationCallback(seeso::ICalibrationCallback* listener);

Set Calibration callback for GazeTracking.
Note that GazeTracker does not own the instance.

ParametersTypeDescription
listenerICalibrationCallback*Address of calibration listener instance.

Example

seeso::ICalibrationCallback calibration_callback;
gaze_tracker.setCalibrationCallback(&calibration_callback);

setUserStatusCallback

void GazeTracker::setUserStatusCallback(seeso::IUserStatusCallback* listener)

Set User Status callback for GazeTracking.
Note that GazeTracker does not own the instance.

ParametersTypeDescription
listenerIUserStatusCallback*Address of status callback instance.

Example

seeso::IUserStatusCallback status_callback;
gaze_tracker.setUserStatusCallback(&status_callback);

removeGazeCallback

void GazeTracker::removeGazeCallback()

Remove gaze callback interface for GazeTracking.

removeFaceCallback

void GazeTracker::removeFaceCallback()

Remove face callback interface for GazeTracking.

removeCalibrationCallback

void GazeTracker::removeCalibrationCallback()

Remove calibration callback interface for GazeTracking.

removeUserStatusCallback

void GazeTracker::removeUserStatusCallback()

Remove user status callback interface for GazeTracking.

setTrackingFPS

void GazeTracker::setTrackingFps(int fps)

Set the maximum InputFrames-per-second of the GazeTracking.
FPS can be lower than the set value due to the device spec. It limits the calculation by omitting the given frame internally.

ParametersTypeDescription
fpsintMaximum FPS(frames per seconds) for gaze tracking

Example

gaze_tracker.setTrackingFps(15);

addFrame

bool GazeTracker::addFrame(int64_t time_stamp, uint8_t *buffer, int width, int height)

Add image frame into GazeTracker for GazeTracking. addFrame should be called continuously, and add image frame from the camera stream. The image should be in RGB color format. Note that some frames can be ignored inside SeeSo SDK, thus not calculated

  • If frames are input very often(Don't worry; SeeSo SDK will perform 30+ FPS in real-time on most of the latest hardware).
  • Even if the hardware specs are good, some frames can be ignored to guarantee OnGazeListener::OnGaze()'s timestamp is in increasing order. This happens rarely. For each calculated input image, OnGazeListener::OnGaze() will be called with the input timestamp and the calculated values.
    Image data is copied inside the function, so it's safe to destroy an image object after calling addFrame.
ParametersTypeDescription
time_stampint64_tTimestamp of the given frame
bufferuint8_t*Image data buffer in RGB format. It is safe to release buffer after passing to GazeTracker since the buffer is deep-copied.
widthintWidth of the image
heightintHeight of the image
Return TypeDescription
boolReturns false if the input frame is omitted, true otherwise.

Example

cv::Mat frame, input;
while(true) {
video >> frame; // read frame
if (frame.empty()) break;

cv::cvtColor(frame, input, cv::COLOR_BGR2RGB); // convert color format
gaze_tracker.addFrame(current_time(), input.data, input.cols, input.rows);
}

startCalibration

void GazeTracker::startCalibration(
SeeSoCalibrationPointNum point_num,
SeeSoCalibrationAccuracy criteria,
float left, float top, float right, float bottom);

Start calibration process with CalibrationMode and Criteria. calibrationMode is number of calibration points. criteria is option that manage calibration process and accuracy.

ParametersTypeDescription
point_numSeeSoCalibrationPointNumNumber of points used for calibration
criteriaSeeSoCalibrationAccuracyAccuracy option of the calibration
leftfloatDistance to left edge of the calibration ROI
topfloatDistance to top edge of the calibration ROI
rightfloatDistance to right edge of the calibration ROI
bottomfloatDistance to bottom edge of the calibration ROI

Example

gaze_tracker.startCalibration(
kSeeSoCalibrationPointFive,
kSeeSoCalibrationAccuracyDefault,
0.0f, 0.0f, 1280.0f, 720.0f);

stopCalibration

void GazeTracker::stopCalibration()

Stop the current calibration process.

startCollectSamples

void GazeTracker::startCollectSamples()

Start collect next calibration point data.

This should be called in onCalibrationProgress when need to collect new calibration point data.

The onCalibrationProgress will give progress (0.0 ~ 1.0), which represents the data collecting progress of current calibration point.

Example

if (isCalibrating) {
gaze_tracker.startCollectSamples();
}

setCalibrationData

void GazeTracker::setCalibrationData(const std::vector<float>& calibrationData)

Set calibration data without calibrating manually.

ParametersTypeDescription
calibrationDataconst std::vector<float>&Stored calibration data

Example

  gaze_tracker.setCalibrationData(calibrationData);

setAttentionRegion

void GazeTracker::setAttentionRegion(float left, float top, float right, float bottom);

Set Status ROI.

ParametersTypeDescription
leftfloatSet distance to the left edge of the ROI rectangle
topfloatSet distance to the top edge of the ROI rectangle
rightfloatSet distance to the right edge of the ROI rectangle
bottomfloatSet distance to the bottom edge of the ROI rectangle

getAttentionRegion

std::vector<float> GazeTracker::getAttentionRegion() const;

Get Status ROI.

Return TypeDescription
std::vectorcontains the value from setAttentionRegion if set, empty std::vector otherwise.

removeAttentionRegion

void GazeTracker::removeAttentionRegion();

Remove Status ROI.

setAttentionInterval

void GazeTracker::setAttentionInterval(int interval)

Set time interval for the UserStatus Attention callback. The UserStatus Attention score will be calculated for the given time interval. The beginning and ending timestamps are passed through the onAttention callback as timestampBegin and timestampEnd. The interval range is 10 to 60 (seconds), and the default value is 30 seconds.

ParametersTypeDescription
intervalintTime interval for the UserStatus Attention score in seconds.

Example

// IUserStatusCallback::OnAttention() will be called every 10 seconds
gaze_tracker.setAttentionInterval(10);

getAttentionScore

float GazeTracker::getAttentionScore()

Get current Attention score from the GazeTracker.

This API does not provide timestamp of the Attention score data.

Use onAttention callback, unless the Attention score is required at a specific time/location.

Return TypeDescription
floatCurrent attention score

Example

float currentAttention = gaze_tracker.getAttentionScore();     

converter

converter_type& GazeTracker::converter()
const converter_type& GazeTracker::converter() const

Access or modify GazeTracker's coordinate converter. See Custom Coordinate.

IGazeCallback

class IGazeCallback {
public:
IGazeCallback() = default;
virtual ~IGazeCallback() = default;

virtual void OnGaze(/* See below */) = 0;
};

Summary

Abstract Functions
onGaze

OnGaze

virtual void IGazeCallback::OnGaze(
std::uint64_t timestamp,
float x, float y,
float fixation_x, float fixation_y,
SeeSoTrackingState tracking_state,
SeeSoEyeMovementState eye_movement_state)

Gaze callback.

ParametersTypeDescription
timestampuint64_tTimestamp of the iamge from GazeTracker::addFrame
xfloatx coordinate of gaze point
yfloaty coordinate of gaze point
fixation_xfloatx coordinate value of last fixation point.
fixation_yfloaty coordinate value of last fixation point.
left_opennessfloatopenness degree of left eye(0.0~1.0). value will only return when userStatusOption in on
right_opennessfloatopenness degree of right eye(0.0~1.0). value will only return when userStatusOption in on
tracking_stateSeeSoTrackingStateTracking state value
eye_movement_stateSeeSoEyeMovementStateEye movement state value

IFaceCallback


class IFaceCallback {
public:
IFaceCallback() = default;
virtual ~IFaceCallback() = default;

virtual void OnFace(/* See below */) = 0;
};

Summary

Abstract Functions
onFace

OnFace

virtual void IFaceCallback::OnFace(
std::uint64_t timestamp,
float score,
float left, float top, float right, float bottom
float pitch, float yaw, float roll
const float* center_xyz, std::uint32_t center_xyz_size)

Face callback.

ParametersTypeDescription
timestampuint64_tTimestamp of the iamge from GazeTracker::addFrame
scorefloatValue of facial recognition confidence (0.0 ~ 1.0).
leftfloatLeft of the face.
topfloatTop of the face.
rightfloatRight of the face.
bottomfloatBottom of the face.
pitchfloatPitch is the rotation around the X-axis of an object. It is also known as the attitude angle. In aviation, it is often referred to as the nose-up or nose-down angle.
yawfloatYaw is the rotation around the Y-axis of an object. It is also known as the heading angle. In aviation, it is often referred to as the bearing or compass angle.
rollfloatRoll is the rotation around the Z-axis of an object. It is also known as the bank angle. In aviation, it is often referred to as the wing-up or wing-down angle.
center_xyzconst float*The x,y,z distance of the center of the face from the camera. The unit is mm.
center_xyz_sizestd::uint32_tcenter_xyz is size

ICalibrationCallback


class ICalibrationCallback {
public:
ICalibrationCallback() = default;
virtual ~ICalibrationCallback() = default;

virtual void OnCalibrationProgress(float progress) = 0;
virtual void OnCalibrationNextPoint(float x, float y) = 0;
virtual void OnCalibrationFinish(const std::vector<float>& data) = 0;
};

Summary

Abstract Functions
onCalibrationNextPoint
onCalibrationProgress
onCalibrationFinished

OnCalibrationNextPoint

void ICalibrationCallback::OnCalibrationNextPoint(float x, float y)

Coordinate of the next calibration point.

x and y are current calibration point coordinate.

Point (x, y) should be focused during the calibration process.

ParametersTypeDescription
timestampuint64_tTimestamp of the iamge from GazeTracker::addFrame
xfloatx coordinate of the current calibration point
yfloaty coordinate of the current calibration point

Example

void OnCalibrationNextPoint(float x, float y) {
this->render_point_x = x;
this->render_point_y = y;
}

OnCalibrationProgress

void ICalibrationCallback::OnCalibrationProgress(float progress)

Calibration progress callback.

progress is the calibration progress of the current calibration point (0.0 ~ 1.0).

Next calibration point will be guided when the progress reaches 1.0.

startCollectSamples shuold be called to collect next calibration data.

Return 0.0 when start calibration progress.

Return 1.0 when finish calibratino progress.

ParametersTypeDescription
progressfloatCalibration progress of the current calibration point

Example

void OnCalibrationProgress(float progress){
if(progress >= 1.0){
// Collect next calibration point data
gaze_tracker.startCollectSamples();
} else {
// Collect current calibration point data
}
}

OnCalibrationFinished

void ICalibrationCallback::OnCalibrationFinished(const std::vector<float>& calibrationData)

Calibration finish callback.

calibrationData is the result calibration data.

ParametersTypeDescription
calibrationDataconst vector<float>&Result of the calibration data

IUserStatusCallback

class IUserStatusCallback {
public:
IUserStatusCallback() = default;
virtual ~IUserStatusCallback() = default;

virtual void OnAttention(uint64_t timestampBegin, uint64_t timestampEnd, float score) = 0;
virtual void OnDrowsiness(uint64_t timestamp, bool isDrowsiness, float intensity) = 0;
virtual void OnBlink(uint64_t timestamp, bool isBlinkLeft, bool isBlinkRight, bool isBlink, float leftOpenness, float rightOpenness) = 0;
};

Summary

Abstract Functions
onAttention
onDrowsiness
onBlink

The user status includes Attention, Drowsiness, and Blink. Attention: How much the user attention is focused on the screen content for interval time (0.0 ~ 1.0)

  • Timestamp range of the data will be passed as timestampBegin and timestampEnd in onAttention callback.
  • The default time interval is 30 seconds.
  • If the user attention level is low, score in onAttention callback will be closed to 0.0.
  • If the user attention level is high, score in onAttention callback will be closed to 1.0. Drowsiness: If the user feel drowsiness (True/False)
  • Timestamp of the data will be passed as timestamp in onDrowsiness callback.
  • If the user feel Drowsiness, isDrowsiness in onDrowsiness callback will be true.
  • Otherwise, isDrowsiness will be false. Blink: If the user blink eyes (left eye, right eye, general(both eyes))
  • Timestamp of the data will be passed as timestamp in onBlink callback.
  • If the user blink left eye, isBlinkLeft in onBlink callback will be true.
  • If the user blink right eye, isBlinkRight in onBlink callback will be true.
  • If the user blink eyes, isBlink in onBlink callback will be true (This is a general blink condition).

OnAttention

void IUserStatusCallback::OnAttention(uint64_t timestampBegin, uint64_t timestampEnd, float score)

Attention User Status callback.

ParametersTypeDescription
timestampBeginuint64_tBeginning timestamp of the data
timestampEnduint64_tEnd timestamp of the data
scorefloatUser attention rate score between the timestamps

OnDrowsiness

void IUserStatusCallback::OnDrowsiness(uint64_t timestamp, bool isDrowsiness, float intensity)

Drowsiness User Status callback.

ParametersTypeDescription
timestampuint64_tTimestamp of the data
isDrowsinessboolUser drowisness flag
intensityfloatLevel of drowsiness intensity (0 to 1)
void IUserStatusCallback::OnBlink(
uint64_t timestamp,
bool isBlinkLeft, bool isBlinkRight, bool isBlink,
float leftOpenness, float rightOpenness)

Blink User Status callback.

ParametersTypeDescription
timestampuint64_tTimestamp of the data
isBlinkLeftboolLeft blink flag
isBlinkRightboolRight blink flag
isBlinkboolBlink flag
leftOpennessfloatOpen ratio of left eye.
rightOpennessfloatOpen ratio of right eye.

CoordConverterV2


template<typename T = double>
class CoordConverterV2 {
public:
using value_type = T;
using transform_type = Matrix<value_type, 2, 2>;
using translate_type = Matrix<value_type, 2, 1>;
using coordinate_type = Matrix<value_type, 2, 1>;

/** default constructor: R=I, T=Zeros */
constexpr CoordConverterV2() = default;
CoordConverterV2(const transform_type& r, const translate_type& t);

CoordConverterV2& transform(const transform_type& t);
transform_type& transform();
const transform_type& transform() const { return transform_; }

CoordConverterV2& translate(const translate_type& t);
translate_type& translate();
const translate_type& translate() const;

coordinate_type convert(const coordinate_type& m) const;
coordinate_type revert(const coordinate_type& m) const;
};

Helper functions

CoordConverterV2<T> seeso::makeCameraToDisplayConverter(const Point<float>& display_tl, const Point<float>& display_pixels, const Point<float>& display_mm)
CoordConverterV2<T> seeso::makeDefaultCameraToDisplayConverter(const Point<float>& display_pixels, const Point<float>& display_mm)
CoordConverterV2<T> seeso::makeDefaultCameraToDisplayConverter(float px_width, float px_height, float mm_width, float mm_height)
CoordConverterV2<T> seeso::makeNoOpConverter()

See the header file comment

ErrorReporter

class ErrorReporter {
public:
virtual ~ErrorReporter() = default;
virtual int report(const char* message) = 0;

int report(const std::string& message);
};

Summary

Constructor and Destructor
~ErrorReporter
Functions
report

Base class of error reporter.
This feature is experimental.

report

virtual int report(const char* message) const noexcept; 
ParametersTypeDescription
messagechar*Message got from the caller
Return TypeDescription
intUnspecified (Experimental)

Helper Functions

ErrorReporter* DefaultErrorReporter();

Get a pointer to the default error reporter in shared memory.
Releasing the returned object is undefined behavior.

ReturnTypeDescription
error_reporterErrorReporter*Pointer to the default error reporter.

SeeSoBoolean

typedef enum SeeSoBoolean {
kSeeSoFalse = 0,
kSeeSoTrue = 1,
} SeeSoBoolean;

Substitution of bool type.

SeeSoStatusModuleOptions

typedef struct SeeSoStatusModuleOptions {
SeeSoBoolean use_attention;
SeeSoBoolean use_blink;
SeeSoBoolean use_drowsiness;
} SeeSoStatusModuleOptions;

Flag of each module when using Status Module

NameDescription
use_attentionInclude attention score
use_blinkInclude blink detection
use_drowsinessInclude drowsiness detection

SeeSoCalibrationPointNum

typedef enum SeeSoCalibrationPointNum {
kSeeSoCalibrationPointOne = 1,
kSeeSoCalibrationPointTwo = 2,
kSeeSoCalibrationPointFive = 5,
} SeeSoCalibrationPointNum;

The number of points for calibration.

NameDescription
kSeeSoCalibrationPointOneOne-point calibration mode
kSeeSoCalibrationPointTwoTwo-point calibration mode
kSeeSoCalibrationPointFiveFive-point calibration mode

SeeSoCalibrationAccuracy

typedef enum SeeSoCalibrationAccuracy {
kSeeSoCalibrationAccuracyDefault = 0,
kSeeSoCalibrationAccuracyLow,
kSeeSoCalibrationAccuracyHigh,
} SeeSoCalibrationAccuracy;

The enum containing options that manage calibration process and accuracy.

NameDescription
kSeeSoCalibrationAccuracyDefaultStarts default calibration
kSeeSoCalibrationAccuracyLowStart calibration with lower threshold for each step. (May result in lower accuracy)
kSeeSoCalibrationAccurarcyHighStart calibration with higher threshold for each step. (Likely to result in higher accuracy)

SeeSoTrackingState

typedef enum SeeSoTrackingState {
kSeeSoTrackingSuccess = 0,
kSeeSoTrackingLowConfidence,
kSeeSoTrackingUnsupported,
kSeeSoTrackingFaceMissing,
} SeeSoTrackingState;

The enum containing tracking state types used in OnGaze callback.

NameDescription
kSeeSoTrackingSuccessFace alignment is in the best position, tracking is successful with valid accuracy
kSeesoTrackingLowConfidenceFace alignment is not in the best position, but tracking is successful with lower accuracy
kSeeSoTrackingUnsupportedFace alignment is not suitable for tracking (Gaze tracking failed with invalud x and y)
kSeeSoTrackingFaceMissingFace is missing (Gaze tracking has failed)

SeeSoEyeMovementState

typedef enum SeeSoEyeMovementState {
kSeeSoEyeMovementFixation = 0,
kSeeSoEyeMovementSaccade = 2,
kSeeSoEyeMovementUnknown = 3,
} SeeSoEyeMovementState;

The enum containing eye movement state types used in OnGaze callback.

NameDescription
kSeeSoEyeMovementFixationEye movement has fixation.
kSeeSoEyeMovementSaccadeEye movement has saccade.
kSeeSoEyeMovementUnknownEye movement is unknown.