API Docs (C++)
Overview
| Class | Description |
|---|---|
| GazeTracker | Class that generates gaze tracking data |
| ITrackingCallback | Gaze tracking metrics data listener |
| ICalibrationCallback | Calibration process listener |
| CoordConverterV2 | Matrix-based coordinate converter |
| EyedidBoolean | Substitution for bool type |
| EyedidTrackerOptions | This is a struct that sets the options related to the creation of a GazeTracker. |
| EyedidCalibrationPointNum | Enum with number of points for calibration |
| EyedidCalibrationAccuracy | Enum containing options that manage calibration process and accuracy |
| EyedidTrackingState | Enum containing tracking state types used in GazeInfo |
| EyedidEyeMovementState | Enum containing eye movement state types used in GazeInfo |
| EyedidGazeData | This is a struct that contains data related to Gaze. |
| EyedidFaceData | This is a struct that contains data related to face recognition. |
| EyedidBlinkData | This is a struct that contains data related to blink detection. |
| EyedidUserStatusData | This is a struct that contains data related to user status. |
global_init
void global_init(const char* file = "eyedid_core.dll")
This function sets up the program environment that Eyedid SDK needs.
This function must be called at least once within a program before the program calls any other function in Eyedid SDK.
It's a dll loader in actual. On other platforms except Windows, this function is no-op.
Exceptions
Failed to load a dll
Eyedid SDK requires the dll files to execute(which are included in the SDK files), including eyedid_core.dll and
several third party library dlls.
See eyedid/bin or error logs for such dll files.
eyedid::global_init() will throw if one or more of these files are missing.
Make sure that they're in the same directory with an executable file, or use add_custom_command in CMakeLists.txt as
the following.
If add_custom_command isn't working, copy and paste the dll files to the directory(where an executable file is located) manually.
Failed to find XXX from dll
Dll files in the Eyedid SDK are broken, or wrong version is used.
Re-download the SDK from Eyedid Console and try again.
getVersionStr
std::string getVersionStr()
return Eyedid SDK version information in string.
| Return | Type | Description |
|---|---|---|
| version_str | std::string | Eyedid SDK version in std::string |
Example
const std::string version = eyedid::getVersionStr();
getVersionInt
int32_t getVersionInt()
Return Eyedid SDK version as 9 digit integer.
Version forms XXYYZZZZZ, where XX is a major version, YY is a minor version, and ZZZZZ is a patch version.
| Return | Type | Description |
|---|---|---|
| version_int | int32_t | Eyedid SDK version in 9 digit integer |
Example
int32_t version = eyedid::getVersionInt();
GazeTracker
class GazeTracker
GazeTracker is the class that generates GazeTracking data.
Summary
| Constructor and Destructor |
|---|
GazeTracker(const converter_type&) |
~GazeTracker() |
GazeTracker
explicit GazeTracker(const converter_type& coord_converter = makeNoOpConverter<float>())
| Parameters | Type | Description |
|---|---|---|
| coord_converter | converter_type | Coordinate system converter. See Custom Coordinate |
Example
// Coordinate system is camera-centered millimeters coordinate(default)
GazeTracker gaze_tracker();
// Coordinate system is display pixel coordinate
const auto camera2display = eyedid::makeDefaultCameraToDisplayConverter<float>(...);
GazeTracker gaze_tracker_custom(camera2display);
~GazeTracker
~GazeTracker()
Effectively calls GazeTracker::deinitialize() and releases internal object.
initialize
int GazeTracker::initialize(
const std::string& licenseKey,
const EyedidTrackerOptions& options
)
The construction of GazeTracker requires authentication.
This example will pass EyedidTrackerOptions to configure GazeTracker.
After initialization, the authentication result will be logged with a status code.
| Parameters | Type | Description |
|---|---|---|
| licenseKey | const std::string& | Key generated from Eyedid SDK Console |
| options | const EyedidTrackerOptions& | These are the options related to configuring GazeTracker. |
| Return Type | Description |
|---|---|
int | Authentication code. See Authentication for more details. |
Example
GazeTrackerOption option = {};
option.use_user_status = kEyedidTrue;
option.use_blink = kEyedidTrue;
const int errCode = gaze_tracker.initialize('licenseKey', option);
if (errCode == 0) {
// Initialization success
} else {
// Initialization failed
}
deinitialize
void GazeTracker::deinitialize()
Deinitialize the internal object. Calling other member functions (except for initialize) after calling deinitialize() results in undefined behavior.
isTrackerInitialized
bool GazeTracker::isTrackerInitialized() const;
| Return Type | Description |
|---|---|
bool | Returns true if initialized, false otherwise. |
Example
auto is_initialized = gaze_tracker.isTrackerInitialized();
setFaceDistance
void GazeTracker::setFaceDistance(int cm)
Set face distance from the camera in cm.
To get better GazeTracking performance, accurate face distance should be set by this function.
Default value is 60 (cm).
| Parameters | Type | Description |
|---|---|---|
| cm | int | Distance between user face and camera in cm. |
Example
gaze_tracker.setFaceDistance(60);
getFaceDistance
int getFaceDistance() const
| Return Type | Description |
|---|---|
int | Distance between user face and camera in cm. |
Example
auto face_distance = gaze_tracker.getFaceDistance();
setCameraFOV
void GazeTracker::setCameraFOV(float fov);
If you know the horizontal field of view (FOV) of the webcam, you can set it using this function. This can help improve accuracy compared to using the setFaceDistance function.
| Parameters | Type | Description |
|---|---|---|
| fov | float | The camera’s horizontal FOV (field of view) is specified as a floating-point value in radians. |
Example
gaze_tracker.setCameraFOV(M_PI / 3.f); // 60 degree.
getCameraFOV
float GazeTracker::getCameraFOV();
Returns the current horizontal field of view (FOV) value of the camera.
| Return Type | Description |
|---|---|
float | horizontal field of view (FOV). |
Example
auto fov = gaze_tracker.getCameraFOV();
std::cout << "Current FOV: " << fov << std::endl;
setTrackingCallback
void GazeTracker::setTrackingCallback(eyedid::ITrackingCallback *listener)
This function sets the callback for gaze tracking metrics.
Note that GazeTracker does not own the instance.
| Parameters | Type | Description |
|---|---|---|
| listener | ITrackingCallback* | Address of gaze tracking metrics listener instance |
Example
eyedid::ITrackingCallback listener;
gaze_tracker.setTrackingCallback(&listener);
setCalibrationCallback
void GazeTracker::setCalibrationCallback(eyedid::ICalibrationCallback* listener);
Set Calibration callback for GazeTracking.
Note that GazeTracker does not own the instance.
| Parameters | Type | Description |
|---|---|---|
| listener | ICalibrationCallback* | Address of calibration listener instance. |
Example
eyedid::ICalibrationCallback calibration_callback;
gaze_tracker.setCalibrationCallback(&calibration_callback);
removeTrackingCallback
void GazeTracker::removeTrackingCallback()
Remove tracking callback interface for GazeTracking.
removeCalibrationCallback
void GazeTracker::removeCalibrationCallback()
Remove calibration callback interface for GazeTracking.
setTrackingFPS
void GazeTracker::setTrackingFps(int fps)
Set the maximum Input Frames-per-second (FPS) for GazeTracking. FPS can be lower than the set value due to device specifications. It limits the calculation by omitting the given frame internally. The default value is set to 30 FPS.
| Parameters | Type | Description |
|---|---|---|
| fps | int | Maximum FPS(frames per seconds) for gaze tracking |
Example
gaze_tracker.setTrackingFps(15);
addFrame
bool GazeTracker::addFrame(int64_t time_stamp, uint8_t *buffer, int width, int height)
Add image frame into GazeTracker for GazeTracking.
addFrame should be called continuously, and add image frame from the camera stream.
The image should be in RGB color format.
Note that some frames can be ignored inside Eyedid SDK, thus not calculated
- If frames are input very often(Don't worry; Eyedid SDK will perform 30+ FPS in real-time on most of the latest hardware).
- Even if the hardware specs are good, some frames can be ignored to guarantee
iTrackingCallback::OnMetrics()'s timestamp is in increasing order. This happens rarely. For each calculated input image,iTrackingCallback::OnMetrics()will be called with the input timestamp and the calculated values.
Image data is copied inside the function, so it's safe to destroy an image object after callingaddFrame.
| Parameters | Type | Description |
|---|---|---|
| time_stamp | int64_t | Timestamp of the given frame |
| buffer | uint8_t* | Image data buffer in RGB format. It is safe to release buffer after passing to GazeTracker since the buffer is deep-copied. |
| width | int | Width of the image |
| height | int | Height of the image |
| Return Type | Description |
|---|---|
bool | Returns false if the input frame is omitted, true otherwise. |
Example
cv::Mat frame, input;
while(true) {
video >> frame; // read frame
if (frame.empty()) break;
cv::cvtColor(frame, input, cv::COLOR_BGR2RGB); // convert color format
gaze_tracker.addFrame(current_time(), input.data, input.cols, input.rows);
}
setTargetBoundRegion
void GazeTracker::setTargetBoundRegion(float left, float top, float right, float bottom)
This function sets the gaze tracking area to a specific region. By default, the entire screen of the main monitor is used.
| Parameters | Type | Description |
|---|---|---|
| left | float | The left boundary of the tracking area. |
| top | float | The top boundary of the tracking area. |
| right | float | The right boundary of the tracking area. |
| bottom | float | The bottom boundary of the tracking area. |
Example
gaze_tracker.setTargetBoundRegion(100, 200, 2000, 1000); // Set the tracking area: left=100px, top=200px, right=2000px, bottom=1000px
startCalibration
void GazeTracker::startCalibration(
EyedidCalibrationPointNum point_num,
EyedidCalibrationAccuracy criteria,
float left,
float top,
float right,
float bottom,
bool use_previous_calibration = false);
Start calibration process with CalibrationMode and Criteria.
calibrationMode is number of calibration points.
criteria is option that manage calibration process and accuracy.
| Parameters | Type | Description |
|---|---|---|
| point_num | EyedidCalibrationPointNum | Number of points used for calibration |
| criteria | EyedidCalibrationAccuracy | Accuracy option of the calibration (Note: Currently, only the default implementation is available.) |
| left | float | Distance to left edge of the calibration ROI |
| top | float | Distance to top edge of the calibration ROI |
| right | float | Distance to right edge of the calibration ROI |
| bottom | float | Distance to bottom edge of the calibration ROI |
| use_previous_calibration | bool | Use previous calibration data. Default is false. |
Example
gaze_tracker.startCalibration(
kEyedidCalibrationPointFive,
kEyedidCalibrationAccuracyDefault,
0.0f, 0.0f, 1280.0f, 720.0f,
true);
stopCalibration
void GazeTracker::stopCalibration()
Stop the current calibration process.
startCollectSamples
void GazeTracker::startCollectSamples()
Start collect next calibration point data.
This should be called in onCalibrationNextPoint when need to collect new calibration point data.
Example
if (isCalibrating) {
gaze_tracker.startCollectSamples();
}
setCalibrationData
void GazeTracker::setCalibrationData(const std::vector<float>& calibration_data)
Set calibration data without calibrating manually.
| Parameters | Type | Description |
|---|---|---|
| calibration_data | const std::vector<float>& | Stored calibration data |
Example
gaze_tracker.setCalibrationData(calibration_data);
setAttentionRegion
void GazeTracker::setAttentionRegion(float left, float top, float right, float bottom);
Set Status ROI.
| Parameters | Type | Description |
|---|---|---|
| left | float | Set distance to the left edge of the ROI rectangle |
| top | float | Set distance to the top edge of the ROI rectangle |
| right | float | Set distance to the right edge of the ROI rectangle |
| bottom | float | Set distance to the bottom edge of the ROI rectangle |
Example
gaze_tracker.setAttentionRegion(100, 200, 2000, 1000); // Set the ROI: left=100px, top=200px, right=2000px, bottom=1000px
getAttentionRegion
std::vector<float> GazeTracker::getAttentionRegion() const;
Get Status ROI.
| Return Type | Description |
|---|---|
std::vector | contains the value from setAttentionRegion if set, empty std::vector otherwise. |
Example
auto region = gaze_tracker.getAttentionRegion();
removeAttentionRegion
void GazeTracker::removeAttentionRegion();
Remove Status ROI.
converter
converter_type& GazeTracker::converter()
const converter_type& GazeTracker::converter() const
Access or modify GazeTracker's coordinate converter.
See Custom Coordinate.
Example
auto converter = gaze_tracker.converter();
ITrackingCallback
class ITrackingCallback {
public:
ITrackingCallback() = default;
virtual ~ITrackingCallback() = default;
virtual void OnMetrics(/* See below */) = 0;
virtual void OnDrop(uint64_t timestamp) = 0;
};
Summary
| Abstract Functions |
|---|
| onMetrics |
| onDrop |
OnMetrics
virtual void ITrackingCallback::OnMetrics(
std::uint64_t timestamp,
EyedidGazeData gaze_data,
EyedidFaceData face_data,
EyedidBlinkData blink_data,
EyedidUserStatusData user_status_data)
Tracking metrics callback.
| Parameters | Type | Description |
|---|---|---|
| timestamp | uint64_t | Timestamp of the image from GazeTracker::addFrame |
| gaze_info | EyedidGazeData | Gaze information of the image with the specified timestamp from GazeTracker::addFrame |
| face_info | EyedidFaceData | Face information of the image with the specified timestamp from GazeTracker::addFrame |
| blink_info | EyedidBlinkData | Blink information of the image with the specified timestamp from GazeTracker::addFrame |
| user_status_info | EyedidUserStatusData | User Status information of the image with the specified timestamp from GazeTracker::addFrame |
OnDrop
virtual void ITrackingCallback::OnDrop(uint64_t timestamp)
Dropped frame callback.
| Parameters | Type | Description |
|---|---|---|
| timestamp | uint64_t | Timestamp of the dropped frame from GazeTracker::addFrame |
ICalibrationCallback
class ICalibrationCallback {
public:
ICalibrationCallback() = default;
virtual ~ICalibrationCallback() = default;
virtual void OnCalibrationProgress(float progress) = 0;
virtual void OnCalibrationNextPoint(float x, float y) = 0;
virtual void OnCalibrationFinish(const std::vector<float>& calibrationData) = 0;
virtual void OnCalibrationCanceled(const std::vector<float>& calibrationData) = 0;
};
Summary
| Abstract Functions |
|---|
| onCalibrationNextPoint |
| onCalibrationProgress |
| onCalibrationFinish |
| onCalibrationCancel |
OnCalibrationNextPoint
void ICalibrationCallback::OnCalibrationNextPoint(float x, float y)
Coordinate of the next calibration point.
x and y are current calibration point coordinate.
Point (x, y) should be focused during the calibration process.
startCollectSamples shuold be called to collect next calibration data.
| Parameters | Type | Description |
|---|---|---|
| timestamp | uint64_t | Timestamp of the iamge from GazeTracker::addFrame |
| x | float | x coordinate of the current calibration point |
| y | float | y coordinate of the current calibration point |
Example
void OnCalibrationNextPoint(float x, float y) {
this->render_point_x = x;
this->render_point_y = y;
gaze_tracker.startCollectSamples();
}
OnCalibrationProgress
void ICalibrationCallback::OnCalibrationProgress(float progress)
Calibration progress callback.
progress is the calibration progress of the current calibration point (0.0 ~ 1.0).
Next calibration point will be guided when the progress reaches 1.0.
Return 0.0 when start calibration progress.
Return 1.0 when finish calibratino progress.
| Parameters | Type | Description |
|---|---|---|
| progress | float | Calibration progress of the current calibration point |
Example
void OnCalibrationProgress(float progress){
if(progress >= 1.0){
// Collect next calibration point data
} else {
// Collect current calibration point data
}
}
OnCalibrationFinish
void ICalibrationCallback::OnCalibrationFinish(const std::vector<float>& calibration_data)
Calibration finish callback.
calibrationData is the result calibration data.
| Parameters | Type | Description |
|---|---|---|
| calibration_data | const vector<float>& | Result of the calibration data |
OnCalibrationCancel
void ICalibrationCallback::OnCalibrationCancel(const std::vector<float>& calibration_data)
Called when calibration is canceled. This method is triggered when the calibration process is stopped manually by calling GazeTracker.stopCalibration(). It returns the calibration results collected up to the point of cancellation.If no points were successfully calibrated, an empty array is returned.
| Parameters | Type | Description |
|---|---|---|
| calibration_data | const vector<float>& | Result of the calibration data |
CoordConverterV2
template<typename T = double>
class CoordConverterV2 {
public:
using value_type = T;
using transform_type = Matrix<value_type, 2, 2>;
using translate_type = Matrix<value_type, 2, 1>;
using coordinate_type = Matrix<value_type, 2, 1>;
/** default constructor: R=I, T=Zeros */
constexpr CoordConverterV2() = default;
CoordConverterV2(const transform_type& r, const translate_type& t);
CoordConverterV2& transform(const transform_type& t);
transform_type& transform();
const transform_type& transform() const { return transform_; }
CoordConverterV2& translate(const translate_type& t);
translate_type& translate();
const translate_type& translate() const;
coordinate_type convert(const coordinate_type& m) const;
coordinate_type revert(const coordinate_type& m) const;
};
Helper functions
CoordConverterV2<T> eyedid::makeCameraToDisplayConverter(const Point<float>& display_tl, const Point<float>& display_pixels, const Point<float>& display_mm)
CoordConverterV2<T> eyedid::makeDefaultCameraToDisplayConverter(const Point<float>& display_pixels, const Point<float>& display_mm)
CoordConverterV2<T> eyedid::makeDefaultCameraToDisplayConverter(float px_width, float px_height, float mm_width, float mm_height)
CoordConverterV2<T> eyedid::makeNoOpConverter()
See the header file comment
EyedidBoolean
typedef enum EyedidBoolean {
kEyedidFalse = 0,
kEyedidTrue = 1,
} EyedidBoolean;
Substitution of bool type.
EyedidTrackerOptions
typedef struct EyedidTrackerOptions {
EyedidBoolean use_blink = kEyedidFalse;
EyedidBoolean use_user_status = kEyedidFalse;
EyedidBoolean use_gaze_filter = kEyedidTrue;
EyedidBoolean stream_mode = kEyedidTrue;
float camera_fov = static_cast<float>(M_PI) / 4.f;
int max_concurrency = 0;
} EyedidTrackerOptions;
This structure defines the module options related to the use of GazeTracker.
| Name | Type | Description |
|---|---|---|
| use_blink | EyedidBoolean | Enables or disables blink detection. Default is kEyedidFalse. |
| use_user_status | EyedidBoolean | Enables or disables user status detection. Default is kEyedidFalse. |
| use_gaze_filter | EyedidBoolean | Enables or disables gaze data filtering. Default is kEyedidTrue. |
| stream_mode | EyedidBoolean | Enables or disables stream mode for gaze tracking. In stream mode, some input frames may be dropped to maintain real-time performance. Default is kEyedidTrue. |
| camera_fov | float | The horizontal field of view (FOV) of the camera, in radians. Default is π/4 radians (45 degrees). |
| max_concurrency | int | Sets the maximum number of concurrent threads for processing, based on the number of hardware cores available. Default is 0, which means it will automatically determine the optimal number of threads based on the system's hardware capabilities. |
EyedidCalibrationPointNum
typedef enum EyedidCalibrationPointNum {
kEyedidCalibrationPointOne = 1,
kEyedidCalibrationPointFive = 5,
} EyedidCalibrationPointNum;
The number of points for calibration.
| Name | Description |
|---|---|
| kEyedidCalibrationPointOne | One-point calibration mode |
| kEyedidCalibrationPointFive | Five-point calibration mode |
EyedidCalibrationAccuracy
typedef enum EyedidCalibrationAccuracy {
kEyedidCalibrationAccuracyDefault = 0,
//kEyedidCalibrationAccuracyLow,
//kEyedidCalibrationAccuracyHigh,
} EyedidCalibrationAccuracy;
The enum containing options that manage calibration process and accuracy.
| Name | Description |
|---|---|
| kEyedidCalibrationAccuracyDefault | Starts default calibration |
| kEyedidCalibrationAccuracyLow | Start calibration with lower threshold for each step. (May result in lower accuracy) |
| kEyedidCalibrationAccurarcyHigh | Start calibration with higher threshold for each step. (Likely to result in higher accuracy) |
EyedidGazeData
typedef struct EyedidGazeData {
float x;
float y;
float fixation_x;
float fixation_y;
EyedidTrackingState tracking_state;
EyedidEyeMovementState movement_state;
} EyedidGazeData;
This struct encapsulates information related to gaze.
| Parameters | Type | Description |
|---|---|---|
| x | float | x coordinate of gaze point |
| y | float | y coordinate of gaze point |
| fixation_x | float | x coordinate value of last fixation point. |
| fixation_y | float | y coordinate value of last fixation point. |
| tracking_state | EyedidTrackingState | Tracking state value |
| eye_movement_state | EyedidEyeMovementState | Eye movement state value |
EyedidFaceData
typedef struct EyedidFaceData {
float score;
float left;
float top;
float right;
float bottom;
float yaw;
float pitch;
float roll;
float center_x;
float center_y;
float center_z;
} EyedidFaceData;
This struct encapsulates information related to face.
| Parameters | Type | Description |
|---|---|---|
| score | float | Value of facial recognition confidence (0.0 ~ 1.0). |
| left | float | Left of the face. |
| top | float | Top of the face. |
| right | float | Right of the face. |
| bottom | float | Bottom of the face. |
| pitch | float | Pitch is the rotation around the X-axis of an object. It is also known as the attitude angle. In aviation, it is often referred to as the nose-up or nose-down angle. |
| yaw | float | Yaw is the rotation around the Y-axis of an object. It is also known as the heading angle. In aviation, it is often referred to as the bearing or compass angle. |
| roll | float | Roll is the rotation around the Z-axis of an object. It is also known as the bank angle. In aviation, it is often referred to as the wing-up or wing-down angle. |
| center_x | float | The x distance of the center of the face from the camera. The unit is mm. |
| center_y | float | The y distance of the center of the face from the camera. The unit is mm. |
| center_z | float | The z distance of the center of the face from the camera. The unit is mm. |
EyedidBlinkData
typedef struct EyedidBlinkData {
EyedidBoolean is_blink;
EyedidBoolean is_blink_left;
EyedidBoolean is_blink_right;
float left_openness;
float right_openness;
} EyedidBlinkData;
This struct encapsulates information related to blink.
| Parameters | Type | Description |
|---|---|---|
| isBlinkLeft | bool | Left blink flag |
| isBlinkRight | bool | Right blink flag |
| isBlink | bool | Blink flag |
| leftOpenness | float | Open ratio of left eye. |
| rightOpenness | float | Open ratio of right eye. |
EyedidUserStatusData
typedef struct EyedidUserStatusData {
EyedidBoolean is_drowsy;
float drowsiness_intensity;
float attention_score;
} EyedidUserStatusData;
This struct encapsulates information related to user's status.
| Parameters | Type | Description |
|---|---|---|
| is_drowsy | EyedidBoolean | User drowsiness flag |
| drowsiness_intensity | float | Level of drowsiness intensity (0 to 1) |
| attention_score | float | User attention rate score |
EyedidTrackingState
typedef enum EyedidTrackingState {
kEyedidTrackingSuccess = 0,
kEyedidTrackingFaceMissing,
kEyedidTrackingGazeNotFound,
} EyedidTrackingState;
The enum containing tracking state types used in GazeInfo.
| Name | Description |
|---|---|
| kEyedidTrackingSuccess | Face alignment is in the best position, tracking is successful with valid accuracy |
| kEyedidTrackingFaceMissing | Face is missing (Gaze tracking has failed) |
| kEyedidTrackingGazeNotFound | Gaze tracking is not possible because the eyes are closed. |
EyedidEyeMovementState
typedef enum EyedidEyeMovementState {
kEyedidEyeMovementFixation = 0,
kEyedidEyeMovementSaccade = 2,
kEyedidEyeMovementUnknown = 3,
} EyedidEyeMovementState;
The enum containing eye movement state types used in GazeInfo.
| Name | Description |
|---|---|
| kEyedidEyeMovementFixation | Eye movement has fixation. |
| kEyedidEyeMovementSaccade | Eye movement has saccade. |
| kEyedidEyeMovementUnknown | Eye movement is unknown. |