Cloud API Documentation
 Back to top

Detection API

Introduction

The Detection API returns the location of any people and faces found in an image. Faces can be analyzed for gender, age, pose, or emotion; and a landmark detector can find the various facial features in the detected faces, including eyes, nose and mouth, by fitting 68 landmark points to those features.

Before proceeding, please ensure you have created a Sighthound Cloud API Token. Code samples for a variety of languages are available on the Sighthound Cloud demo page.

Usage Notes

API Base URL

Select the appropriate base URL for your environment and Cloud API account level. All Cloud accounts can use the free, development service for up to 5,000 API calls per month. Paid accounts (Basic, Pro, and Enterprise) have access to separate, production-ready servers.

  • Development: https://dev.sighthoundapi.com

  • Production (Paid Accounts): https://prod.sighthoundapi.com

General

  • Blurry scenes (possibly caused by camera settings or weather conditions) will reduce the accuracy of the object detector. The blurrier the object the more difficult to detect.

  • Good lighting conditions are important for detection. Detections may not trigger if the scene is too dark.

  • Angle of the camera to the object being detected should be within 45 degrees for best results.

Person Detection

  • The size of the person in the image should be at least 96x40 for upper/full body and 72x64 for head and shoulders.

  • For detecting people, the head and shoulders of the person are important. In cases where the head and shoulders of the person are occluded detection might not be possible.

  • Unusual poses (e.g. crawling, lying down) are currently not handled. The system can be trained to handle such poses in the future.

Face Detection

  • The size of the face in the image should be at least 40x40 pixels.

  • The detection will handle faces occluded up to approximately 50%, given that the eyes are not occluded. Eyes are an essential for reliable detection.

  • The system is able to handle frontal with a high degree of accuracy. Profile and rotated faces (maximum of 45 degree, both in-plane and out-of-plane) are handled but are less accurate.

  • Wearable objects (e.g. glasses, cap, hat) should not cause as problems as long as more than 50% of the face is visible (including the eyes).

  • Scores greater than 0.5 are fairly confident for attributes returned with confidence scores (e.g. age and gender).

Detections

Detect Objects

Notes

  • Supported image formats are JPEG, GIF, and PNG.

  • Images larger than 1920x1080 will be scaled to a size within those bounds while respecting aspect ratio.

  • The minimum size supported for detection of a face in the image is 40x40 pixels. For better results, use faces sized 256x256 pixels or higher.

Using Image URL
POST/v1/detections?type={type}&faceOption={faceOption}

Example URI

POST https://dev.sighthoundapi.com/v1/detections?type=face,person&faceOption=gender,landmark,age,emotion,pose
URI Parameters
HideShow
type
string (optional) Example: face,person

Comma separated list of object categories to detect. Valid options are ‘all’, ‘face’, ‘person’. Default is ‘all’.

faceOption
string (optional) Example: gender,landmark,age,emotion,pose

For type ‘face’, additional detections can be performed by passing comma-separated list of values. Valid options are ‘gender’, ‘landmark’, ‘emotion’, ‘pose’, and ‘age’. Default returns face bounding box only.

Request

Request Body Attributes (JSON)

  • image (string): The image to analyze. This can be either a URL to an image (authentication data in URL is accepted) or inline as base64 encoded data. The maximum accepted image size is 10Mb.
Headers
Content-Type: application/json
X-Access-Token: developersOwnCloudAccessToken
Body
{
  "image": "http://www.example.com/path/to/image.jpg"
}
Response  200

Successful queries will return a 200 response with a JSON body describing all detected objects and the attributes of the processed image.

Response Attributes (JSON)

  • image (object): Information about the processed image.

    • width (integer): Pixel width of source image
    • height (integer): Pixel height of source image
    • orientation (integer): EXIF code representing rotation and/or flipping of source Image needed to align the source photo with the detection coordinates. Only applicable for photos with EXIF data (e.g., photos taken on a mobile device). Defaults to 1 (no rotation or flip), else the value found in the image’s EXIF data and indicates that bounding vertices have been translated to match that coordinate space.
  • objects (array): An array of all detected objects. Each object contains:

    • type (string): The label applied to the object, face or person.
    • boundingBox- (object): An object containing x, y, width, and height values defining the location of the object in the image. The top left pixel of the image represents coordinate (0,0).
    • attributes (object): describes attributes of the detected Object
      • gender (string): The computed gender of the face. Always returns a value of either male or female. Returned only for objects with type=face.
      • genderConfidence (float): A normalized value between 0 - 1 that represents the confidence that the computed gender is correct.
      • age (integer): The estimated age of the face. Always returns a value from 0 to 100. Returned only for objects with type=face.
      • ageConfidence (float): A normalized value between 0 - 1 that represents the confidence of the estimated age.
      • emotion (string): The top computed emotion of the face. Returned only for objects with type=face.
      • emotionConfidence (float): A normalized value between 0 - 1 that represents the confidence of the top computed emotion. Returned only for objects with type=face.
      • emotionsAll (object): All computed emotions. Returned only for objects with type=face.
      • pose (object): The pitch, roll, and yaw of the detected pose. Returned only for objects with type=face.
      • frontal (string): Frontal is true if the face is looking at camera and false if it’s a profile view. Returned only for objects with type=face.
    • landmarks (object): Contains x/y coordinates for several parts of the detected face. See the Sighthound Cloud Demo for placement.
Headers
Content-Type: application/json
Body
{
    "image": {
        "width": 1280, 
        "height": 960, 
        "orientation": 1
    },
    "objects": [
        {
            "type": "person",
            "boundingBox": { 
                "x": 363, 
                "y": 182, 
                "height": 778, 
                "width": 723
            } 
        },
        {
            "type": "face",
            "boundingBox": {
                "x": 508, 
                "y": 305, 
                "height": 406, 
                "width": 406
            },
            "attributes": {
                "gender": "male", 
                "genderConfidence": 0.9883, 
                "age":25,
                "ageConfidence": 0.2599,
                "emotion": "happiness",
                "emotionConfidence": 0.9943,
                "emotionsAll": {
                    "neutral": 0.0018,
                    "sadness": 0.0009,
                    "disgust": 0.0002,
                    "anger": 0.0003,
                    "surprise": 0,
                    "fear": 0.0022,
                    "happiness": 0.9943
                },
                "pose":{
                    "pitch":-18.4849,
                    "roll":0.854,
                    "yaw":-4.2123
                },
                "frontal": true
            },
            "landmarks": {
                "faceContour": [[515,447],[517,491]...[872,436]],
                "noseBridge": [[710,419],[711,441]...[712,487]],
                "noseBall": [[680,519],[696,522]...[742,518]],
                "eyebrowRight": [[736,387],[768,376]...[854,394]],
                "eyebrowLeft": [[555,413],[578,391]...[679,391]],
                "eyeRight": [[753,428],[774,414]...[777,432]],
                "eyeRightCenter": [[786,423]],
                "eyeLeft": [[597,435],[617,423]...[619,442]],
                "eyeLeftCenter": [[630,432]],
                "mouthOuter": [[650,590],[674,572]...[675,600]],
                "mouthInner": [[661,587],[697,580]...[697,584]]
            }
        }
    ]
}
Response  400
HideShow

Invalid request.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  401
HideShow

API Token does not exist, is inactive, suspended, or deleted.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  403
HideShow

The API Token’s limit has been exceeded.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  405
HideShow

Invalid request method. Only POST is supported.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  406
HideShow

Invalid request path.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  413
HideShow

The image content is too large.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  415
HideShow

Invalid Content-Type

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  500
HideShow

Detection failed due to an unexpected condition.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  502
HideShow

The system is busy or unavailable.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

From Image File
POST/v1/detections?type={type}&faceOption={faceOption}

Example URI

POST https://dev.sighthoundapi.com/v1/detections?type=face,person&faceOption=gender,landmark,age,pose,emotion
URI Parameters
HideShow
type
string (optional) Example: face,person

Comma separated list of object categories to detect. Valid options are ‘all’, ‘face’, ‘person’. Default is ‘all’.

faceOption
string (optional) Example: gender,landmark,age,pose,emotion

For type ‘face’, additional detections can be performed by passing comma-separated list of values. Valid options are ‘gender’, ‘landmark’, ‘emotion’, ‘pose’, and ‘age’. Default returns face bounding box only.

Request

Request Body Attributes (JSON)

  • Binary Stream of an image file to upload
Headers
Content-Type: application/octet-stream
X-Access-Token: developersOwnCloudAccessToken
Body
< Image binary stream >
Response  200

Successful queries will return a 200 response with a JSON body describing all detected objects and the attributes of the processed image.

Response Attributes (JSON)

  • image (object): Information about the processed image.

    • width (integer): Pixel width of source image
    • height (integer): Pixel height of source image
    • orientation (integer): EXIF code representing rotation and/or flipping of source Image needed to align the source photo with the detection coordinates. Only applicable for photos with EXIF data (e.g., photos taken on a mobile device). Defaults to 1 (no rotation or flip), else the value found in the image’s EXIF data and indicates that bounding vertices have been translated to match that coordinate space.
  • objects (array): An array of all detected objects. Each object contains:

    • type (string): The label applied to the object, face or person.
    • boundingBox (object): An object containing x, y, width, and height values defining the location of the object in the image. The top left pixel of the image represents coordinate (0,0).
    • attributes (object): describes attributes of the detected Object
      • gender (string): The computed gender of the face. Always returns a value of either male or female. Returned only for objects with type=face.
      • genderConfidence (float): A normalized value between 0 - 1 that represents the confidence that the computed gender is correct.
      • age (integer): The estimated age of the face. Always returns a value from 0 to 100. Returned only for objects with type=face.
      • ageConfidence (float): A normalized value between 0 - 1 that represents the confidence of the estimated age.
      • emotion (string): The top computed emotion of the face. Returned only for objects with type=face.
      • emotionConfidence (float): A normalized value between 0 - 1 that represents the confidence of the top computed emotion. Returned only for objects with type=face.
      • emotionsAll (object): All computed emotions. Returned only for objects with type=face.
      • pose (object): The pitch, roll, and yaw of the detected pose. Returned only for objects with type=face.
      • frontal (string): Frontal is true if the face is looking at camera and false if it’s a profile view. Returned only for objects with type=face.
    • landmarks (object): Contains x/y coordinates for several parts of the detected face. See the Sighthound Cloud Demo for placement.
Headers
Content-Type: application/json
Body
{
    "image": {
        "width": 1280, 
        "height": 960, 
        "orientation": 1
    },
    "objects": [
        {
            "type": "person",
            "boundingBox": { 
                "x": 363, 
                "y": 182, 
                "height": 778, 
                "width": 723
            } 
        },
        {
            "type": "face",
            "boundingBox": {
                "x": 508, 
                "y": 305, 
                "height": 406, 
                "width": 406
            },
            "attributes": {
                "gender": "male", 
                "genderConfidence": 0.9883, 
                "age":25,
                "ageConfidence": 0.2599,
                "emotion": "happiness",
                "emotionConfidence": 0.9943,
                "emotionsAll": {
                    "neutral": 0.0018,
                    "sadness": 0.0009,
                    "disgust": 0.0002,
                    "anger": 0.0003,
                    "surprise": 0,
                    "fear": 0.0022,
                    "happiness": 0.9943
                },
                "pose":{
                    "pitch":-18.4849,
                    "roll":0.854,
                    "yaw":-4.2123
                },
                "frontal": true
            },
            "landmarks": {
                "faceContour": [[515,447],[517,491]...[872,436]],
                "noseBridge": [[710,419],[711,441]...[712,487]],
                "noseBall": [[680,519],[696,522]...[742,518]],
                "eyebrowRight": [[736,387],[768,376]...[854,394]],
                "eyebrowLeft": [[555,413],[578,391]...[679,391]],
                "eyeRight": [[753,428],[774,414]...[777,432]],
                "eyeRightCenter": [[786,423]],
                "eyeLeft": [[597,435],[617,423]...[619,442]],
                "eyeLeftCenter": [[630,432]],
                "mouthOuter": [[650,590],[674,572]...[675,600]],
                "mouthInner": [[661,587],[697,580]...[697,584]]
            }
        }
    ]
}
Response  400
HideShow

Invalid request.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  401
HideShow

API Token does not exist, is inactive, suspended, or deleted.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  403
HideShow

The API Token’s limit has been exceeded.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  405
HideShow

Invalid request method. Only POST is supported.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  406
HideShow

Invalid request path.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  413
HideShow

The image content is too large.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  415
HideShow

Invalid Content-Type

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  500
HideShow

Detection failed due to an unexpected condition.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }
Response  502
HideShow

The system is busy or unavailable.

Response Attributes (JSON)

  • error (string): A human readable interpretation of the HTTP status code.

  • reason (string): A short description of what went wrong.

  • reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.

  • details (string): An optional key which will contain additional information on the error if available.

Headers
Content-Type: application/json
Body
{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }