Refonte AI

Create Semantic Segmentation Annotation Task

The segmentannotation task is created by this endpoint. One of our labelers will view the submitted image in this task and categorize the image's pixels based on the labels provided. You will obtain a detailed, meaningful segmentation of the image at the pixel level. Moreover, we provide LabelDescription objects for instance-aware semantic segmentations, commonly known as panoptic segmentations. The required parameters for this task are attachment and labels

The attachment is a URL to an image you'd like to be segmented. labels is an array of strings or LabelDescription objects describing the different types of objects you'd like to segment the image with. You can optionally provide additional markdown-enabled or Google Doc-based instructions via the instruction parameter.

You can also optionally set `allow_unlabeled` to true, which will allow the existence of unlabeled pixels in the task response - otherwise, all pixels in the image will be classified (in which case it's important that there are labels for everything in the image, to avoid misclassification). The response you will receive will be a series of images where each pixel's value corresponds to the label, either via a numerical index or a color mapping. You will also get separate masks for each label for convenience. You will also get separate masks for each label for convenience.If the request successful, refonte will return the generated task object, at which point you should store the `task_id` to have a permanent reference to the task.

Body Params

project string

The name of the project to associate this task with.

batch string

A google Doc embedded in an iframe that supports markdown and explains how to perform the segmentation. Markdown can be used to display sample images, organize your instructions, and much more. For further information, see our best practices for instruction. Please do not set this option for Refonte Rapid projects unless you want to override the project level instructions.

callback string

The full url (including the scheme http:// or https://) or email address of the callback that will be used when the task is completed.

attachment string required

A URL to the image you'd like to be segmented.

attachment_type string

Describes what type of file the attachment is. We currently only support image for the segmentannotation.

labels array of strings required

An array of strings or LabelDescription objects describing the different types of objects you'd like to be used to segment the image. You may include at most 50 labels.

annotation_attributes object

You can add other attributes here that you would like to record for each annotation. This is limited to annotations on instances. For additional information regarding annotation attributes, see Annotation Attributes.

allow_unlabeled boolean

Whether or not this image can be completed without every pixel being labeled.

hypothesis object

Annotations that can be edited and used to initiate a job. When you want annotators to further improve the prelabels that a model you ran to prelabel the task has produced, this is helpful. For further information, see the Segmentation Hypothesis Format.

metadata object

A group of key-value pairs that can be fastened to an object used for tasks. It can be helpful for organizing and storing more task-related information. 10KB maximum. For further information, see the Metadata section.

context_attachments array of objects

An array of objects in the form of ["attachment": "link to actual attachment] to show to taskers as a reference. Context images themselves can not be labeled. Context images will appear like this in the UI. You cannot use the task's attachment url as a context attachment's url.

unique_id string

A arbitrary ID that you can assign to a task and then query for later. This ID must be unique across all projects under your account, otherwise the task submission will be rejected. See Avoiding Duplicate Tasks for more details.

clear_unique_id_on_error boolean

If set to true, the task's unique id will be unset if a task fails after it has been submitted. This parameter enables procedures in which errors can be automatically corrected by resubmitting the same unique ID.

tags array of strings

Labels that you can give a task at random. Each job may have a maximum of 5 tags. The task retrieval API allows you to query tasks that have particular tags.

Request

Python

POST  /v1/task/segmentannotation
import requests

# Replace with your actual API key
API_KEY = 'your_api_key_here'

# Define the URL for the API endpoint
url =
 "https://api.refonte.com/v1/task/segmentannotation"

# Define the payload for the segment annotation task
payload = {
    "instruction": "**Instructions:** Please label all the things",
    "attachment": "https://i.imgur.com/iDZcXfS.png",
    "attachment_type": "image",
    "annotation_attributes": {
        "newKey": {
            "type": "type",
            "description": "description",
            "choices": "choices",
            "conditions": {
                "label_condition": ["car", "car2"],
                "attribute_conditions": {
                    "newKey": "New Value",
                    "newKey-1": "New Value"
                }
            }
        }
    },
    "allow_unlabeled": False,
    "metadata": {
        "newKey": "New Value",
        "newKey-1": "New Value"
    },
    "project": "Project Name",
    "batch": "Batch Name",
    "callback_url": "http://www.example.com/callback",
    "labels": [["vehicle"], "vehicle 2", "vehicle 3"],
    "context_attachments": [{"attachment": "attachment"}, {"attachment": "attachment2"}],
    "unique_id": "unique_id",
    "clear_unique_id_on_error": True,
    "tags": ["tag", "tag2"]
}

# Set up the headers for the request
headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

# Adding authentication to the POST request
# The auth parameter requires a tuple with the API key and an empty string
response = requests.post(url, json=payload, headers=headers, auth=(API_KEY, ''))

# Print the response text to see the result
print(response.text)

Response

{
  "task_id": "string",
  "created_at": "string",
  "type": "segmentannotation",
  "status": "pending",
  "instruction": "string",
  "is_test": false,
  "urgency": "standard",
  "metadata": {},
  "project": "string",
  "callback_url": "string",
  "updated_at": "string",
  "work_started": false,
  "params": {
    "allow_unlabeled": false,
    "labels": [
      null
    ],
    "instance_labels": [
      null
    ],
    "attachment_type": "image",
    "attachment": "https://i.imgur.com/SudOKhq.jpg"
  }
}

Create General Video Annotation Task

The videoannotation job is created by this endpoint. Refonte will annotate each frame of a sequence of photos taken from a movie (referred to as "frames") with the Geometries (box, polygon, line, point, cuboid, and ellipse) that you designate.

You can optionally provide additional markdown-enabled or Google Doc-based instructions via the instruction parameter. You may also optionally specify events_to_annotate, a list of strings describing events section to annotate in the video. If the request is successful, Refonte will return the generated task object, at which point you should store the task_id to have a permanent reference to the task.

Body Params

project string

The name of the project to associate this task with.

batch string

The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch's project. For Refonte Rapid projects specifying a batch is required. See Batches section for more details.

instruction string

A markdown-enabled string or iframe embed google doc explaining how to do the task. You can use markdown to show example images, give structure to your instructions, and more. See our instruction best practices for more details. For Refonte Rapid projects, DO NOT set this field unless you specifically want to override the project level instructions.

callback_url string

The full url (including the scheme http:// or https://) or email address of the callback that will be used when the task is completed.

attachments array of strings

An array of URLs for the frames you'd like to be annotated. These image frames are stitched together to create a video. This is required if attachment_type is image and must be omitted if attachment_type is video.

attachments string

A URL pointing to the video file attachment. Only the mp4, webm, and ogg formats are supported.

attachment_type string

Describes what type of file the attachment(s) are. The only options are image and video.

geometries object required

An object mapping `box`, `polygon`, `line`, `point`, `cuboid`, or `ellipse` to Geometry objects

annotation_attributes object

See the Annotation Attributes section for more details about annotation attributes.

events_to_annotate array of strings

The list of events to annotate.

links object

Use this field to define links between annotations. See Links for more details about links.

frame_rate int32

The number of frames per second to annotate.

paddingX int32

The amount of padding in pixels added to the left and right of each video frame. Overrides padding if set.

paddingY int32

The amount of padding in pixels added to the top and bottom of each video frame. Overrides padding if set.

hypothesis object

Annotations that can be edited and used to initiate a job. When you want annotators to further improve the prelabels that a model you ran to prelabel the task has produced, this is helpful. For further information, see the Segmentation Hypothesis Format.

base_annotations object

Annotations that can be edited and have the ability to be "locked" are what a task should start with. When you want annotators to further improve the prelabels that a model you ran to prelabel the task has produced, this is helpful. The annotations field, which follows the same format as the response's annotations field, must be present.

can_add_base_annotations boolean

Whether or not new annotations can be added to the task if base_annotations are used. If set to true, new annotations can be added to the task in addition to base_annotations. If set to false, new annotations will not be able to be added to the task.

can_edit_base_annotations boolean

Whether using base_annotations allows for the addition of new annotations to the task. Set to true to allow for the adding of annotations to the job beyond base_annotations. New annotations cannot be added to the task if this option is set to false.

can_edit_base_annotation_labels boolean

Whether base_annotations in the job are editable. Base_annotations, if set to true, allows the tasker to modify the annotation's position, characteristics, and other details. Every feature of base_annotations will be locked if set to false.

can_delete_base_annotations boolean

Whether base_annotations labels in the job are editable. The tasker can modify the base_annotations label if it is set to true. The label will become locked if it is set to false.

metadata boolean

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. Max 10KB.

priority int32

A value of 10, 20, or 30 that defines the priority of a task within a project. The higher the number, the higher the priority.

unique_id string

A randomly generated ID that you can link to a job and then retrieve at a later time. If this ID isn't distinct across all projects associated with your account, the task submission will be declined.

clear_unique_id_on_error boolean

If set to true, the task's unique id will be unset if a task fails after it has been submitted. This parameter enables procedures in which errors can be automatically corrected by resubmitting the same unique ID.

tags array of strings

Labels that you can give a task at random. Each job may have a maximum of 5 tags. The task retrieval API allows you to query tasks that have particular tags.

Request

Python

POST  /v1/task/videoannotation
import requests

# Replace with your actual API key
API_KEY = 'your_api_key_here'

# Define the URL for the API endpoint
url = "https://api.refonte.com/v1/task/videoannotation"

# Define the payload for the video annotation task
payload = {
    "instruction": "**Instructions:** Please label all the things",
    "attachments": ["https://static.refonte.com/rive_0071_sync/image_02/data/05.png",
    "https://static.refonte.com/rive_0071_sync/image_02/data/08.png"]
    "attachment_type": "image",
    "geometries": {
        "box": {
            "min_height": 10,
            "min_width": 10,
            "can_rotate": True,
            "integer_pixels": False
        },
        "polygon": {
            "min_vertices": 10,
            "max_vertices": 20,
            "objects_to_annotate": ["large vehicle"]
        },
        "line": {
            "min_vertices": 10,
            "max_vertices": 20,
            "objects_to_annotate": ["large vehicle"]
        },
        "point": {
            "objects_to_annotate": ["large vehicle", "large vehicle"]
        },
        "cuboid": {
            "min_height": 10,
            "min_width": 10,
            "camera_intrinsics": {
                "fx": 10,
                "fy": 10,
                "cx": 10,
                "cy": 10,
                "skew": 10,
                "refontefactor": 10
            },
            "camera_rotation_quaternion": {
                "w": 10,
                "x": 10,
                "y": 10,
                "z": 10
            },
            "camera_height": 10
        },
        "ellipse": {
            "objects_to_annotate": ["large vehicle"]
        }
    },
    "events_to_annotate": ["event_1_name", "event_2_name"],
    "frame_rate": 1,
    "start_time": 10,
    "padding": 10,
    "paddingX": 10,
    "metadata": {
        "newKey": "New Value",
        "newKey-1": "New Value"
    },
    "priority": 30,
    "project": "Project Name",
    "batch": "Batch Name",
    "callback_url": "http://www.example.com/callback",
    "attachment": "attachment_url",
    "duration_time": 10,
    "paddingY": 10,
    "unique_id": "unique_id",
    "clear_unique_id_on_error": True,
    "tags": ["tag1", "tag2"]
}

# Set up the headers for the request
headers = {
    "accept": "application/json", 
    "content-type": "application/json" 
}

# Adding authentication to the POST request
# The auth parameter requires a tuple with the API key and an empty string
response = requests.post(url, json=payload, headers=headers, auth=(API_KEY, ''))

# Print the response text to see the result
print(response.text)

Response

{
  "task_id": "string",
  "created_at": "string",
  "type": "videoannotation",
  "status": "pending",
  "instruction": "string",
  "is_test": false,
  "urgency": "standard",
  "metadata": {},
  "project": "string",
  "callback_url": "string",
  "updated_at": "string",
  "work_started": false,
  "params": {
    "attachment_type": "website",
    "attachment": [
      null
    ],
    "geometries": {
      "box": {
        "objects_to_annotate": [
          null
        ],
        "min_height": 10,
        "min_width": 10
      },
      "polygon": {
        "objects_to_annotate": [
          null
        ]
      },
      "point": {
        "objects_to_annotate": [
          null
        ]
      }
    },
    "annotation_attributes": {
      "additionalProp": {
        "description": "string",
        "choice": "string"
      }
    },
    "events_to_annotate": [
      null
    ],
    "with_labels": true
  }
}

Create Video Playback Annotation Task

This endpoint creates a `videoplaybackannotation`task. In this task, we will view the given video file and draw annotations around the specified objects.

A URL to the video file must be included in the attachment. It is available in ogg, webm, or mp4 formats. With the instruction argument, you may optionally supply more Google Doc-based or markdown-enabled instructions.

The number of frames per second that will be utilized to annotate the provided video will depend on the optional frame_rate that you supply. One is the default value. Additionally, you have the option to provide events_to_annotate, a list of strings that describe the area of the video that you want to annotate. Refonte will return the produced task object if the request is successful, at which time you should save the task_id so you always have a reference to the task.

Body Params

project string

The name of the project to associate this task with.

Body Params

batch string

The batch name that this job should be associated with. It should be noted that if a batch is supplied, the task will be automatically connected with the batch's project; thus, there is no need to mention the project. It is necessary to define a batch for Refonte Rapid projects.

Body Params

instruction string

The instructions can be found in an iframe embedded Google Doc or a markdown-enabled string. Markdown can be used to display sample images, organize your instructions, and much more. For further information, see our best practices for instruction. Please do not set this option for Refonte Rapid projects unless you want to override the project level instructions.

Body Params

callback_url string

The full url (including the scheme http:// or https://) or email address of the callback that will be used when the task is completed.

attachment string

A URL pointing to the video file attachment. Only the mp4, webm, and ogg formats are supported.

attachment_type string

Describes what type of file the attachment(s) are. The only options are image and video.

geometries object required

Describes An object mapping box, polygon, line, point, cuboid, or ellipse to Geometry objects

annotation_attributes object

Describes See the Annotation Attributes section for more details about annotation attributes.

events_to_annotate int32

The list of events to annotate.

duration_time array of strings

The duration of the video in seconds. This is ignored if attachment_type is image. Default is full video length.

frame_rate object

The number of frames to capture in one second. This is ignored if attachment_type is image.

start_time int32

The start time in seconds. This is ignored if attachment_type is image.

padding int32

The amount of padding in pixels added to the top, bottom, left, and right of each video frame. This allows labelers to extend annotations outside of the frames.

paddingX int32

The amount of padding in pixels added to the left and right of each video frame. Overrides padding if set.

paddingY int32

The amount of padding in pixels added to the top and bottom of each video frame. Overrides padding if set.

base_annotations object

Annotations that can be edited and have the ability to be "locked" are what a task should start with. When you want annotators to further improve the prelabels that a model you ran to prelabel the task has produced, this is helpful. The annotations field, which follows the same format as the response's annotations field, must be present.

can_add_base_annotations boolean

If base_annotations are utilized, whether or not new annotations can be added to the task. In addition to base_annotations, additional annotations can be provided to the job if set to true. The task will not allow the addition of additional annotations if set to false.

can_edit_base_annotations boolean

Whether or not base_annotations can be edited in the task. If set to true, base_annotations can be edited by the tasker (position of annotation, attributes, etc). If set to false, all aspects of base_annotations will be locked.

can_edit_base_annotation_labels boolean

Whether or not base_annotations labels can be edited in the task. If set to true, the label of base_annotations can be edited by the tasker. If set to false, the label will be locked.

can_delete_base_annotations boolean

Whether or not base_annotations can be removed from the task. If set to true, base_annotations can be deleted from the task. If set to false, base_annotations cannot be deleted from the task.

metadata object

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. Max 10KB.

priority int32

A value of 10, 20, or 30 that defines the priority of a task within a project. The higher the number, the higher the priority.

unique_id string

A arbitrary ID that you can assign to a task and then query for later. This ID must be unique across all projects under your account, otherwise the task submission will be rejected.

clear_unique_id_on_error boolean

If set to be true, if a task errors out after being submitted, the unique id on the task will be unset. This param allows workflows where you can re-submit the same unique id to recover from errors automatically

tags array of strings

Arbitrary labels that you can assign to a task. At most 5 tags are allowed per task. You can query tasks with specific tags through the task retrieval API.

Request

Python

POST  /task/videoplaybackannotation
import requests

# Replace with your actual API key
API_KEY = 'your_api_key_here'

# Define the URL for the API endpoint
url = "https://api.refonte.com/v1/task/videoplaybackannotation"

# Define the payload for the video playback annotation task
payload = {
    "instruction": "**Instructions:** Please label all the things",
    "attachments": ["https://static.refonte.com/rive_0071_sync/image_02/data/05.png",
    "https://static.refonte.com/rive_0071_sync/image_02/data/08.png"]
    "attachment": "s3-us-west-2.amazonaws.com/demos/multimodal-video.mp4",
    "attachment_type": "image",
    "geometries": {
        "box": {
            "min_height": 10,
            "min_width": 10
        },
        "polygon": {
            "min_vertices": 1,
            "max_vertices": " "
        },
        "line": {
            "min_vertices": 1,
            "max_vertices": " "
        },
        "point": {
            "x": " ",
            "y": " "
        },
        "cuboid": {
            "min_height": 0,
            "min_width": 0,
            "camera_intrinsics": {
                "fx": " ",
                "fy": " ",
                "cx": " ",
                "cy": " ",
                "skew": 0,
                "refontefactor": 1
            },
            "camera_rotation_quaternion": {
                "w": " ",
                "x": " ",
                "y": " ",
                "z": " "
            },
            "camera_height": " "
        }
    },
    "frame_rate": 1,
    "padding": 0,
    "paddingX": 0,
    "paddingY": 0,
    "priority": 30
}

# Set up the headers for the request
headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

# Adding authentication to the POST request
# The auth parameter requires a tuple with the API key and an empty string
response = requests.post(url, json=payload, headers=headers, auth=(API_KEY, ''))

# Print the response text to see the result
print(response.text)

Response

{
  "task_id": "string",
  "created_at": "string",
  "type": "videoannotation",
  "status": "pending",
  "instruction": "string",
  "is_test": false,
  "urgency": "standard",
  "metadata": {},
  "project": "string",
  "callback_url": "string",
  "updated_at": "string",
  "work_started": false,
  "params": {
    "attachment_type": "website",
    "attachment": [
      null
    ],
    "geometries": {
      "box": {
        "objects_to_annotate": [
          null
        ],
        "min_height": 10,
        "min_width": 10
      },
      "polygon": {
        "objects_to_annotate": [
          null
        ]
      },
      "point": {
        "objects_to_annotate": [
          null
        ]
      }
    },
    "annotation_attributes": {
      "additionalProp": {
        "description": "string",
        "choice": "string"
      }
    },
    "events_to_annotate": [
      null
    ],
    "with_labels": true
  }
}

Updated about 2 months ago