Refonte AI

Project Overview

Using projects, one may group related jobs together, communicate parameters between them, and decide which employees are permitted to work on certain task classes. Tasks that are created under a project will inherit its associated parameters. See Update Parameters (Project) for more information. Defining and updating project-level instructions and allowing tasks to inherit from the most recent version of the project instructions is a popular use-case. When creating a task, include a reference to the project name in the `project` parameter. For example, to associate a task with project `my_project`, include `project: my_project` in the task creation request. Keep in mind that projects made with your test API key can only be used to make test tasks; the same is true for projects made with your live API key and live tasks.

Using Project-level parameters

Refonte.Ai organizes Tasks, the work unit, using Projects. There are numerous adjustable input parameters available for each type of job (Image Annotation, Text Collection, Lidar Annotation) that can be utilized during task creation to specify the exact form of the task and the format of the result. Task definitions may be kept in a hybrid manner, on the task level, or on the project level.

Task vs. Project Level Parameters

Task Only

The API call to create a Task sets all of the Task parameters (definitions). The Project's sole purpose is to gather and arrange comparable groupings of work; it has no further restrictions. A "Task Only" approach would be achieved using the task-specific endpoints and parameters that are outlined in our documentation.

Project Only

The API call to create a Task sets all of the Task parameters (definitions). The Project's sole purpose is to gather and arrange comparable groupings of work; it has no further restrictions. A "Task Only" approach would be achieved using the task-specific endpoints and parameters that are outlined in our documentation.

Hybrid Approach

With the hybrid method, certain task-level parameters are established on a project-by-task basis and others at the task-level. This could be helpful in utilizing project versioning while providing flexibility for some criteria that might vary from task to task.

Project-level Parameter Features

Versioning

Refonte.Ai has an index of previous project iterations that may be queried. It is easy to see what was in a previous version of a certain project. The `project_param_version`` column on every task will indicate which version of the project parameters was applied to that job. You can designate which version of the project parameters should be utilized when creating tasks.

Separation of Concerns

It's advised to divide the task submission pipeline and taxonomy management as you become more experienced. The task submission should be somewhat "dumb" in that it only needs to know that it needs to add a new attachment to a project for labeling, and that the project will determine how this task is labeled. It doesn't need to know every detail about how a task should be labeled.

The project params can then be updated independently of the task submission process.

Default Values

At the project level When tasks are created within that project, parameters are immediately transferred and integrated with the task parameters. You can then select default values that you wish to make sure are set for all tasks submitted to the project, regardless of how they are presented. The instructions could be a typical use-case, but you can choose any default you'd like.

How do I start using Project-level Parameters?

At the Project Level:

We will be leveraging the Update Parameters endpoint to set and create project parameter versions.

When Retrieving or Listing projects, information about the project parameters will be available in the `paramHistory` field.

At the Task Level:

The most recent version of the project parameters will be automatically used by tasks. The task-level parameters are automatically combined with the project-level parameters. With the exception of the instruction field, which functions by concatenating the task level and project level instructions together, parameters supplied at the task level do override any default settings that the project would inherit.

You are able to specify previous versions of the project to use when creating tasks by specifying a `project_param_version`.

Examples

Check out some of our code examples of using this workflow to find out what it all looks like.

Batches Object Overview

`Refonte.Ai` Enterprise Workflow:

  • Batch is created (status = in_progress)
  • Tasks are added to the batch by specifying the batch field on the task to be the name of your batch
  • The final task is completed, the batch is completed (status completed) and the callback is fired with the example to the right.

`Refonte.Ai` Rapid Workflow:

When you use Refonte.Ai Rapid, batches serve an additional function and become closely associated with the task delivery mechanism.

Production batches and calibration batches are the two different categories of batches. When using "normal" batches, calibration batches assist make sure your project is ready for human intervention on your production data.

  • Batch is created (status = staging)
  • Tasks are added to the batch by specifying the batch field on the task to be the name of your batch
  • Batches need to be finalized (status = in_progress)
  • The final task is completed, the batch is completed (status completed) and the callback is fired with the example to the right.

`Refonte.Ai` Studio Workflow:

  • Batch is created (status = in_progress)
  • Tasks are added to the batch by specifying the batch field on the task to be the name of your batch
  • Batches need to be finalized (status = in_progress)
  • Once a batch is finalized, tasks will be submitted to your team workers to begin labeling. No tasks may be added to a batch once it has been finalized.
  • The final task is completed, the batch is completed (status completed) and the callback is fired with the example to the right.

Example Batch Object

{
  "project": "kitten_labeling",
  "name": "kitten_labeling_2020-07",
  "status": "staging",
  "callback": "https://example.com/callback",
  "created_at": "2020-07-01T09:09:10.108Z"
}

Tasks Object Overview

A task is a discrete piece of labor that a Tasker needs to do. The task and the data that needs to be tagged are mapped one to one. For instance, there would be a single job for every image, video, or lidar series that required labeling.

When you make an API request with a set of task parameters to the desired endpoint, you indicate how the labeling for a certain job should be done.

Tasks have a type such as "Image Annotation", "Video Annotation", "Lidar Segmentation", or "Document Transcription".

Learn more about our key concepts and workflows in our "https://refonte-ai.com/docs/key-conceptsale 101" guides.

Example Task Object

{
  "task_id": "576ba74eec471ff9b01557cc",
  "created_at": "2016-06-23T09:09:34.752Z",
  "updated_at": "2016-06-23T09:10:02.798Z",
  "completed_at": "2016-06-23T09:10:02.798Z",  
  "type": "categorization",
  "status": "completed",
  "instruction": "Would you say this item is big or small?",
  "params": {
    "attachment_type": "text",
    "attachment": "car",
    "categories": [
      "big",
      "small"
    ]
  },
  "callback_url": "http://www.example.com/callback",
  "callback_completed": true,
  "response": {
    "category": "big"
  },
  "metadata": {},
  "audits": [
    {
      "audited_by": "david@company.com",
      "audited_at": "2016-06-24T15:32:03.585Z",
      "audit_time_secs": 120,
      "audit_result": "accepted",
      "audit_source": "customer"
    },
    {
      "audited_by": "auditor@refone-ai.com",
      "audited_at": "2016-06-23T10:01:02.352Z",
      "audit_time_secs": 511,
      "audit_result": "fixed",
      "audit_source": "refonte"
    }
  ],
  "tags": ["experiment_1", "owner:david"],
  "unique_id": "product_experiment_dg3d9x83"
}

Property

Type

Description

task_id

string

The task_id is the unique identifier for the task.

type

string

The type of the task, for example, imageannotation or lidarsegmentation

instructions

object

A markdown-enabled string explaining the instructions for the task. You can use markdown to show example images, give structure to your instructions, and more. HTML tags are unsupported.

params

object

An object with the parameters of the task based on the type. For `imageannotation` type tasks, for example, this will include `attachment`, and `geometries`.

response

object

An object corresponding to the response once the task is completed. Each task type has its response format documented.

status

string

The status of the task, one of pending, completed, canceled or error.

created_at

timestamp

A string of the UTC timestamp for when the task was created.

updated_at

timestamp

A string of the UTC timestamp for when the task was last updated. If a task is completed, this timestamp is usually the same as the completed_at timestamp. However, sometimes a task can be redone, and updated after its completion.

completed_at

timestamp

A string of the UTC timestamp for when the task was completed. This will only be filled in after it is completed.

callback_completed

boolean

A boolean stating whether or not the callback succeeded. If the callback returns with a 2xx status code, the value will be true. If the callback fails to return a 2xx status code through all retries, then the value will be false.

customer_review_status

string

The status of the QA'd task, one of pending, accepted, fixed or rejected.

customer_review_status

string

The status of the QA'd task, one of pending, accepted, fixed or rejected.

times_redone

number

A counter for the number of times the task has been redone. It will be present if the task has been redone at least once.

metadata

object, default

A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. See Task Metadata for more information.

project_param_version

number | null

Corresponds to the project version a task was created with, or null if no project version exists.

audits

array, default

An array of audit records.

unique_id

string, optional

A arbitrary ID that you can assign to a task and then query for later. This ID must be unique across all projects under your account, otherwise the task submission will be rejected. See Avoiding Duplicate Tasks for more details.

tags

array, optional

Arbitrary labels that you can assign to a task. At most 5 tags are allowed per task. You can query tasks with specific tags through the task retrieval API.

Annotation Object Overview

Annotation Attributes

Having additional human-evaluated metadata over each annotation for a particular task—like calculating the degree of occlusion of every car in an image—is often beneficial.

We enable annotation_attributes, an object that represents extra attributes you would like to record for each picture annotation, in order to accomplish this.

If no type is supplied, the element will default to a category type attribute. You can define the type of the attribute using its type property. For more information, see the Annotation Attribute Types listed below.

An object whose key-value pairs collectively indicate the properties of each annotation that you want to record is the format for annotation_attributes. The kind of attribute determines how the schema varies slightly.

Attribute Type

Description

Categorical

Multiple choice attribute, with an optional ability to enable selecting multiple options simultaneously.

Numerical

Integer input

Angle

Input to select a value between 0 and 360 with a visual interface supporting angles.

Text

Input for free text

X/Y Offset

Input to select width or height within a box annotation

Linked

Input to link one annotation to another annotation

Request Format

To create a task with attributes, simply add the annotation_attributes parameter to your task creation request using the format described above.

Example task payload for an ¨annotation¨ task

{
  "callback_url": "http://www.example.com/callback",
  "instruction": "Draw boxes around the vehicles in the image.",
  "attachment_type": "image",
  "attachment": "http://i.imgur.com/v4cBreD.jpg",
  "geometries": {
    "box": {
      "objects_to_annotate": ["car","pedestrian"],
    }
  }
  "annotation_attributes": {
    "parked": {
      "description": "Is the car currently parked?",
      "choices": [
        "Yes",
        "No"
      ]
    },
    "heading": {
      "description": "Which direction is the car heading",
      "choices": [
        "left",
        "right",
        "back",
        "front"
      ],
      "conditions": {
        "label_condition": {
          "label": "car"
        },
        "attribute_conditions": [
          {
            "parked": "No"
          }
        ]
      }
    }
  }
}

Updated about 2 months ago