πŸ™‹ Requesting data with asapiΒΆ

OverviewΒΆ

asapi is an asynchronous control mechanism for the ricloud API. It enables users to:

  • Discover services, actions and endpoints they are authorized to use
  • Register, deregister and learn about accounts
  • Submit asynchronous tasks
  • Query the status of tasks

Once completed, data from tasks can be obtained from either the aschannel or the asstore, depending on the client’s use-case.

Important

The ricloud overview goes into detail on whether a client should be using asmaster or asapi. It is important to choose the right service for one’s use-case.

In particular, users who require regular access to cloud account data should probably not directly use asapi. Instead, asmaster can safely manage regular refreshes of this data.

Note

The sample client implementations handle much of asapi‘s functionality without the developer needing to get involved. The examples below are shown for developers who wish to dig into the fundamentals of the API, or to build their own clients.

Service, action and endpoint discoveryΒΆ

/account/ΒΆ

Using the /account/ endpoint, clients can obtain information on:

  • Available services
  • Available actions and their parameters and permissions
  • Endpoints available for task submission and retrieval

To do so, a discovery request is made:

$ curl \
    -X GET \
    -H 'Authorization: Token <TOKEN>' \
    https://asapi.reincubate.com/account/ -D -

This will return a set of JSON describing the available services, actions and endpoints. The example below is abbreviated:

{ "services": [{
    "name": "iCloud",
    "actions": [{
      "description": "",
      "parameters": [{
        "type": "string",
        "description": "",
        "optional": false,
        "name": "Device",
        "slug": "device"
      }, {
        "type": "date",
        "description": "",
        "optional": true,
        "name": "Since",
        "slug": "since"
      }],
      "name": "Fetch Data",
      "execution": "Asynchronous",
      "slug": "fetch-data",
      "permissions": {
        "data": ["sms"]
      }
    }],
    "slug": "icloud"
  }],
  "task_submission_endpoint": {
    "host": "asapi.reincubate.com",
    "protocol": "https",
    "uri": "/submit-task/"
  },
  "stream_endpoints": [{
    "host": "aschannel.reincubate.com",
    "protocol": "https",
    "uri": "/stream/"
  }]
}

In this case, the client would have access to the icloud service and the fetch-data action. They would need to submit device and data parameters when using it, and optionally also since.

This last data parameter must be in list format; this parameter can contain any values listed in the permissions > data of the fetch-data section.

The task_submission_endpoint is defined in this response, and it informs clients where actions may be submitted for execution.

The stream_endpoint in the example above is defined as aschannel.reincubate.com, which means that aschannel is configured as the results endpoint. Alternatively, this could direct clients to asstore, depending on the configuration of their token.

Note

The results endpoint to be used will be listed when performing the discovery request. This is defined per-token: clients cannot switch between aschannel and asstore using a single token.

Account registration and deregistrationΒΆ

register-accountΒΆ

In order to submit actions, we need to inform asapi of the accounts that may have actions performed against them. To do so, we will use the /register-account/ endpoint:

$ curl \
    -X POST \
    -H 'Authorization: Token <TOKEN>' \
    -d "service=<SERVICE>" \
    -d "username=<USERNAME>" \
    https://asapi.reincubate.com/register-account/ -D -

In this request, <SERVICE> is service which owns the specific action, whereas <USERNAME> represents for the account which will be used in the actions.

Note

For more information, please check the service list or the enabled services in the service, action and endpoint discovery method.

deregister-accountΒΆ

If an account is not to be used any further, it can be deregistered. To deregister an account, a similar request can be sent to the /deregister-account/ endpoint:

$ curl \
    -X POST \
    -H 'Authorization: Token <TOKEN>' \
    -d "service=<SERVICE>" \
    -d "username=<USERNAME>" \
    https://asapi.reincubate.com/deregister-account/ -D -

Both methods, on success, will return a 200 response with

{ "success": true
}

Possible errors include:

  • unique_together: “Account with this Service, Client and Username already exists”

Warning

Deregistered or unregistered (ie. not registered) accounts are not valid for use against any action or any service. Accounts must be registered in order to use them.

Task submissionΒΆ

[TASK-SUBMISSION]ΒΆ

Asynchronous tasks can be submitted using the endpoint specified in task_submission_endpoint.

Note

Every action available for public use has its services.actions.execution value set to Asynchronous. Synchronous task execution is not available for public use. Consequently, clients need only implement support for the Asynchronous keyword.

To submit a task, the following command is used:

$ curl \
    -X POST \
    -H "Authorization: Token <TOKEN>" \
    -d "service=<SERVICE>" \
    -d "action=<ACTION>" \
    -d "account=<ACCOUNT>" \
    -d "param1=<VALUE1>" \
    -d "param2=<VALUE2>" \
    <TASK_SUBMISSION_ENDPOINT> -D -

On that request:

  • <SERVICE> and <ACTION> represent the target service and action
  • <ACCOUNT> refers to a previously registered account
  • param1, param2 and subsequent parameters form the specific payload of the task, requirements for which are described by the service, action and endpoint discovery method

Note

To check all available services, endpoints and their required payloads, please check the service list.

Note

Clients may check on the status of tasks at any time. Please see Querying a task status. for more details.

On success, the request will return a 200 with:

{ "task_id": "<TASK_ID>",
  "retrieval_protocol": "<RETRIEVAL_PROTOCOL>",
  "stream": "<STREAM>",
  "success": true
}

In these responses:

  • <TASK_ID> provides an ID for the task which can be used to query its status and to retrieve the task’s result, if using the asstore
  • <RETRIEVAL_PROTOCOL> indicates the method used to retrieve the task’s results and can take the value of aschannel or asstore
  • <STREAM> is stream ID for use in retrieving the task’s results, if using the aschannel

If the request was malformed, the “success” property will be false and the response will include an “errors” section with feedback. For example:

{ "errors": {
    "service": [
      ["[u'Invalid service name']", "invalid_service"]
    ]
  },
  "success": false
}

Possible errors include:

  • invalid_service: ‘Invalid service name’
  • invalid_parameter_value: ‘Value not permitted for parameter’
  • no_such_account: ‘No such account’
  • invalid: ‘Invalid value’
  • limit_exceeded: ‘Too many task IDs submitted’
  • invalid_action: ‘Invalid action’
  • missing-param: ‘Parameter is missing’

Querying task statusesΒΆ

/task-status/ΒΆ

For each task submitted to the asapi endpoint, a task_id is provided. The client can periodically submit this task_id to the asapi endpoint in order to discover a task’s status. Its status can be queried with a POST request on the /task-status/ endpoint. Clients may request the status of up to 10,000 tasks simultaneously.

Note

If the task is listed as STATE_PUBLISH_AVAILABLE or STATE_PUBLISH_COMPLETE then the result is available for download from asstore. The updated task status will include a retrieval_endpoint URL from which an authenticated client may begin downloading the results over HTTPS. The task result may be consumed only once, after which it will be deleted.

$ curl \
    -X POST \
    -H "Authorization: Token <TOKEN>" \
    -d "task_ids=<TASK_ID_1>,<TASK_ID_2>,<TASK_ID_3>,<TASK_ID_4>,<TASK_ID_5>,<TASK_ID_6>,<TASK_ID_7>" \
    https://asapi.reincubate.com/task-status/

Given at least one correct <TASK_ID>, the response will be a 200 in this format:

{ "<TASK_ID_1>": {
    "status": "<TASK_STATUS>",
    "retrieval_protocol": "<RETRIEVAL_PROTOCOL>",
    "result_retrieved": "<RETRIEVAL_STATUS>",
    "success": "<TASK_SUCCESS>"
  },
  "success": "<CHECK_SUCCESS>"
}

If <TASK_SUCCESS> is “True”, then the results are immediately available at the appropriate endpoint.

The status property can take the following values:

  • Pending
  • In progress
  • Work complete
  • Publish ongoing, with result available
  • Publish complete (in this case, the success value be “true”)
  • Publish failed (in this case, the success value be “false”)
  • Task did not complete successfully. (in this case, the success value be “false”)

<RETRIEVAL_PROTOCOL> indicates the method used to retrieve the task results and can take the value of aschannel or asstore depending on the token.

<RETRIEVAL_STATUS> is a boolean, indicating whether the task’s results have already been consumed by a client. If <RETRIEVAL_STATUS> is “True”, then the results have already been consumed and are no longer available.

Note

When using the aschannel endpoint, if <TASK_STATUS> is “Publish complete” or “Publish ongoing, with result available”, and the client was not connected to the aschannel stream, all or part of the results will have been discarded.

See streaming data with aschannel for details.

The <CHECK_SUCCESS> parameter located at the root of the JSON tree refers to the success of status checking itself. If this is “false”, the response will contain an additional “errors” entry.

Possible errors include:

  • invalid: “Invalid task_id value: <TASK_ID>”
  • limit_exceeded: “Too many task ids submitted, limit is

Warning

Do not conflate TASK_SUCCESS with CHECK_SUCCESS!

An example of output is shown below:

{
  "<TASK_ID_1>": {
    "success": false,
    "error": "No task with this ID was found."
  },
  "<TASK_ID_2>": {
    "status": "Publish complete",
    "retrieval_protocol": "aschannel",
    "result_retrieved": false,
    "success": true
  },
  "<TASK_ID_3>": {
    "status": "Publish complete",
    "retrieval_protocol": "asstore",
    "retrieval_endpoint": "https://asstore.reincubate.com/fetch/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/",
    "result_retrieved": true,
    "success": true
  },
  "<TASK_ID_4>": {
    "status": "Publish failed",
    "retrieval_endpoint": "https://asstore.reincubate.com/fetch/1111111-2222-3333-4444-555555555555/",
    "success": false,
    "error": "Task did not complete successfully.",
    "retrieval_protocol": "asstore",
    "result_retrieved": false
  },
  "<TASK_ID_5>": {
    "status": "Pending",
    "result_retrieved": false,
    "success": false
  },
  "<TASK_ID_6>": {
    "status": "In progress",
    "result_retrieved": false,
    "success": false
  },
  "<TASK_ID_7>": {
    "status": "Work complete",
    "result_retrieved": false,
    "success": false
  },
  "success": true,
}