Apple iCloud service

Overview

The ricloud API’s icloud service provides functionality for accessing information stored on the iCloud, both in batch form from iOS device backups, and in real-time and near-time from other sources. The API has supported access to the iCloud since its original release.

Note

Note that the sample client implementations handle much of the complexity shown below, without the developer needing to get involved. The curl methods below are shown for developers who wish to dig into the fundamentals of the API, or to build their own native clients.

There are four core operations that a client may need to perform with the iCloud service.

  • Authentication: log-in, perform-2fa-challenge, submit-2fa-challenge
  • Device enumeration: list-devices
  • Requesting feeds: fetch-data
  • Requesting raw files: download-file

Note

Examples commands in this section are given in bash terminal notation, using curl. The <TASK_SUBMISSION_ENDPOINT> value is provided by the asapi using the service, action and endpoint discovery command. It may be https://asapi.reincubate.com/submit-task/.

Working with mock data from the API

In order to assist development of integrations with the API, a mock account populated with a set of rich data is made available. This allows clients to test many of the ricloud API’s functions without needing to work with real user data.

The mock account is named Jonny Appleseed, and is accessible by using the iCloud account ID john.appleseed@reincubate.com and password joshua.

Authentication

Logging in: log-in

The first step to interacting the icloud service is logging in.

Note

After the first log in, it is possible to use the authentication token provided to re-authenticate the account. This avoids the need for clients to store account credentials. For more information, see refreshing an account login using authentication tokens.

For the request, we must specify:

  • The service to interact with: icloud
  • The action to perform: log-in
  • The parameter account, representing the iCloud account ID such as john.appleseed@reincubate.com
  • The action parameters required by the action, which in this case is just password
  • We may also specify optional action parameters listed by the /account/ endpoint
$ curl \
    -X POST \
    -H "Authorization: Token <TOKEN>"
    -d "service=icloud" \
    -d "action=log-in" \
    -d "account=<ACCOUNT>" \
    -d "password=<PASSWORD>" \
    <TASK_SUBMISSION_ENDPOINT>

The asapi endpoint will reply, in JSON format:

{ "retrieval_protocol": "aschannel",
  "stream": "<STREAM>",
  "success": true,
  "task_id": "<TASK_ID_1>"
}

If you are connected to the correct stream_endpoint on the aschannel endpoint, you will see a result come in matching the <TASK_ID_1> you received from the asapi endpoint.

{ "message": "Log-in successful",
  "auth_token": "ABCDEFGABCDEFGABCDEFGABCDEFGABCDEFGABCDEFGABCDEFGABCDEFG",
  "success": true
}

The results will include a success parameter which will be true if the login task succeeded, along with a message and an Apple authentication token for use with other service actions.

Note

Only one login should be performed against an iCloud account at any time, regardless of the number of other service actions performed. A new login should only be performed when the existing iCloud session expires.

Troubleshooting login

The following table shows the error codes that the log-in action can raise.

Response Summary
unable-to-login Credentials are incorrect
task-failed Unknown failure
2fa-required 2FA required
invalid-2fa-code Invalid 2FA code
too-many-2fa-requests 2FA rate-limited
deactivated-account Account deactivated
deactivated-device Device deactivated
client-account-disabled API token disabled
account-locked Account locked
account-credentials-blocked Account locked and protected
terms-of-service-update-client Terms of service not accepted
push-api-timeout Internal timeout

Credentials are incorrect

The credentials specified for the account were incorrect, and authentication is not possible.

Unknown failure

The login task failed and the reason was unknown. In the event of task failure with this code, clients should contact the support team.

2FA required

Logging in to accounts which use two-factor authentication (2FA) or two-step verification (2SV) requires some additional steps. See the “2FA and 2SV” sections below.

Invalid 2FA code

An invalid or otherwise incorrect 2FA code was sent in response to the 2FA challenge.

2FA rate-limited

Too many 2FA requests have been sent recently for the account: it has been rate-limited. Retry after 30 minutes.

Account deactivated

This signifies that the account has been deactivated using the account deactivation management command. It will not be accessible unless a manual activation command is sent.

Device deactivated

This signifies that the device has been deactivated using the device deactivation management command. It will not be accessible unless a manual activation command is sent.

API token disabled

The token being used to access the API has been disabled, please contact the support team.

Account locked

The iCloud account has been locked by Apple. If another attempt is made to access the account in a short period of time, an account-credentials-blocked error will instead be returned.

Account locked and protected

The iCloud account has been locked by Apple and ricloud has already informed the client with account-locked. This message is substituted to indicate that the service is not relaying the response on to Apple.

Terms of service not accepted

The iCloud account is disabled pending acceptance of the latest iCloud terms and conditions. The user can accept these on their device or by logging in to icloud.com.

Internal timeout

If the iCloud session expires while performing other service actions, clients will receive the error code: push-api-timeout and must log in again.

{ "message": "Unable to recover the session. Please re-login and try again",
  "error": "push-api-timeout"
}

2FA and 2SV

After attempting to log in to an iCloud account that is set up with 2SV or 2FA, the normal login will fail and an error message will be returned similar to the one below.

{ "message": "This account has Two Step Verification enabled, please select a device to challenge.",
  "data": {
    "trustedDevices": ["********12", "Renate's iPhone - iPhone 6s"],
  },
  "error": "2fa-required"
}

The data dictionary will contain a list devices, keyed by device_id. These are the devices that are trusted by the iCloud account to perform the next authentication step.

For two-step verification (2SV)

In the case of 2SV, the list of trustedDevices will contain one or more devices. Clients must select a listed device to challenge.

This is done by performing perform-2fa-challenge with challenge set to the chosen device’s key from trustedDevices.

For two-factor authentication (2FA)

In the case of 2FA, the list of trustedDevices will be set to ["Challenge all 2FA devices"], indicating that a challenge must be sent to all iOS devices connected to that account. This is done by performing perform-2fa-challenge with challenge set to the value of ["Challenge all 2FA devices"].

Using perform-2fa-challenge

This method will send a request in realtime to all devices on the account in the case of 2FA, or the chosen device in the case of 2SV.

$ curl \
    -X POST \
    -H "Authorization: Token <TOKEN>"
    -d "service=icloud" \
    -d "action=perform-2fa-challenge" \
    -d "challenge=<TRUSTED_DEVICE>" \
    <TASK_SUBMISSION_ENDPOINT>

<TRUSTED_DEVICE> must be set to a device_id, or to "Challenge all 2FA devices"

asapi will reply as usual:

{ "retrieval_protocol": "aschannel",
  "stream": "<STREAM>",
  "success": true,
  "task_id": "<TASK_ID_X>"
}

The retrieval endpoint will reply with:

{ "message": "Challenge has been submitted.",
  "success": true
}

Once the challenge has been sent to the user’s device, the two-factor code must be relayed back to the API using submit-2fa-challenge. In the request below, <CODE> is the numeric code provided by the end-user, which represents the codes transmitted to the device for 2FA/2SV login.

$ curl \
    -X POST \
    -H "Authorization: Token <TOKEN>"
    -d "service=icloud" \
    -d "action=submit-2fa-challenge" \
    -d "code=<CODE>" \
    <TASK_SUBMISSION_ENDPOINT>

asapi will reply as usual:

{ "retrieval_protocol": "aschannel",
  "stream": "<STREAM>",
  "success": true,
  "task_id": "<TASK_ID_X>"
}

The retrieval endpoint will reply with a normal log-in message:

{ "message": "Log-in successful",
  "auth_token": "ABCDEFGABCDEFGABCDEFGABCDEFGABCDEFGABCDEFGABCDEFGABCDEFG",
  "success": true
}

Refreshing an account login using authentication tokens

ricloud provides authentication tokens for use in refreshing an logins without the need to use full iCloud credentials. Users of the tokenisation mechanism need not persist any user credentials.

Warning

This is the recommended approach for working with all accounts, as it removes the require to persist credentials

Tokens remain valid for a period of months before expiry. Other factors can premature expiry a token, including irregular use. If a token remains unused for several days, it may be expired by Apple.

To do this, clients must use the auth_token field retrieved during login. This must be sent to the refresh-session service action endpoint.

$ curl \
    -X POST \
    -H "Authorization: Token <TOKEN>"
    -d "service=icloud" \
    -d "action=refresh-session" \
    -d "auth_token=<ICLOUD_AUTH_TOKEN>" \
    <TASK_SUBMISSION_ENDPOINT>

With this done, any service action may be performed against the service.

Device enumeration

Enumeration of devices: list-devices

One logged in to an account, clients can enumerate devices and device backups associated with the iCloud account. This is done by performing the list-devices action against the iCloud service.

This is done in the same way as any other service action. Here is an example curl request to this action. The <ACCOUNT> parameter below can be filled with an account ID, such as john.appleseed@reincubate.com.

$ curl \
    -X POST \
    -H "Authorization: Token <TOKEN>"
    -d "service=icloud" \
    -d "action=list-devices" \
    -d "account=<ACCOUNT>" \
    <TASK_SUBMISSION_ENDPOINT>

The asapi endpoint will reply, in JSON format:

{ "retrieval_protocol": "aschannel",
  "stream": "<STREAM>",
  "success": true,
  "task_id": "<TASK_ID_2>"
}

Your result will appear shortly on the retrieval endpoint. Below is an example response:

{ "auth_token": "ABCDEFGABCDEFGABCDEFGABCDEFGABCDEFGABCDEFGABCDEFGABCDEFG",
  "devices": {
    "<DEVICE_ID_1>": {
      "ios_version": "10.0",
      "colour": "1",
      "device_name": "Johnny's iP7 iOS10 Black",
      "latest-backup": "2016-09-16 14:08:13.000000",
      "model": "D101AP",
      "serial": "ABC123AAAAAA",
      "name": "iPhone 7"
    },
    "<DEVICE_ID_2>": {
      "ios_version": "9.1",
      "colour": "#e1e4e3",
      "device_name": "Johnny's iPad",
      "latest-backup": "2016-11-16 16:36:33.000000",
      "model": "J98aAP",
      "serial": "ABC123BBBBBB",
      "name": "iPad Pro"
    },
    "<DEVICE_ID_3>": {
      "ios_version": "9.2.1",
      "colour": "#3b3b3c",
      "device_name": "Johnny's other iPhone",
      "latest-backup": "2016-06-16 16:00:49.000000",
      "model": "N49AP",
      "serial": "DFVDASDDSVAS",
      "name": "iPhone 5c"
    },
    "<DEVICE_ID_4>": {
      "ios_version": "9.0.2",
      "colour": "#e1e4e3",
      "device_name": "Johnny's white iPhone",
      "latest-backup": "2016-02-25 15:37:38.000000",
      "model": "N61AP",
      "serial": "HYTWGFHGHDSF",
      "name": "iPhone 6"
    }
  }
}

This response contains the auth_token for the specified account’s iCloud session, along with a list of devices associated with that account in the devices section. Every device entry contains the following keys:

  • ios_version: The device’s iOS version
  • colour: The device’s colour as a hex code
  • device_name: The device’s name as set by the user
  • latest-backup: The date the device was last backed up to the iCloud
  • model: The device’s model number
  • serial: The device’s serial number
  • name: The device’s product name as advertised by Apple

Requesting feeds

Retrieval of feed data: fetch-data

The fetch-data action is used to access feeds of JSON data from the API. It requires the following parameters:

  • device: The device to retrieve data from or about
  • data: A CSV of the feed modules requested

Optional parameters are:

  • since: The time period from which results should correspond

Below is an example fetch-data action.

$ curl \
    -X POST \
    -H "Authorization: Token <TOKEN>" \
    -d "service=icloud" \
    -d "action=fetch-data" \
    -d "account=<ACCOUNT>" \
    -d "device=<DEVICE_ID>" \
    -d "data=<FEED_MODULES>" \
    <TASK_SUBMISSION_ENDPOINT>

<FEED_MODULES> can be any of the feed modules listed in feed modules.

For example, using the value sms in <FEED_MODULES> in the above command one can access SMS, MMS and iMessages in a backup. Using the value sms,whatsapp_messages,snapchat_messages in <FEED_MODULES> in the command we can access SMS, MMS and iMessages, WhatsApp messages, and Snapchat messages, all in the same JSON result.

The asapi will respond with:

{ "retrieval_protocol": "aschannel",
  "stream": "<STREAM>",
  "success": true,
  "task_id": "<TASK_ID_3>"
}

Shortly after, results will appear on the results endpoint with the same task_id:

{ "sms": [
    { "group_handles": [
        "+441234567890",
        "renate@reincubate.com"
      ],
      "attachments": [],
      "deleted": false,
      "text": "Welcome to Vodafone!",
      "conversation_id": "vodafone",
      "from_me": false,
      "date": "2015-10-28 09:18:17.000000",
      "handle": "vodafone",
      "type": "SMS",
      "id": 6
    }
  ]
}

Other feed modules provide information that can be used to access other forms of content. For example, a fetch-data action using ‘photos’ as <FEED_MODULES> for a given device would look like this:

$ curl \
    -X POST \
    -H "Authorization: Token <TOKEN>" \
    -d "service=icloud" \
    -d "action=fetch-data" \
    -d "account=<ACCOUNT>" \
    -d "device=<DEVICE_ID>" \
    -d "data=photos" \
    <TASK_SUBMISSION_ENDPOINT>

Will cause asapi to respond with:

{ "retrieval_protocol": "aschannel",
  "stream": "<STREAM>",
  "success": true,
  "task_id": "<TASK_ID_4>"
}

The following result will appear on the appropriate endpoint:

{ "photos": [{
    "file_path": "Media/DCIM/100APPLE/IMG_0001.PNG",
    "last_modified": "2016-10-05 10:24:03.000000",
    "file_id": "c8ada38b9acf7368c6347be1c353dc68ed2c7741",
    "filename": "IMG_0001.PNG"
  }, {
    "file_path": "Media/DCIM/100APPLE/IMG_0002.MOV",
    "last_modified": "2016-10-11 09:39:25.000000",
    "file_id": "5c3e35cc01689340a34d13d34a0591e2ed450e63",
    "filename": "IMG_0002.MOV"
  }]
}

JSON feed formats

The JSON feeds are designed to be as simple to parse as possible. The feed will return all of the data types requested within a single response.

Each feed module specified in the module flag will have its own key in the top level JSON dictionary that is returned.

{
    "first_module_name": "Module's data",
    "second_module_name": "Module's data",
    "etc.": "etc."
}

Requesting raw files

Retrieval of raw-files: download-file

The download-file action is available for downloading files or message attachments directly from the iCloud. The parameters needed are the the target <DEVICE_ID> and a <FILE_ID>.

Some of the feed modules including the file_list and photo modules contain references to files or attachments available for download. These contain a <FILE_ID> used to identify and download the file, and often some extra metadata such as filename, file_path, size and type. Please see Information extractors for more details.

The command

$ curl \
    -X POST \
    -H "Authorization: Token <TOKEN>" \
    -d "service=icloud" \
    -d "action=download-file" \
    -d "account=<ACCOUNT>" \
    -d "device=<DEVICE_ID>" \
    -d "file=<FILE_ID>" \
    <TASK_SUBMISSION_ENDPOINT>

asapi will respond with:

{ "retrieval_protocol": "aschannel",
  "stream": "<STREAM>",
  "success": true,
  "task_id": "<TASK_ID_5>"
}

Shortly after the results will appear on the endpoint with the same task_id.

Note

A large file sent via aschannel will be chunked and will have to be reassembled before being used. A code sample for reconstructing large files is available in the aschannel documentation.

Note

file_id values are either files stored directly by the iCloud, or hosted elsewhere in the cloud. In the former case, file_id values are built from SHA-1 hashes of a file’s AppDomain and filename.

Common hash keys associated with apps for direct file access include the following.

  • 3d0d7e5fb2ce288813306e4d4636395e047a3d28 SMS
  • 1b6b187a1b60b9ae8b720c79e2c67f472bab09c0 WhatsApp
  • 1c6a49018bcace96656e4fe8f08d572ce071b92c WhatsApp
  • 7c7fba66680ef796b916b067077cc246adacf01d WhatsApp
  • b39bac0d347adfaf172527f97c3a5fa3df726a3a Viber
  • 8e281be6657d4523710d96341b6f86ba89b56df7 Kik
  • ff1324e6b949111b2fb449ecddb50c89c3699a78 Calls
  • a49bfab36504be1bf563c1d1813b05efd6076717 Calls
  • 2b2b0084a1bc3a5ac8c27afdf14afb42c61a19ca Calls
  • 5a4935c78a5255723f707230a451d79c540d2741 Calls
  • 12b144c0bd44f2b3dffd9186d3f9c05b917cee25 Photos
  • adb8c77534444e97c31ff15924d50f3ed1fbd3b1 Contacts
  • 2041457d5fe04d39d0ab481178355df6781e6858 Appointments
  • 3ecf3efff3a55d6155efce2828579e8a3cd881c1 Browsing history
  • cd89f9e10d3497912bfc92e5dc674ca989cfdd44 Browsing history
  • 2d711a1f5613f5259730b98328a3f7e816698f88 Line

Some messenger applications – such as Facebook Messenger and WeChat – vary the file_id dependent on the conversation. A number of third-party applications also apply additional layers of encoding or encryption to files, and ricloud will decode these on the fly.