Paging through historical Mentions

Another common integration use case is to request a large Mentions set within a date range and then iterate through them all. You may want to use this approach if you want to download a substantial dataset from your Query into your own integration.

In this tutorial, we'll take the following approach:

We will request for a batch of Mentions across a specified date range.
We will then use the cursor filter functionality to access additional results.

In this tutorial we will assume that the application has already logged in and has a token.
We also recommend that you read the section in our documentation about Mentions, in addition to this tutorial.

Requesting Mentions for a given date range

In a similar manner to the other tutorial on Polling for new Mentions, we'll begin by requesting the newest page of Mentions within a date range.

We'll use the following parameters to achieve this:

Parameter	Value	Reasoning
`pageSize`	100	This returns 100 Mentions per request. You can change this to a value between 1 and 5000 that suits you. Getting more than 5000 at a time is not possible.
`orderBy`	`date`	This orders the Mentions by their `date` (any valid `orderBy` should work fine here)
`orderDirection`	`asc`	This changes the sort direction to ascending.

Here is the request as a cURL command:

curl -X GET 'https://api.brandwatch.com/projects/1998232810/data/mentions?queryId=1999264284&startDate=2018-10-26&endDate=2018-11-09&pageSize=100&orderBy=date&orderDirection=asc'

This will return a list of Mentions. The following example response has had the Mentions removed for brevity.

{
  "results":[...],
  "resultsPage":0,
  "resultsPageSize":100,
  "resultsTotal":234,
  "nextCursor":"AQ=AA=AWbKauTY=Mlz58Vk=dypaHA",
  "startDate":"2018-10-26T00:00:00.000+0000",
  "endDate":"2018-11-09T00:00:00.000+0000",
  "maximumIdInResult":217365452890,
  "maximumId":217365452890
}

We can see from the result that there were a total of 234 Mentions returned for that date range and there is also a nextCursor field containing a cursor value. We can use this cursor value to retrieve Mentions beyond the first 100 results.

Iterating through all available Mentions

After the first response, the client should store the cursor value returned (identified by nextCursor: "AQ=AA=AWbKauTY=Mlz58Vk=dypaHA"). You can think of the cursor like a bookmark, that we can use to remember the previous position in the results.

With this cursor we can retrieve additional Mentions using the below parameter:

Parameter	Value	Reasoning
`cursor`	AQ=AA=AWbKauTY=Mlz58Vk=dypaHA	Specifies the "cursor" position to start returning results from

With this parameter we can now retrieve the next 100 Mentions:

curl -X GET 'https://api.brandwatch.com/projects/1998232810/data/mentions?queryId=1999264284&startDate=2018-10-26&endDate=2018-11-09&pageSize=100&orderBy=date&orderDirection=asc&cursor=AQ%3DAA%3DAWbKauTY%3DMlz58Vk%3DdypaHA'

Nb. the cursor has been URL encoded in the request above (AQ%3DAA%3DAWbKauTY%3DMlz58Vk%3DdypaHA).
This will return the next 100 results. The new response will look similar to this example:

{
  "results":[...],
  "resultsPage":1,
  "resultsPageSize":100,
  "resultsTotal":234,
  "prevCursor":"AQ=AQ=AWbKbU4I=Ml2eY8E=dypaHA",
  "nextCursor":"AQ=AA=AWbldT7V=Mpojoo8=dypaHA",
  "startDate":"2018-10-26T00:00:00.000+0000",
  "endDate":"2018-11-09T00:00:00.000+0000",
  "maximumIdInResult":217349261612,
  "maximumId":217349261612
}

We can now repeat this process with the new nextCursor value, until no more results are returned, to retrieve all available Mentions.