Signals Notebook: Search API guidelines

This article contains a set of general guidelines for effective use of Signals Notebook Search API.

It's highly recommended to first read our Developer's Guide for core concepts on our REST API:

Revvity Signals Developer Guide - REST API

There you can find sample search queries that we will not cover on this article.

When building your search request its important to understand the search parameters, here is a breakdown of the most essential ones:

page[limit]

Signals Notebook Search API implements pagination, this ensures efficient data consumption. The page[limit] parameter specifies how many items are returned for your request, you can return up to 1000 items in a single request.

page[offset] 

This parameter allows you to move across your search results. If your request returns more than the page[limit] you specified, in order to return the following set of results, you will need to set this value to the first item after your last result set. You can paginate over a maximum of 5000 results. Refer to guidelines below for how to approach larger result sets.

sort

This is a critical parameter, you must always specify a sort order. This ensures that the results are ordered as you traverse result pages with page[offset]. Failing to specify a sort order will make the result set appear to have a random order and paginating will yield unexpected results.

stopAfterItems

Stops the query after 'stop-after-items' number of rows have been examined. You can currently set this to up to 1000000000 to avoid your query from reaching this limit.

source

This is by default set to SN which will return results from Signals Notebook entities , you can also specify CONNECTED (ELN Archive) , IVT (Inventory), CHEMICALS (Chemical Drawings).

These parameters can be specified in your request URL (URL encoded):

*The source parameter can only be specified in the request URL

/entities/search?page%5Boffset%5D=0&page%5Blimit%5D=100&source=SN&sort=modifiedAt&stopAfterItems=1000000

Or as part of your request body in an "options" object:

"options": {		
		"offset": 0,
		"limit": 100,
		"stop-after-items": 1000000000,
		"sort": {
			"modifiedAt": "desc"
		}
		
	} 

After executing your search, its important to review the search results metadata to understand your result set, take the following example:

"meta": {
        "took-ms": 31,
        "query-timed-out": false,
        "query-reached-limit": false,
        "count": 100,
        "total": 486
    } 

From the above results metadata, we can tell the following:

• The query did not reach the value specified in stopAfterItems ("query-reached-limit": false)
• our limit is set to 100 items per page, this means our "data" array should have 100 items.
• the total result set is 486 items

For the purpose of this example the limit is set to 100, the same logic can be applied for up to 1000 results per request.

Considering offset is a zero-based index, the first 100 results are offset 0-99.

In order for us to read the entire 486 results, we'll need to now set our offset to 100 and repeat the search request.

Continue the above until no results are returned. You can evaluate this in two ways:

• count = 0
• data array has zero items

The maximum offset you can set is 5000, if your results are more than 5000, there are a couple of considerations to make:

First, evaluate if your search criteria is as efficient as possible, you may be returning unnecessary hits which are saturating your result set. Consider filtering by entity type, a creation or modification date range, owner, entity metadata fields etc.

If a 5000+ result set is expected, then you will need to make sure you include a search criteria like createdAt or modifiedAt. For example:

{
	"$and": [
		{
			"$range": {
				"field": "createdAt",
				"as": "date",
				"from": "2023-06-01T00:00:00Z",
				"to": "2024-08-01T00:00:00Z"
			}
		}
	]
} 

Assuming a search query containing the above filter returns 7000 hits, you will need to consume the first 5000 as discussed previously, once you consume the last item:

• capture that items createdAt value
• modify your query range "from" to one second after that value
• set your offset back to zero

This new search criteria will return the remaining 2000 items, starting from offset 0, allowing you to consume the entire result set from the original query.