Syndication Exports v3
The API enables partners to build their own navigation and representation of solute's price comparison data. It is accessed via an HTTPS connection using Basic Authentication (RFC 2617).
Export Description
The export feed structure is made up of base and delta exports. The delta exports are generated every 20 minutes and include only changes to the previous data state. The base exports are generated in much larger intervals including all data and represent the new reference point for the following changes.
The export files are available for download via HTTPS with this API (see section Exports).
File and Export Types
The export files are files in the format returned by the API. Each file has got its own data structure in which related info is stored.
If the format is JSON, each line of the file contains a valid JSON-object. Every returned file is UTF-8 encoded. If the format is CSV, the file starts with a header row with the names of the columns. The columns are semicolon-separated.
There are following types of export files. Some are delivered in as base and also as delta.
- offersv3: base and delta (JSON)
- shopsv3: base (JSON)
- clicklog_export: daily-base (CSV)
Detailed explanation for all files:
- An "Offers Base Feed" includes all offers.
- An "Offers Delta Feed" includes only offers which were deleted, added or updated after the previous last base/delta.
- A "Shops Base Feed" includes all shops. You are only allowed to deliver traffic to shops which are included in this feed with is_online = True.
- A "clicklog_export Base Feed" includes all clicks you generated on a specified day.
In addition to those features, a cleaning process regularly deletes old files. This process deletes all export files (both delta feeds and base feeds) except for the newer files (~last 14 days).
Column Names & Order
The content of the exports are configured individually.
How To Use
- Use the exports endpoint of the API to get a list of available files.
- Download the file using export-files endpoint of the API to download the export.
- Execute a full import of the recent base export.
- Bind the shops and offers data together.
- Watch out for the next offers delta feed and shops feed.
- Evaluate the operations and time stamps in the delta feed (offers).
- If the operation is “delete”, exclude the data entity from your system. If the operation is “update”, update all of the data related to the given feed info. There is no indication which part of the data has changed. If the operation is “add”, insert the new data into your system. Sometimes it is possible that a "update" never had a "insert" row.
- Check if the shops are still online by using the shops base feed. There is no indication in the offers delta feed if the offer is online or offline. This is because an offer can't go offline independently from its shop or from the product that the offer belongs to. Exclude all offers whose shop is not listed in the current shops base feed or where is_online isn't True.
- Do the same procedure until the next complete base feed generation comes up. This workflow is suggested but not mandatory.
Every time a new base gets created, you have to import this new base. Based on different circumstances, there is the possibility that the consumed deltas aren't resulting in the next base - so consuming the bases is mandatory.
Get the next export to apply
We provide the
endpoint /exports/next
to help our customers keep their data up to date.
The endpoint needs to know which data-source the customer is interested in
(shopsv3
, offersv3
or clicklog_export
) and uses a token to obtain
the next export to handle.
Usage
Initial base
The request for an initial base export is done by requesting the
/exports/next
endpoint without a token.
If an initial base is available, the export info of the base to download and
a new token are returned.
If no initial base is available the status code of the response is 204
.
No export info and no token are returned in this case.
This should be repeated (at an interval of around 5 minutes) until the status code of
the response is 200
.
In this case, the export info of the initial base export and a token are returned.
Each id-item inside export_file_ids
of export-info
is used to download
the export using the /export-files/{export_file_id}
endpoint.
The token
should be stored by the customer for future requests to the
/exports/next
endpoint.
Get next export
The current token
is used to request the /exports/next
endpoint for newer
exports.
If there is no newer export the status-code of the response is 204
.
In this case request /exports/next
again with the token until the
status code of the response is 200
and a new export-info
and a new token
are returned (at an interval of around 5 minutes).
Each id-item inside export_file_ids
of export-info
is used to download
the export-file using the /export-files/{export_file_id}
endpoint.
The new token
should be stored by the customer.
It is required for future requests of the /exports/next
endpoint.
Whenever you want to search for newer exports use the previously collected
token
and make a request to the /exports/next
endpoint.
If status-code of the response is 200
a newer export-file is available and
a new export-info
and a new token
are returned.
Call the /export-files/{export_file_id}
endpoint for each id inside the
export_file_ids
of export-info
to download the export-file(s).
Now update the stored token
with the new one for next request.