Solute

Back to index

Syndication Exports v3

The solute API gives partners access to the major part of data for navigation and representation of billiger.de's content. It is accessed via an HTTPS connection using Basic Authentication (RFC 2617).

Export Description

The export feed structure proposes two separate export types. These are respectively base (full) and delta exports. The base exports are the reference point and are genereted regularly. The delta exports are generated every 20 minutes.

The export files are available via https for download with this api (see section Exports).

File and Export Types

The export files are files in the format returned by the API. Each file has got its own data structure in which related info is stored.

If the format is json, each line of the file contains a valid json-object. Every file returned has utf-8 encoding. If the format is csv, the file starts with a header row with the names of the columns. The columns are seperated with semicolon (;).

There are following types of export files. Some are delivered in as base and also as delta.

Detailed explanation for all files:

In addition to those features, a cleaning process regularly deletes old files. This process deletes all export files (both delta feeds and base feeds) except for the newer files (~last 14 days).

Column Names & Order

The content of the exports are configured individually.

How To Use

  1. Use the exports endpoint of the api to get a list of available files.
  2. Download the file using export-files endpoint of the api to download the export.
  3. Execute a full import of the recent base export
  4. Bind the shops and offers data together.
  5. Watch out for the next offers delta feed and shops feed.
  6. Evaluate the operations and time stamps in the delta feed (offers).
  7. If the operation is “delete”, exclude the data entity from your system. If the operation is “update”, update all of the data related to the given feed info. There is no indication which part of the data has changed. If the operation is “add”, add the new data into your system. Somtimes it is possible that a "update" never had a "insert" row.
  8. Check if the shops are still online by using the shops base feed. There is no indication in the offers delta feed if the offer online or offline is. It is because an offer can't go offline independently from its shop or from the product that the offer belongs to. Exclude all offers whose shop is not listed in the current shops base feed or is_online isn't True.
  9. Do the same procedure until the next Complete Base Feed Generation comes up. This work-flow is suggested but not mandatory. Every partner can build their own procedure by playing around the structure.

Every time a new base exists, you have to import this new base. With bugs and other circumstances there are possiblities that the consumed deltas aren't resulting in the next base. So you have to consume the newest base if we generate it.

Get the next export to apply

We offer the endpoint /exports/next that will help our customers keeping their data up to date. The endpoint needs to know which data-source the customer is interested in (shopsv3, offersv3 or clicklog_export) and uses a token to obtain the next export to handle.

Usage

Initial base

The request for an initial base export is done by requesting the /exports/next endpoint without a token.

If an initial base is available the export-info of the base to download and a new token are returned. If no initial base is available the status-code of the response is 204. No export-info and no token are returned in this case. This has to be repeated (at around every 5 minutes) until the status-code of the response is 200. In this case the export-info of the initial base export and a token are returned.

Each id-item inside export_file_ids of export-info is used to download the export using the /export-files/{export_file_id} endpoint. The token is stored by the customer for future requests to the /exports/next endpoint.

Get next export

The current token is used to request the /exports/next endpoint for newer exports.

If there is no newer export the status-code of the response is 204. In this case request /exports/next again with the token until the status-code of the response is 200 and a new export-info and a new token are returned (use an interval of around 5 minutes for this).

Each id-item inside export_file_ids of export-info is used to download the export-file using the /export-files/{export_file_id} endpoint. The new token should be stored by the customer. It is required for future requests of the /exports/next endpoint.

Whenever you want to search for newer exports use the previously collected token and make a request to the /exports/next endpoint. If status-code of the response is 200 a newer export-file is available and a new export-info and a new token are returned. Call the /export-files/{export_file_id} endpoint for each id inside the export_file_ids of export-info to download the export-file(s). Now update the stored token with the new one for next request.

FAQ

How can we download the clicklog-reports via API v3?

We prepared two step-by-step instructions on how to do this in english and german.

Can we get the export file as JSON-Response?

At the moment we only support the download of a file. The file is compressed linewise-json. Every line in the file is valid json. We have millions of offers if we return it in a single response in one json-object this object would be multiple gigabytes big in some cases. The download of the offers would be more complicated if we add pagination or other structure to get the data in chunks. If you wish to have a endpoint which returns chunks with valid json each, please contact your syndication contact for a feature request.

Click-Statistics

Click-Statistics are available as data_source equal clicklog_export. To be compatible to the old version the format is at the moment csv. The field "format" shows the current format, if we change the format from csv to json we will also change the value of this field.

Shop traffic disable

The field "is_online" in shopsv3 is a bool. If the bool is false it is not allowed to send traffic to this shop. If a shop is missing in the shop export it is not allowed to send traffic to this shop.

Changelog

Date Version Message Developer
2021-01-13 3.7 Add new endpoint /exports/next sika
2020-11-30 3.6 Announcement: Deprecation for field price returned by GET /offers/{ids}. Deprecated: Thu, 03 Dec 2020. Sunset: Wed, 01 Feb 2021. sika
2020-09-07 3.5 Breaking change: remove GET /offers (keeping GET /offers/{ids}) jol
2020-08-27 3.4 Breaking change: rename endpoint /search_results to /search-results jol
2020-06-03 3.2 Remove "gets_export" permission. Every user with access to the api can use every endpoint. blb
2020-02-17 3.1 We changed the "time" field format for offers from the "offersv3" data source to fully comply with ISO 8601. Instead of a space character, an upper-case "T" is now used as the separator between date and time. blb