Discuss (0)

Print

RSS

Developers » Forecasting API » here

Programmer's guide for the Forecasting API v2

This document contains programmer's insights about the Forecasting API v2. It focuses on Why? questions as opposed to How to? questions. This document should help you to understand how this API is organized and why we did it that way. In particular, this document is not specific of client-side technology used to call our API.

Contrary to competing technologies, Lokad is 100% automated (check our technology page for details). Hence, with Lokad you can bring automation in your company to the next level by matching automatically the demand. Our philosophy is: just let automation take care of low-level realtime business optimization, so that people could actually focus on things that really matters.

Table of Contents [Hide/Show] Programmer's guide for the Forecasting API v2    References    Authentication    Sample web request and response    Time-series as data model    Call flow on web methods       Initialization       Data upload       Wait       Retrieve    Maximal capacity of web requests    Paging through your Lokad account    Flat forecasts    Partner's tracking

Lokad exposes a web API based on SOAP, an XML dialect. This page is giving you a high-level yet technical understanding of our API.

As a starting point, although our API is XML, we suggest not to deal directly with the XML, as most major programming environments feature already build-in libraries for SOAP (hence, you won't have to deal with raw XML). This slightly outdated list of SOAP implementations will give you some further insights. Chances are that your environment already support SOAP.

References

• Latest Technical Documentation on all methods exposed by this API
• Sandbox server for development and testing
  

Authentication

All the requests made to our web services are authenticated to keep your data secure. Lokad provides **two ways** to authenticate your request:

• API keys (preferred way)
• user credentials
  

Historically, Lokad did start by providing an authentication scheme based on username and password, matching the credentials used by users to log on into their account. Yet, this approach is problematic. Indeed, since constraints on password complexity are loose (we don't force our users into 12 chars passwords), Lokad has an auto-locking mechanism to prevent brute force attacks to be applied on API credentials. The downturn of this security policy is that a failing scripts is very likely to try multiple times to login, and likely to cause a consequent lock-out too.

Hence, at present time, we favor the approach of Lokad API Keys that can be generated from your the Users page in your Lokad account. API keys have a sufficient complexity to nullify any brute force attack. As a consequence, there is no more lock-out mechanism, and recovering a failing script is less troublesome for the system administrator.


User is authenticated by the username and password being passed with each request inside an Identity object. When using an API key, the username should defined as auth-with-key@lokad.com (that's a special value that triggers API key authentication).

Sample web request and response

For the sake of clarity, we are providing below a sample web request, along with its associated response.

Request:

<?xml version="1.0" encoding="utf-8"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
  <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <GetAccountInfo xmlns="http://lokad.com/ws_2/">
      <identity Username="auth-with-key@lokad.com" Password="i92ks92dkb2odl20slnkskwspm2b92s"/>
    </GetAccountInfo>
  </s:Body>
</s:Envelope>

Response:

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <soap:Body>
    <GetAccountInfoResponse xmlns="http://lokad.com/ws_2/">
      <GetAccountInfoResult AccountID="d66a5894-77be-43f3-80f8-acdfa147186c" AccountHRID="12000" SerieCount="661" />
    </GetAccountInfoResponse>
  </soap:Body>
</soap:Envelope>

Time-series as data model

You can think of Lokad as a forecasting pipe: historical data is pushed in one end, and forecasts are retrieved in the other end.

Yet, Lokad does not accept any kind of data. Quite the contrary actually, our approach is **narrow and restrictive**. Lokad handles only decorated time-series. A time-series is nothing more than a list of data points.



For example, the Dow Jones illustrated here above is a typical time-series representation.

Then, although we said that our approach was very restrictive, it must be noted that there are tons of typical business situations that could be represented as time-series:

• sales of a product over time.
• inbound call volumes in a call center.
• cash flow (in/out) in a company.
• electricity consumption.
• web traffic workload on a web server.
• ...
  

Basically any measurable phenomenon that change over time can be represented as a time-series (although, it's not necessarily relevant from a business perspective); and for any time-series, Lokad can provide a corresponding forecasts, that is to say the projection of the historical data in the future.

Hence, our Forecasting API is primarily manipulating time-series, as both inputs (historical data) and outputs (forecasts) are represented as time-series.


A time-serie is nothing more than an ordered list of time-value pairs (a time-value being the specific term to refer to an elementary data point); the order being nothing more than the canonical time ordering (earliest dates come first in the list).

Usually, time-series are equispaced, that is to say that points come at successive times spaced at uniform time intervals. Although, it's frequently more convenient to deal with equispaced data, Lokad does not force you to hard-code such regularity constraint in your input data.

For example, in the case of a call center, if you don't have any inbound calls outside business hours, you can skip nightly values altogether (instead of uploading lists of zeroes).

Then, although the input data is not (necessarily) equispace, returned forecasts are always equispaced following the period provided as an configuration setting for the forecasting operations (more details in the following). This behavior facilitates the integration of the Lokad forecasts within your system. We try to be as tolerant as we can for input data, and as simple as we can for output data.

Then, when you upload aggregated data toward Lokad (for example: sales of a given product aggregated weekly), the date chosen within the aggregation interval does not matters. We suggest to use a date at the very beginning of the past interval (ex: for weekly sales of a product, dates would be set on Mondays at 00:00 am). Note that this comes as a convention, not a requirement from the API.


We will see later on that Lokad offer the possibility to decorate time-series with tags and events. Such decoration is particularly helpful to refine the forecasts though additional data. Yet, it's usually not required in the first approach.

Call flow on web methods

Although, our Forecast API includes nearly two dozens of distinct web methods, a typical call flow is very plain and linear. Indeed, as explained previously, Lokad is nothing more than a forecasting pipe: historical data in and forecast out. Hence, the call flow typically reflects this simplicity.

The typical usage of our API comes in 4 stages:

• Initialization: series get created with AddSeries, forecast settings are tuned with {AddTasks}.
• Data Upload: series content get uploaded with UpdateSerieSegments. In particular, our API accommodates incremental data upload.
• Wait: we recommend 1h of delay to ensure quality forecasts.
• Retrieve: forecasts are retrieved through {GetForecasts}.
  

Initialization

The first step is the initialization. The web method AddSeries will be used to setup the series as time-value containers. AddSeries takes series with empty identifiers (since they are new) as an argument and returns an array of assigned identifiers.


Then, once series are setup, the next step is to setup the configuration settings for the forecasts. In the Lokad terminology, the forecast configuration is done through objects named Task (shorthand for forecasting task). Tasks are created through the web method named AddTasks.

For each Task, there are two very important settings, and chances are that you will never need more than those two settings:

• the Period property which represents the aggregation interval to be applied to your data. Lokad supports a wide range of intervals from 1/4h up to yearly aggregations. The period should be chosen within the following values: QuarterHour, HalfHour, Hour, Day, Week, Month, Quarter, Semester or Year.
• the FuturePeriods property which represents the number of forecasts intervals to be forecasted by Lokad. For example, if you choose Period=week and FuturePeriods=4, Lokad will return 4 forecasts for the 4 weeks ahead.
  

Another possibly useful task setting is the PeriodStart (2001-01-01 is to be used by default). More details are available here.

Note that AddTasks takes items with empty task identifiers (since they are new) as an argument and returns an array of assigned identifiers. The process is very similar to time-series initialization in AddSeries. A time-serie can have only one task assigned.

Data upload

The amount of data uploaded toward Lokad isn't usually that large. Indeed, Lokad does not usually deal with raw sales data (or call volume data) but rather with aggregated data. For example, 3 years of weekly sales data for 1000 products represents 150k time-values, which can be easily uploaded for any regular internet connection within a reasonable delay (a few minutes actually).

Yet, Lokad also supports incremental data upload. Indeed, previous data uploads are persisted in your Lokad account. In other words, Lokad offers a server-side representation of your data, that can be updated over time, minimizing the amount of data to be transferred each time.

Both data upload, either one-shot or incremental, are done through the same web method named UpdateSeriesSegments}. The graph below illustrates how the upload is performed.



The method UpdateSeriesSegments takes one or many segments of time-series as input, and push them toward your Lokad account. If time-series already exist, then time-series get automatically overwritten by the new segment being uploaded, as illustrated in the graph below.



Simply put, UpdateSeriesSegments overwrites old data whenever newer data is provided. Note that overwrite only happens if old and new data overlap. If not, the new segment is just appended to the existing time-serie.

Wait

After completing the last call to UpdateSeriesSegments, the client should be put on hold for 1h. In other words, the API expects a delay of 1h before the client resumes its process and starts requesting the forecasts. Implementing this behavior is usually rather straightforward, as it comes with a typical call to sleep(1h) in the client code.

Retrieve

Forecasts are retrieved through the web method named GetForecasts}. This method takes as input the identifiers named TaskID that have been previously retrieved at initialization-time through the call to AddTasks.

Forecasts are returned as time-series virtually identical to the input data format used for UploadSeriesSegments. Yet, Lokad not only returns forecasts, but returns the expected forecast accuracy as well. Every forecasted time-series come with a MAPE (Mean Absolute Percentage Error) reflected in the Error} property of the Forecast element.

Our forecasts aren't magic, and we are well aware that uncertainties lie in values returned by Lokad. As a matter of fact, a significant portion of the Lokad technology is precisely dedicated to measurement of forecast errors (hint: you can't optimize what you don't measure). Thus, we take a great care of accurately estimating the forecast error as well as the forecast itself.

Finally, a slightly subtle aspect of the Lokad forecasts is the following: forecasts start where the data ends. In other words, Lokad has a data-centric approach where forecasts automatically match data provided as input. This behavior is especially useful to benchmark the quality of Lokad against historical situations.

Maximal capacity of web requests

Due to various constraints, our Forecasting API does not handle arbitrarily large web requests. Every web method comes with capacity limits that specify how may objects (ex: time-series or time-values) can be processed in a single call. Those limitations do not represent a scalability barrier for Lokad, it just means that large processes must be split into multiple small web request toward Lokad.

This file lists capacity limitations for each web method. Please, make sure your code is compliant with those limitations, because without some extra logic, it will work on small datasets, typically the ones that you are using for testing, but fails once in production because the datasets will be much larger.

Additionally, please make sure to check additional constraints on object validity (illegal characters, maximum string lengths etc)

Paging through your Lokad account

Although this is usually not a requirement, Lokad also offers the possibility to read the data stored in your Lokad account. This could handy to wipe out the content of your account if identifiers (time-series or tasks) have been lost on the client side.

Yet, the amount of data lying in a Lokad account can be arbitrarily large, and the capacity limits assigned to each web method prevent retrieving everything in a single web request. Hence, to enable the retrieval of large datasets, paging has been implemented for some methods such as:

• GetTasks
• GetSeries
• GetSerieSegments
  

Each time the paging process is similar: You start a page iteration by including an empty cursor in the initial web method call. It will return a result collection that contains:

• flag indicating if more pages are expected
• cursor that will allow to retrieve the next page
  

In order to get the next page you pass the cursor returned by the previous call to the next call. Different web methods might have different cursor definitions, but generally you initialize the first cursor with:

• 00000000-0000-0000-0000-000000000000 for Guid
• 0001-01-01T00:00:00 for DateTime
  

Flat forecasts

It is normal to observe flat (i.e. constant) forecasted time-series if input data are limited. This situation only reflects the fact that the input data does not hold enough information to generate a varying forecasts. Thus don't be surprised, if you get flat time-series out of Lokad on toy datasets, this is the expected behavior.

Then, even with large datasets, you might observe that if you try to forecasts far in the future the forecasted time-series become flat after a while. Again, this situation simply reflects that sometimes it is not possible to accurately forecast far in the future. Intuively, it's not because you've observed a consistent sales growth of 2% per month for the last 10 months, that it makes sense to predict a 2% growth per month for the next 10 years.

Partner's tracking

We propose a revenue sharing program for partners. In order to track affiliate customers, you need to call once the web method SetPartner with your Partner ID identifier and the authentication tokens of your customers.

Affiliate customers appear in the dashboard of the Partner Web Application.