Developers » Web Services » Here

Web Services Tutorial with Python

Last revision: 2008-01-02


Target audience: developers, system administrators

In this tutorial, we will see how to use Lokad Web Services with Python. Lokad specializes in business time-series forecasting, but this document is neither an introduction to Lokad nor an introduction to Web Services.

Download: source code for this tutorial

Requirements

• A Lokad account (registration is free).
• Web Services: limited knowledge.
• Python knowledge level: intermediate to advanced (check BeginnersGuide)
• Python version 2.4 or higher installed.
• ZSI (Zolera SOAP Infrastructure): web services toolkit for Python,

latest version installed (ZSI homepage).

(*) We suggest to use the Sandbox server because its usage is free for development purposes.

Introduction

Web Services are an industrial-strength standard currently embraced by most (if not all) major software vendors. Lokad features Web Services: all the operations that can be performed through the web application can also be performed programmatically through our Web Services. Lokad Web Services can be reached at

• (Production) http://ws.lokad.com/TimeSeries.asmx (WSDL)
• (Sandbox) http://sandbox-ws.lokad.com/TimeSeries.asmx (WSDL)

In this tutorial, we will see how to use Lokad Web Services within a Python application. For this task we'll use the Zolera SOAP Infrastructure (ZSI). ZSI appears to be the most mature Web Services tool available for Python, yet it is under active development and some features familiar to other web services toolkits may be still missing or incomplete.

At this time, the documentation on the official site may be out of date. If in trouble try consulting the Python Web Services mailing list.

Installing ZSI

The latest version of ZSI on the time of writing this tutorial was 2.1-a1.

The easiest way to install ZSI is through setuptools and easy_install. Download ZSI-2.1_a1-py2.5.egg from SourceForge and run, from the terminal,

easy_install ZSI-2.1_a1-py2.5.egg

If you're using Python 2.4, download and install ZSI-2.1_a1-py2.4.egg instead.

Alternatively, you can download the source tarball ZSI-2.1-a1.tar.gz and install it using:

tar xzvf ZSI-2.1-a1.tar.gz
cd ZSI-2.1
python setup.py

Generating code stubs with wsdl2py

ZSI provides a tool, called wsdl2py, which reads web service description from a WSDL file or URL, and constructs helper Python modules that one can use to access the service.

To generate code to access Lokad TimeSeries service, it should be called as follows:

wsdl2py --complexType http://sandbox-ws.lokad.com/TimeSeries.asmx?wsdl

Here the '--complexType' switch indicates that we wish to add convenience functions for managing complex data types, and the URL points to the WSDL service description (note that we use sandbox version for testing purposes). Alternatively, one could replace the URL with the path to a local file, containing the WSDL data.



The above command will download and parse WSDL file and create in the current folder three Python modules:

• TimeSeries_client.py,
• TimeSeries_server.py,
• TimeSeries_types.py.

The standard UNIX command wc counts the number of lines in each file.

Accessing the web service

Due to the dynamic nature of the ZSI framework, powerful IDEs (such as Eclipse), which rely on code inspection, are not of great use. Thus we will use runtime reflection in interactive mode, from an advanced shell like IPython.

We begin with importing the client module - one of the three modules generated by wsdl2py, and we'll abbreviate it's name as c for convenience:



Locator is a helper class; the method getTimeSeriesSoap() takes an optional url parameter, which can be used to bind to a service to a different host from than the one defined in the WSDL. For now, we will use the default URL. The returned value is an instance of the TimeSeriesSoapSOAP class, which contains all of the methods defined by the web service. The screenshot displays the auto-completion feature of IPython:



However, simply calling srv.IsAuthenticated() fails: ZSI requires each call to a web service method to have exactly one parameter - the message sent to the server - and it's required even when the remote method takes no meaningful arguments. Therefore, the correct call to srv.IsAuthenticated() will look as following (remember that c stands for the TimeSeries_client module):



Here msg is an instance of the class IsAuthenticatedSoapIn, which contains the data (actually void) transmitted to IsAuthenticated(); rep contains the server response, and the actual value returned by the method is available through IsAuthenticatedResult attribute - this name is most easily guessed using reflection (i.e. typing <TAB> twice after rep. in IPython shell).

Web Services Authentication

The reason why IsAuthenticated() returned False in the last example is because Lokad Web Services are authenticated, which means that one have to provide the username and password associated with a Lokad Web Account to access the Lokad Web Services. Technically, Lokad relies on SOAP headers to authenticate a Web Method call. The code provided below illustrates how you can check that your call is correctly authenticated.

#
# test1.py
#

import TimeSeries_client as c

username = "email@example.com"
password = "mypassword"

AuthHeader = c.GED("lokad.com/ws", "AuthHeader").pyclass

ah = AuthHeader()
ah.UserName = username
ah.Password = password

loc = c.TimeSeriesLocator()
srv = loc.getTimeSeriesSoap()

print "Call with no headers..."
msg = c.IsAuthenticatedSoapIn()
rep = srv.IsAuthenticated(msg)
print "Is Authenticated:", rep.IsAuthenticatedResult

print "Call with AuthHeader added..."
msg = c.IsAuthenticatedSoapIn()
rep = srv.IsAuthenticated(msg, soapheaders = [ah])
print "Is Authenticated:", rep.IsAuthenticatedResult

Here AuthHeader is a Python class, corresponding to the AuthHeader entity in the namespace lokad.com/ws. The way we obtain this class may look a bit mysterious, and if you dig into ZSI documentation, you'll find that GED stands for get global element declaration. But hopefully this is the only place we're going to use this function.

Now that we have the class AuthHeader, we create an instance of this class ah and assign authentication credentials to it's attributes. On the invocation of IsAuthenticated method, an optional argument soapheaders is passed, which contains a list of SOAP headers to be sent to the server with the SOAP request.

The expected console output for the above code sample is:

Call with no headers...
Is Authenticated: False
Call with AuthHeader added...
Is Authenticated: True

Currently ZSI does not allow default SOAP headers to be sent with every request, which means that the soapheaders argument must be added manually to every remote method invocation.

This is quite a bit inconvenient, thus in the following part of this tutorial we use a helper class TimeSeriesService to hide this inconvenience from the user and to interface the Web Service in a more pythonic way. Here is a part of the code that performs authentication (the full code is contained in lokad.py; hopefully the future versions of wsdl2py will be able to generate such code automatically):

#
# lokad.py
#

import TimeSeries_client as c

class TimeSeriesService:

    def __init__(self, url=None):
        self.loc = loc = c.TimeSeriesLocator()
        self.srv = srv = loc.getTimeSeriesSoap(url)
        self.soapheaders = []

    def SetAuth(self, username, password):
        AuthHeader = c.GED("lokad.com/ws", "AuthHeader").pyclass

        auth = AuthHeader()
        auth.UserName = username
        auth.Password = password
        self.soapheaders = [auth]

    def IsAuthenticated(self):
        msg = c.IsAuthenticatedSoapIn()
        rep = self.srv.IsAuthenticated(msg, soapheaders=self.soapheaders)
        return rep.IsAuthenticatedResult

Now the previous authentication example looks much nicer:

import lokad

tss = lokad.TimeSeriesService()
tss.SetAuth("email@example.com", "mypassword")
tss.IsAuthenticated() # returns True

Time-series management

Lokad specializes in time-series forecasting. Before actually performing a forecast, you first need to upload your data; and this can be easily achieved with our Web Services. Technically, a time-series is simply a named array of time-value pairs. We represent a time-series using the following Python class (again defined in lokad.py):

class TimeSerie(list):

    def __init__(self, time_value_pairs=(), name=None):
        list.__init__(self, time_value_pairs)
        self.Name = name
        self.UnitName = None
        self.UtcOffset = None

Let's see how a time-series can be generated programmatically.

Since in ZSI the complex data types are not available directly, but rather as a part of a message sent to a web service method, we begin with implementing a wrapper for the AddSerie method as a part of the TimeSeriesService class:

class TimeSeriesService:

    # part of the code skipped

    def AddSerie(self, serie):
        msg = c.AddSerieSoapIn()
        msg.Serie = msg.new_serie()
        msg.Serie.set_attribute_Name(serie.Name)

        msg.Serie.TimeValues = msg.Serie.new_TimeValues()

        timevalues = []

        for time, value in serie:
            timevalue = msg.Serie.TimeValues.new_TimeValue()
            timevalue.set_attribute_Time(time)
            timevalue.set_attribute_Value(value)
            timevalues.append(timevalue)

        msg.Serie.TimeValues.TimeValue = timevalues

        rep = self.srv.AddSerie(msg, soapheaders=self.soapheaders)
        return

Let's follow this piece of code step by step (so far, ZSI turns out to be exceedingly verbose for Python!):

• TimeSeriesService.AddSerie takes a TimeSerie object as argument.
• it creates a message object to be passed to self.srv.AddSerie.
• the message object contains a constructor new_series() for an empty object of the class TimeSeries_types.TimeSerie_Holder (which seems to be not available otherwise!)
• an instance of this object is assigned to msg.Serie.
• in a similar way, a list of TimeValue() objects, created with msg.Serie.TimeValues.new_TimeValue(), is assigned to msg.Serie.TimeValues.
• after the message is constructed, it's sent to the server together with SOAP headers. We assume that self.soapheaders contains a valid AuthHeader at the time of this call.

Now we can create a sample time-series:

#
# test2.py
#

import lokad
import time

now = time.time()
day = 24*60*60
values = [ (now+day*i, i) for i in range(10) ]

data = lokad.TimeSerie(values)
data.name = "MySampleSerie"

tss = lokad.TimeSeriesService()
tss.SetAuth("email@example.com", "mypassword")
tss.AddSerie(serie)

Note: the time-series names should be unique within the scope of your Lokad account (you cannot have two distinct time-series identified by the same name). Therefore the script here above will fail on a second run if executed twice.

To retrieve the data back from the server we need another couple of simple wrappers

class TimeSeriesService:

    # some code skipped

    def GetAllSerieNames(self):
        msg = c.GetAllSerieNamesSoapIn()
        rep = self.srv.GetAllSerieNames(msg, soapheaders=self.soapheaders)
        return rep.GetAllSerieNamesResult.String

    def GetSerie(self, SerieName):
        msg = c.GetSerieSoapIn()
        msg.SerieName = SerieName
        rep = self.srv.GetSerie(msg, soapheaders=self.soapheaders)
        return parse_serie(rep.GetSerieResult)

where parse_serie is another helper function, which retrieves the data from the SOAP response and puts it into a TimeSerie object:

def parse_serie(obj):
    values = [ (v.get_attribute_Time(), v.get_attribute_Value())
                for v in obj.TimeValues.TimeValue ]
    serie = TimeSerie(values)
    serie.Name = obj.get_attribute_Name()
    serie.UnitName = obj.get_attribute_UnitName()
    serie.UtcOffset = obj.get_attribute_UtcOffset()

    return serie

Now assuming test2.py was executed successfully, we should be able to see the data from MySampleSeries:



Note that the time was passed to AddSeries as a long integer, but is returned as a tuple. Both time representations are equally valid within ZSI. For more details on date-time representation in Python see the documentation.

Forecasting task management

Before you can retrieve a time-series forecast, you need to define a forecasting task that will specify how the forecasted time-series should be computed. In the Lokad data model, a forecasting task is always associated with a particular time-series; several forecasting tasks can be associated with the same time-series.

You can add a task to your Lokad account with the following code.

#
# test3.py
#

import lokad

tss = lokad.TimeSeriesService()
tss.SetAuth("email@example.com", "mypassword")

# note: here we use keyword arguments for better readability only,
# this is not mandatory

tss.AddTask("MySampleSerie", "MySampleTask", aggregator="Sum", period="Day",
                      future_periods = 10, past_periods = 5)

The forecasts can be retrieved as follows:



As you can see in the output below, 5 time-value pairs are actually returned, matching the future_periods value specified for this task.

To retrieve your real forecasts through our Web Services, the Enterprise subscription plan must be selected in your Lokad account (default settings). Note that you won't be charged upfront, but only at the end of each 30-day period (or you won't be charged at all if you are using the Sandbox server).