Web Services

This tutorial describes how to create a Web Application with a HTML/JavaScript front end that run in the users browser and a backend written in Visual Prolog.

The emphasis is on the Visual Prolog back-end, which implement a JSON-RPC Web Service. The HTML/JavaScript code is mainly included to make the example self-contained and to illustrate how font-end code can interact with a Visual Prolog web service.

It is described how the same service can:

• Run as a stand-alone HTTP/HTTPS server
• Run embedded as an ISAPI extension in Microsoft Information Services (IIS)

Notice that web services requires the commercial edition of Visual Prolog. Also notice that this tutorial will not consider SOAP at all.

The tutorial corresponds to the webRPC examples in the commercial edition (IDE: Help ->Install Examples...).

Overview

In this section we will try to describe the overall picture of the entire application and the way it works. What takes place is actually rather complex, but fortunately most of the complexity is handled automatically. An understanding of the overall process including knowledge of some of the complexity will however make it easier to understand what needs to be done and why.

The overall work method of the example application is as follows: The user browses (in a web browser) to the page http://<server>/file/test.htm (or http://<server>/file/calendar.htm), the server returns the corresponding file to the browser. The browser evaluates the contents of the file, which may result in additional requests for several cascading style scripts (css) files, JavaScript (js) files, images, etc. Furthermore, then html file contains embedded JavaScript code, which will make remote procedure calls (RPC) to the web server. The server will execute the corresponding procedures and return the result to the browser. Finally, the embedded JavaScript uses the returned data to update the HTML contents of the browser.

The communication between the client and server side is performed by means of the HTTP (or HTTPS) protocol. So on the server side we need an HTTP server, which can both return files and implement remote procedure calls. The program in jsonRpcService_httpApi runs the service as a stand alone HTTP server that both handles file requests and implements the remote procedure calls. The program in jsonRpcService_isApi implements the remote produre calls as an ISAPI plugin, the IIS (Microsoft Internet Information Services) is then configured to handle the file requests and to use the ISAPI for the remote procedure calls.

The stand alone HTTP server is implemented by means of Microsofts HTTP Server API, which can share ports and url's with the Internet Information Services (IIS), such that http://<server>/file, etc can go to our stand alone server while http://<server>/<something> can be handled by the IIS.

JSON & JSON-RPC

The remote procedure calls are implemented in accordance with the JSON-RPC 2.0 Specification specification, which implies that data is encoded in JSON (JavaScript Object Notation) format.

JSON syntax is a sub-set of JavaScript (hence the name): When evaluating a valid JSON text it will be come a corresponding JavaScript object. You should however always call a JSON parser instead of evaluating the text: evaluating text is a security risk, since the text could be "code" rather than "data".

It is not necessary to understand the JSON format in details, because on the client side you will work with JavaScript and on the server side you will work with an object/domain representation of JSON: parsing and writing is left to standard software.

When performing a JSON-RPC call the client side sends a JSON request object, and then server side will respond with a JSON response object, unless the request is a so called notification in which case there is no response.

Request object have four members:

• jsonrpc A String specifying the version of the JSON-RPC protocol. MUST be exactly "2.0".
• method A String containing the name of the method to be invoked.
• params The method parameters (omitted if the method does not take parameters)
  • by-position: a JSON array, containing the values in the Server expected order.
  • by-name: a JSON object, with member names that match exactly, including case, to the method's expected parameters
• id: an client created string or number used to identify the request. The id field is omitted for notifications.

A response object contains three members:

• jsonrpc A String specifying the version of the JSON-RPC protocol. MUST be exactly "2.0".
• id The value submitted by the client

The third member is one of these:

• result (implying that the call succeeded) The result of the procedure call
• error (implying that the call succeeded) An object describing the error which occurred

Client side

Example This code shows one way to make a procedure call on the client side:

function httpPOST(URL, method, params) {
    var ReqObj = { jsonrpc: "2.0", id: getId(), method: method, params: params };
    var Req = JSON.stringify(ReqObj);
    var xmlHttp = new XMLHttpRequest();
    xmlHttp.open("POST", URL, false);
    xmlHttp.setRequestHeader('Content-Type', 'application/json-rpc; charset=UTF-8');
    xmlHttp.send(Req);
    var resp = xmlHttp.responseText;
    var respObj = JSON.parse(resp);
    var error = respObj.error;
    if (error) {
        if (Data = error.data) {
            log("<pre>" + Data +"</pre>\n");
        }
        log("<p>error : "+ error.code +": " + error.message + "</p>\n");
    } else {
        // respObj.result contains the result
    }
}

In brief:

• Create a request object
• JSON "stringify" it
• POST it using an XMLHttpRequest object
• JSON parse the response text
• Check for "error" member

There are many other ways to make client side code, typically you will use some standard JavaScript library packages, that handles errors and perform the calls asynchroneously. calendar.htm uses such library code, but that is outside the scope of this tutorial.

Service Code

In Visual Prolog JSON is represented by the domain json::jsonValue:

domains
    jsonValue =
        n; % null
        f; % false
        t; % true
        r(real Number);
        s(string String);
        a(jsonValue* Array);
        o(jsonObject Object).

There are only 7 kinds of JSON values null, false, true, a number, a string, an array and an object.

An array is a sequence of JSON values represented as a lins in Visual Prolog.

A JSON object represented by the jsonObject interface in Visual Prolog:

interface jsonObject supports mapM{string Key, jsonValue Value}
...
end interface jsonObject

The interface contains a number of convenience predicates/properties (which have been omitted above), but in essence a jsonObject is simply a map from Key's (names) to JSON values.

Example This code:

J = jsonObject::new(),
J:set_boolean("bbb", true),
J:set_string("sss", "The String"),
J:writeTo(stdio::outputStream),
stdio::nl,
jsonPrettyPrint::ppObject(stdio::outputStream, J)

will produce this output:

{"bbb":true,"sss":"The String"}
{
    "bbb" : true,
    "sss" : "The String"
}

writeTo produces a condensed JSON representation suitable interprocess communication, etc. jsonPrettyPrint can be used to pretty print the values (for example when debugging).

When creating an RPC service you will have to implement an object that supports the rpcService interface:

interface testService supports rpcService
end interface testService
 
class testService : testService
end class testService

In the implementation you will inherit from jsonRpcServiceSupport which will handle the parsing, writing, etc of JSON:

implement testService
    inherits jsonRpcServiceSupport
    open core, json
...
 
end implement testService

You will register your procedures in the constructor:

clauses
    new() :-
        setProc_name("hello", hello),
        setProc_name("calendar", calendar).

There are four kinds of procedures, whith corresponding registration predicates:

• setProc_array for registering a procedure that uses positional arguments in an array
• setProc_name for registering a procedure that uses named arguments in an object
• setListener_array for registering a notification listener that uses positional arguments in an array
• setListener_name for registering a notification listener that uses named arguments in an object

In the example there are two procedures "hello" and "calendar" that both uses named arguments. The "hello" code looks like this:

constants
    testService_error : jsonRpcError::serverErrorCode = jsonRpcError::serverError_last+1.
    % grows up from "last", which is used for unspecified server error
 
predicates
    hello : jsonProc_name.
clauses
    hello(_Context, ArgMap) = o(Result) :-
        Result = jsonObject::new(),
        Username = ArgMap:get_string("username"),
        if "error" = Username then
            raise_serviceError(testService_error, "'error' is an invalid user name")
        elseif "userError" = Username then
            exception::raise_user("'%Name%' is an invalid user name", [namedValue("Name", string(Username))])
        elseif "definiteUserError" = Username then
            exception::raise_definiteUser("'%Name%' is an invalid user name", [namedValue("Name", string(Username))])
        else
            Result:set_string("token", string::concat("Hello ", Username))
        end if.

First the hello procedure creates a jsonObject for the Result. Any kind of JSON value are legal as result, but hello is designed to return a JSON object with a "token" member.

Next it fetches the arguments from ArgMap. It expects a string argument named "username".

hello uses the value of Username to illustrate exception handling and success.

The "error" case illustrates how you can raise "service errors", that is errors that are specifically designed for the RPC service. The "userError" and "definiteUserError" cases illustrates what will happen with other exceptions that occurs in the procedure call.

Notice that the client side code should be programmed to deal with errors. On the server side you may of course also handle specific exceptions as in any other program, and perhaps you also want to log all errors somewhere.

If Username is not one of the error cases, we map "token" to a string in the Result.

Example We reccomend that your jsonProc's only deal with argument extraction and result construction, and that you keep the real functionaly in a separate business layer as illustrated by this code:

predicates
    hello : jsonProc_name.
clauses
    hello(_Context, ArgMap) = o(Result) :-
        Result = jsonObject::new(),
        Username = ArgMap:get_string("username"),
        Token = businessLayer::hello(Username),  % actual functionalty is in a separate business layer
        Result:set_string("token", Token).

Stand-alone HTTP server

As illustrated by the jsonRpcService_httpApi program the testService (supporting the rpcService) can be used in a stand-alone HTTP server. It does not require much code and it is quite simple to adjust the code to specific needs. Nevertheless the complexity is quite large.

The server is based on Microsofts HTTP Server API which cooperates well with Microsoft Internet Information Services (IIS). In fact, it seems that IIS is built on top of the HTTP Server API (or at least that both are built on top of some common base).

The HTTP Server api operates with the following entites:

• Creator/Controller Process
• Server Session
• URL groups
• Request queues
• Worker Processes

The creator/controller process creates a server session, which defines URL grups and request queues. The worker processes attach to the request queues and process the requests they receive.

In the example program the creator/controller process is also the only worker process. And it is outside the scope of this tutorial to consider more complex scenarios.

The server session is the "owner"/"holder" of the URL groups and the request queues, which are the things we really need to consider.

An URL group defined by a set uf URL patterns of the forms:

http://<host>:<port>/<relativeURI>

https://<host>:<port>/<relativeURI>

When a request arrives at the computer a macthing mechanism will match it agains the URL patterns in the active URL groups, the matching will either fail or determine an URL group that covers the URL.

Each URL group is attached to a request queue, and requests matched to a certain URL group is placed in the attached request queue. URL groups that are not attached to a request queue are inactive and does not participate in the matching.

Worker processes dequeue requests from the request queues and handle them. A request queue can have several URL groups attached and each URL group can have several URL patterns. A worker process dequeuing from a certain request queue will therefore receive requests that match som URL pattern in some URL group attached to that queue.

In the example server we create two request queues each having a single URL group attached, and each of these URL groups having one URL pattern. Effectively:

• reqQueue_rpc receives requests of the form http://localhost:5555/jsonrpc/<something>, and
• reqQueue_file receives requests of the form http://localhost:5555/file/<something>'

reqQueue_rpc is a requestQueue_rpc handles RPC requests by feeding them to our testService.

reqQueue_file is a requestQueue_file which uses fileMapper to map the requests to a file name and an associated ContentType, the requestQueue_file will then handle the transfer of the file.

= Server configuration: netsh

The example runs smoothly because it uses port 5555 on localhost. Normally you will however want to run applications on port 80 for http and 443 for https, and that will cause some security problems.

If you just change the port number and uses a public hostname you server program will get an 'Access is denied' exception when trying to add the URL patterns to the URL groups.

If you run the server "as Administrator" you will not get the access violation, but it is abaolutely not advisable to run the program as administrator, because then it also have rights to do many other harmful things.

To make the program run in normal modes you will have to reserve the URL patterns in question for the user that run the server program.

This is done using the netsh.exe (net shell) program. You will need to manipulate the http URL acl's. For a detailed description see Netsh commands for HTTP.

Example This will give an idea of what to do (in a cmd prompt):

>netsh http add urlacl http://www.something.com:80/file/ user=SMT\serviceuser
>netsh http add urlacl http://www.something.com:80/jsonrpc/ user=SMT\serviceuser

Notice that the cmd prompt be run "as Administrator" to make modifcantions. If you just write "netsh" it will go into a prompt mode where change to http mode and then just write http commands

>netsh
netsh>http
netsh http>add urlacl http://www.something.com:80/file/ user=SMT\serviceuser
netsh http>add urlacl http://www.something.com:80/jsonrpc/ user=SMT\serviceuser
netsh http>exit

It is possible to make the URL reservations from Visual Prolog, but in case of conflicts and problems it may be better to use the standard program netsh.

Notice that the example program is a console program. In an larger scale context it would seem better to create a genuine windows service. Visual Prolog has a template project for creating services, but that subject is outside the scope of this tutorial.

IISAPI plugin

As illustrated by the jsonRpcService_isApi program the testService (supporting the rpcService) can also be used in an ISAPI dll, which can be used by the Internet Information Services to run the RPC requests.

jsonRpcService_isApi does not deal with file requests, because it is more natural to leave that to the IIS itself. The dll code is actually rather trivial, the key issue is the IIS configuration.

The configuration of the IIS has been put into its own turorial, see IIS configuration.

Existing GUI program

In the webRPC example directory there is also a jsonRpcService_httpGUI, which illustrated how a GUI program can act like a stand-alone Web RPC Service. You should nottice that it is not recommended to let a real service program have a GUI, but it can be quite useful if you need to let other programs interoperate with the GUI program, i.e. in a normal user login session. It can also be used as a "quick and dirty" way to expose already existing GUI code as a Web Service (perhaps as a test before creating the real service).

Multi-threading

No matter how you run the web service (stand-alone with or without GUI, or as ISAPI) you should notice that the HTTP requests run in parallel in several threads. So it is important to ensure that the request handlings are thread safe. The requestQueue_rpc::synchronizer property can be used to synchronize the execution of the RPC's, the property is set to a predicate which will receive runnables, when the runnable is executed the actual RPC call is performed. The runnable's can for example be executed in a monitor ensuring that only one at the time is exectet, or be put on a queue and executed sequentialy from that or as jsonRpcService_httpGUI does be posted to the GUI queue and thus executed by the GUI thread.

See also

• JSON
• JSON-RPC 2.0 Specification
• Microsofts HTTP Server API