Web Services

This tutorial describes how to create a Web Application with a HTML/JavaScript front-end that run in the users browser and a backend written in Visual Prolog.

The emphasis is on the Visual Prolog back-end, which implement a JSON-RPC web service. The HTML/JavaScript code is mainly included to make the example self-contained and to illustrate how front-end code can interact with a Visual Prolog web service.

It is described how the same service can:


 * Run as a stand-alone HTTP/HTTPS server
 * Run embedded as an ISAPI extension in Microsoft Information Services (IIS)

Notice that web services requires the commercial edition of Visual Prolog. Also notice that this tutorial will not consider SOAP at all.

The tutorial corresponds to the webRPC examples in the commercial edition (IDE: Help ->Install Examples...).

Overview
In this section we will try to describe the overall picture of the entire application and the way it works. What takes place is actually rather complex, but fortunately most of the complexity is handled automatically. An understanding of the overall process including knowledge of some of the complexity will however make it easier to understand what needs to be done and why.

The overall work flow of the example application is as follows: The user browses (in a web browser) to the page http: // /file/test.htm (or http: // /file/calendar.htm), and the server returns the corresponding file to the browser. The browser evaluates the contents of the file, which may result in additional requests for several cascading style scripts (css) files, JavaScript (js) files, images, etc. Furthermore, the HTML file contains embedded JavaScript code, which will make remote procedure calls (RPC) to the web server. The server will execute the corresponding procedures and return the result to the browser. Finally, the embedded JavaScript uses the returned data to update the HTML contents of the browser.

The communication between the client and the server is performed by means of the HTTP (or HTTPS) protocol. So on the server side we need an HTTP server, which can both return files and implement remote procedure calls. The program in jsonRpcService_httpApi runs the service as a stand alone HTTP server that both handles file requests and implements the remote procedure calls. The program in jsonRpcService_isApi implements the remote procedure calls as an ISAPI plug-in, the IIS (Microsoft Internet Information Services) is configured to handle the file requests and to use the ISAPI for the remote procedure calls.

The stand alone HTTP server is implemented by means of Microsoft's HTTP Server API, which can share ports and URL's with the Internet Information Services (IIS), such that http: // /file, etc can go to our stand alone server while http: // /  can be handled by the IIS.

JSON & JSON-RPC
The remote procedure calls are implemented in accordance with the JSON-RPC 2.0 Specification specification, which implies that data is encoded in JSON (JavaScript Object Notation) format.

JSON syntax is a sub-set of JavaScript (hence the name): When evaluating a valid JSON text it will become a corresponding JavaScript object. You should however always call a JSON parser instead of evaluating the text: evaluating text is a security risk, since the text could be "code" rather than "data".

It is not necessary to understand the JSON format in details, because on the client side you will work with JavaScript and on the server side you will work with an object/domain representation of JSON: parsing and writing is left to standard software.

When performing a JSON-RPC call the client sends a JSON request object, and the server will respond with a JSON response object, unless the request is a notification, in which case there is no response.

Request objects have four members:
 * jsonrpc: A string specifying the version of the JSON-RPC protocol. MUST be exactly "2.0".
 * method: A string containing the name of the method to be invoked.
 * params: The method parameters (omitted if the method does not take parameters).
 * by-position: a JSON array, containing the values in the Server expected order.
 * by-name: a JSON object, with member names that match exactly, including case, to the method's expected parameters
 * id: A client created string or number used to identify the request. The id field is omitted for notifications.

A response object contains three members: The third member is one of these:
 * jsonrpc A String specifying the version of the JSON-RPC protocol. MUST be exactly "2.0".
 * id The value submitted by the client.
 * result (implying that the call succeeded) The result of the procedure call
 * error</vp> (implying that the call failed) An object describing the error which occurred

Client side
There are many other ways to make client side code, typically you will use some standard JavaScript library packages that handles errors and perform the calls asynchronously. calendar.htm uses such library code, but that is outside the scope of this tutorial.

Service Code
In Visual Prolog JSON is represented by the domain json::jsonValue</vp>:

domains jsonValue = n; % null f; % false t; % true r(real Number); s(string String); a(jsonValue* Array); o(jsonObject Object).

There are only 7 kinds of JSON values: null, false, true, a number, a string, an array and an object.

An array is a sequence of JSON values represented as a list in Visual Prolog.

A JSON object is represented by the jsonObject</vp> interface in Visual Prolog:

interface jsonObject supports mapM{string Key, jsonValue Value} ... end interface jsonObject

The interface contains a number of convenience predicates/properties (which have been omitted above), but in essence a jsonObject</vp> is simply a map from Key's (names) to JSON values.

When creating an RPC service you will have to implement an object that supports the rpcService</vp> interface:

interface testService supports rpcService end interface testService

class testService : testService end class testService

In the implementation you will inherit from jsonRpcServiceSupport</vp> which will handle the parsing, writing, etc of JSON:

implement testService inherits jsonRpcServiceSupport open core, json ...

end implement testService

You will register your procedures in the constructor:

clauses new :- setProc_name("hello", hello), setProc_name("calendar", calendar).

There are four kinds of procedures, with corresponding registration predicates:


 * setProc_array</vp> for registering a procedure that uses positional arguments in an array
 * setProc_name</vp> for registering a procedure that uses named arguments in an object
 * setListener_array</vp> for registering a notification listener that uses positional arguments in an array
 * setListener_name </vp> for registering a notification listener that uses named arguments in an object

In the example there are two procedures "hello" and "calendar" that both uses named arguments. The "hello" code looks like this:

constants testService_error : jsonRpcError::serverErrorCode = jsonRpcError::serverError_last+1. % grows up from "last", which is used for unspecified server error

predicates hello : jsonProc_name. clauses hello(_Context, ArgMap) = o(Result) :- Result = jsonObject::new, Username = ArgMap:get_string("username"), if "error" = Username then raise_serviceError(testService_error, "'error' is an invalid user name") elseif "userError" = Username then exception::raise_user("'%Name%' is an invalid user name", [namedValue("Name", string(Username))]) elseif "definiteUserError" = Username then exception::raise_definiteUser("'%Name%' is an invalid user name", [namedValue("Name", string(Username))]) else Result:set_string("token", string::concat("Hello ", Username)) end if.

First the hello</vp> procedure creates a jsonObject</vp> for the Result</vp>. Any kind of JSON value are legal as result, but hello</vp> is designed to return a JSON object with a "token"</vp> member.

Next it fetches the arguments from ArgMap</vp>. It expects a string argument named <vp>"username"</vp>.

<vp>hello</vp> uses the value of <vp>Username</vp> to illustrate exception handling and success.

The <vp>"error"</vp> case illustrates how you can raise "service errors", that is errors that are specifically designed for the RPC service. The "userError" and "definiteUserError" cases illustrates what will happen with other exceptions that occurs in the procedure call.

Notice that the client side code should be programmed to deal with errors. On the server side you may of course also handle specific exceptions as in any other program, and perhaps you also want to log all errors somewhere.

If <vp>Username</vp> is not one of the error cases, we map "token" to a string in the <vp>Result</vp>.

Stand-alone HTTP server
As illustrated by the <vp>jsonRpcService_httpApi</vp> program the <vp>testService</vp> (supporting the <vp>rpcService</vp>) can be used in a stand-alone HTTP server. It does not require much code and it is quite simple to adjust the code to specific needs. Nevertheless the complexity is quite large.

The server is based on Microsoft's HTTP Server API which cooperates well with Microsoft Internet Information Services (IIS). In fact, it seems that IIS is built on top of the HTTP Server API (or at least that both are built on top of some common base).

The HTTP Server api operates with the following entities:
 * Creator/Controller Process
 * Server Session
 * URL groups
 * Request queues
 * Worker Processes

The creator/controller process creates a server session, which defines URL groups and request queues. The worker processes attach to the request queues and process the requests they receive.

In the example program the creator/controller process is also the only worker process. And it is outside the scope of this tutorial to consider more complex scenarios.

The server session is the "owner"/"holder" of the URL groups and the request queues, which are the things we really need to consider.

An URL group defined by a set of URL patterns of the forms:
 * http: // : /<relativeURI>
 * https: // : /<relativeURI>

When a request arrives at the computer a matching mechanism will match it against the URL patterns in the active URL groups, the matching will either fail or determine an URL group that covers the URL.

Each URL group is attached to a request queue, and requests matched to a certain URL group is placed in the attached request queue. URL groups that are not attached to a request queue are inactive and does not participate in the matching.

Worker processes dequeue requests from the request queues and handle them. A request queue can have several URL groups attached and each URL group can have several URL patterns. A worker process dequeuing from a certain request queue will therefore receive requests that match some URL pattern in some URL group attached to that queue.

In the example server we create two request queues each having a single URL group attached, and each of these URL groups having one URL pattern. Effectively:
 * <vp>reqQueue_rpc</vp> receives requests of the form http: //localhost:5555/jsonrpc/ , and
 * <vp>reqQueue_file</vp> receives requests of the form http: //localhost:5555/file/ '

<vp>reqQueue_rpc</vp> is a <vp>requestQueue_rpc</vp> handles RPC requests by feeding them to our <vp>testService</vp>.

<vp>reqQueue_file</vp> is a <vp>requestQueue_file</vp> which uses <vp>fileMapper</vp> to map the requests to a file name and an associated <vp>ContentType</vp>, the <vp>requestQueue_file</vp> will then handle the transfer of the file.

Server configuration: netsh
The example runs smoothly because it uses port 5555 on localhost. Normally you will however want to run applications on port 80 for http and 443 for https, and that will cause some security problems.

If you just change the port number and uses a public hostname you server program will get an 'Access is denied' exception when trying to add the URL patterns to the URL groups.

If you run the server "as Administrator" you will not get the access violation, but it is absolutely not advisable to run the program as administrator, because then it also have rights to do many other harmful things.

To make the program run in normal modes you will have to reserve the URL patterns in question for the user that run the server program.

This is done using the netsh.exe (net shell) program. You will need to manipulate the http URL acl's. For a detailed description see Netsh commands for HTTP.

It is possible to make the URL reservations from Visual Prolog, but in case of conflicts and problems it may be better to use the standard program netsh.

Notice that the example program is a console program. In an larger scale context it would seem better to create a genuine windows service. Visual Prolog has a template project for creating services, but that subject is outside the scope of this tutorial.

IISAPI plug-in
As illustrated by the <vp>jsonRpcService_isApi</vp> program the <vp>testService</vp> (supporting the <vp>rpcService</vp>) can also be used in an ISAPI dll, which can be used by the Internet Information Services to run the RPC requests.

<vp>jsonRpcService_isApi</vp> does not deal with file requests, because it is more natural to leave that to the IIS itself. The dll code is actually rather trivial, the key issue is the IIS configuration.

The configuration of the IIS has been put into its own tutorial, see IIS configuration.

Existing GUI program
In the webRPC example directory there is also a <vp>jsonRpcService_httpGUI</vp>, which illustrated how a GUI program can act like a stand-alone Web RPC Service. You should notice that it is not recommended to let a real service program have a GUI, but it can be quite useful if you need to let other programs interoperate with the GUI program, i.e. in a normal user login session. It can also be used as a "quick and dirty" way to expose already existing GUI code as a Web Service (perhaps as a test before creating the real service).

Multi-threading
No matter how you run the web service (stand-alone with or without GUI, or as ISAPI) you should notice that the HTTP requests run in parallel in several threads. So it is important to ensure that the request handling's are thread safe. The <vp>requestQueue_rpc::synchronizer</vp> property can be used to synchronize the execution of the RPC's, the property is set to a predicate which will receive <vp>runnable</vp>s, when the runnable is executed the actual RPC call is performed. The runnable's can for example be executed in a monitor ensuring that only one at the time is execute, or be put on a queue and executed sequentially from that or as <vp>jsonRpcService_httpGUI</vp> does be posted to the GUI queue and thus executed by the GUI thread.

Visual Prolog Client
This tutorial and the mentioned examples has focused on the Visual Prolog server side, using a web browser as client side. But sometimes a Visual Prolog program may need to access a web service as client. The example <vp>jsonRpcClient</vp> illustrates how this can be done. The method is quite similar to the one used in JavaScript:


 * Create a <vp>jsonRpcRequest</vp> object, with method name and parameters
 * JSON "stringify" it using <vp>asString</vp>
 * POST it using an XMLHttpRequest object (an instance of <vp>xmlHTTP</vp> or <vp>xmlHTTP60</vp>)
 * JSON parse the response text (using <vp>jsonObject::fromString</vp>)
 * Check for "error" member

In the example program you see a <vp>delayCall</vp>, whose purpose is to allow the GUI to be updated, without this call the GUI would not update until after the RPC have completed. The call is however irrelevant for the RPC itself.

HTTPS
HTTPS is the HTTP protocol used over a secure socket. Both the IIS and the stand alone HTTP Server can use HTTPS. To use HTTPS the server computer must have a certificate. It is possible to create a certificate yourself, but a self created certificate is "untrusted" will therefore cause security notifications. This is sufficient for testing and development purposes, but for real scale purposes you will have to aquire a trusted certificate.

In brief a trusted certificate have to be obtained from a trusted provider. The trust lies in three things:
 * The provider is itself trusted
 * The provider have knowledge about the "real identity" of the certificate holder (through normal purchase chains)
 * The provider can revoke the certificate if it is misused somehow

Notice that HTTPS by default uses port 443 (and HTTP uses port 80), and that you have to write "https" in the places where you would otherwise write "http" (e.g. https: //+:443/mypath)

It is outside the scope of this tutorial to consider certificates any further.