Difference between revisions of "Web Services"

From wiki.visual-prolog.com

m (Line breaking)
(rollback (wrong content inserted))
Line 1: Line 1:
This tutorial describes how to perform various configurations of the Microsoft Internet Information Services (IIS).
+
This tutorial describes how to create a Web Application with a HTML/JavaScript front end that run in the users browser and a backend written in Visual Prolog.
  
The tutorial will be based on the settings needed for the <vp>jsonRpcService_isApi</vp> in the <vp>webRPC</vp> example directory (see [[Web Services]]), but most of these things will be relevant and easily adjusted to other related cases.
+
The emphasis is on the Visual Prolog back-end, which implement a JSON-RPC Web Service. The HTML/JavaScript code is mainly included to make the example self-contained and to illustrate how font-end code can interact with a Visual Prolog web service.
  
The <vp>jsonRpcService_isApi</vp> project creates is an ISAPI plug-in (dll), which implements a JSON-RPC service.  The example also contains some related HTML, CSS and JavaScript files that must be accessible to the client browser.  So the IIS needs to be configured for two purposes:
+
It is described how the same service can:
* A file provider
 
* An ISAPI service
 
  
All IIS configuration takes place in the '''Internet Information Services Manager'''.
+
* Run as a stand-alone HTTP/HTTPS server
 +
* Run embedded as an ISAPI extension in Microsoft Information Services (IIS)
  
=== Configuring Windows Components ===
+
Notice that web services requires the '''commercial edition''' of Visual Prolog.
 +
Also notice that this tutorial will not consider SOAP at all.
  
To use the Internet Information Services, Manager and run ISAPI's and/or CGI's plugins you will have to ensure that it is at all available on the computer.
+
The tutorial corresponds to the '''webRPC''' examples in the commercial edition (IDE: '''Help ->Install Examples...''').
  
In '''Programs and Features -> Turn Windows Features on or off''' make sure that the following is available:
+
=== Overview ===
* Internet Information Services
 
* IIS Management Console
 
* CGI (not actually required for the <vp>jsonRpcService_isApi</vp> example)
 
* ISAPI Extensions
 
  
[[Image:IISM_prerequisites_1.png]]
+
In this section we will try to describe the overall picture of the entire application and the way it works. What takes place is actually rather complex, but fortunately most of the complexity is handled automatically.  An understanding of the overall process including knowledge of some of the complexity will however make it easier to understand what needs to be done and why.
[[Image:IISM_prerequisites_2.png]]
 
  
 +
The overall work method of the example application is as follows:  The user browses (in a web browser) to the page '''http:<nowiki />//<server>/file/test.htm''' (or '''http:<nowiki />//<server>/file/calendar.htm'''), the server returns the corresponding file to the browser.  The browser evaluates the contents of the file, which may result in additional requests for several cascading style scripts (css) files, JavaScript (js) files, images, etc.  Furthermore, then html file contains embedded JavaScript code, which will make remote procedure calls ('''RPC''') to the web server.  The server will execute the corresponding procedures and return the result to the browser.  Finally, the embedded JavaScript uses the returned data to update the HTML contents of the browser.
  
=== Application Pool ===
+
The communication between the client and server side is performed by means of the HTTP (or HTTPS) protocol.  So on the server side we need an HTTP server, which can both return files and implement remote procedure calls. The program in '''jsonRpcService_httpApi''' runs the service as a stand alone HTTP server that both handles file requests and implements the remote procedure calls.  The program in '''jsonRpcService_isApi''' implements the remote procedure calls as an ISAPI plug-in, the IIS (Microsoft Internet Information Services) is then configured to handle the file requests and to use the ISAPI for the remote procedure calls.
  
The ISAPI service must run in a suitable Application Pool. Application Pools provide process isolation between various parts of the IIS, which gives the following advantages:
+
The stand alone HTTP server is implemented by means of Microsoft's [http://msdn.microsoft.com/en-us/library/windows/desktop/aa364510(v=vs.85).aspx HTTP Server API], which can share ports and URL's with the Internet Information Services (IIS), such that '''http:<nowiki />//<server>/file''', etc can go to our stand alone server while '''http:<nowiki />//<server>/<something>''' can be handled by the IIS.
* It makes the overall IIS more robust as problem in one application pool is less likely to have impact on the other application pools (exhausting the entire system will of course cross interfere)
 
* Each pool can have different settings (of for example .net platform)
 
* Each pool can be restarted independent of the other pools
 
  
To create an application pool, select "Add Application Pool..." in the context menu on the '''Application Pools'''. Pools suitable for Visual Prolog programs, does not require any .net platform, and it is recommended to have no .net support at all to minimize the interference risks. You give it a name and (unless you for other reasons need it) select "No Managed Code" (the rest you leave as it is):
+
=== JSON & JSON-RPC ===
  
[[Image:IISM_addAppPool_1.png]]
+
The remote procedure calls are implemented in accordance with the [http://www.jsonrpc.org/specification JSON-RPC 2.0 Specification] specification, which implies that data is encoded in [http://www.json.org/ JSON] ('''J'''ava'''S'''cript '''O'''bject '''N'''otation) format.
[[Image:IISM_addAppPool_2.png]]
 
  
=== 32 bit support ===
+
JSON syntax is a sub-set of JavaScript (hence the name): When evaluating a valid JSON text it will be come a corresponding JavaScript object.  You should however ''always'' call a JSON parser instead of evaluating the text: evaluating text is a security risk, since the text could be "code" rather than "data".
  
Notice that by default (on a 64 bit platform) the application pool cannot run 32 bit programs, so you may need to switch this "Advanced Setting" on:
+
It is not necessary to understand the JSON format in details, because on the client side you will work with JavaScript and on the server side you will work with an object/domain representation of JSON: parsing and writing is left to standard software.
  
[[Image:IISM_32bit_1.png]]
+
When performing a JSON-RPC call the client side sends a JSON request object, and then server side will respond with a JSON response object, unless the request is a so called notification in which case there is no response.
[[Image:IISM_32bit_2.png]]
 
  
=== File Serving ===
+
Request object have four members:
 +
* <vp>jsonrpc</vp> A String specifying the version of the JSON-RPC protocol. MUST be exactly "2.0".
 +
* <vp>method</vp> A String containing the name of the method to be invoked.
 +
* <vp>params</vp> The method parameters (omitted if the method does not take parameters)
 +
**by-position: a JSON array, containing the values in the Server expected order.
 +
**by-name: a JSON object, with member names that match exactly, including case, to the method's expected parameters
 +
* <vp>id</vp>: an client created string or number used to identify the request.  The <vp>id</vp> field is omitted for notifications.
  
The <vp>jsonRpcService_isApi</vp> example needs to serve files to the Web Browser that run the Web Application. Such one can be set up either as a "Virtual Directory" or an "Application" ("Application" is only needed if programs also have to be run in that part of the Web).
+
A response object contains three members:
 +
* <vp>jsonrpc</vp> A String specifying the version of the JSON-RPC protocol. MUST be exactly "2.0".
 +
* <vp>id</vp> The value submitted by the client
 +
The third member is one of these:
 +
* <vp>result</vp> (implying that the call succeeded) The result of the procedure call
 +
* <vp>error</vp> (implying that the call succeeded) An object describing the error which occurred
  
[[Image:IISM_add_webRPC_1.png]]
+
=== Client side ===
[[Image:IISM_add_webRPC_2.png]]
 
  
The '''Alias''' defines name relative to the '''Path''' in the WEB, and '''Physical path''' defines the disk location that this path corresponds to, in this case
+
{{Example|This code shows one way to make a procedure call on the client side:
'''http:<nowiki />//localhost/webRPC''' will correspond to '''C:\webRPC\web''', which is where the html, css and JavaScript files for the example are placed (in my case).
+
<source lang="JavaScript">
 +
function httpPOST(URL, method, params) {
 +
    var ReqObj = { jsonrpc: "2.0", id: getId(), method: method, params: params };
 +
    var Req = JSON.stringify(ReqObj);
 +
    var xmlHttp = new XMLHttpRequest();
 +
    xmlHttp.open("POST", URL, false);
 +
    xmlHttp.setRequestHeader('Content-Type', 'application/json-rpc; charset=UTF-8');
 +
    xmlHttp.send(Req);
 +
    var resp = xmlHttp.responseText;
 +
    var respObj = JSON.parse(resp);
 +
    var error = respObj.error;
 +
    if (error) {
 +
        if (Data = error.data) {
 +
            log("<pre>" + Data +"</pre>\n");
 +
        }
 +
        log("<p>error : "+ error.code +": " + error.message + "</p>\n");
 +
    } else {
 +
        // respObj.result contains the result
 +
    }
 +
}</source>
 +
In brief:
 +
* Create a request object
 +
* JSON "stringify" it
 +
* POST it using an XMLHttpRequest object
 +
* JSON parse the response text
 +
* Check for "error" member
 +
}}
  
=== ISAPI setup ===
+
There are many other ways to make client side code, typically you will use some standard JavaScript library packages, that handles errors and perform the calls asynchronously.  '''calendar.htm''' uses such library code, but that is outside the scope of this tutorial.
  
For the example the path '''http:<nowiki />//localhost/webRPC/jsonrpc''' should run the <vp>jsonRpcService_isApi</vp> ISAPI dll.  So we add an application below the webRPC virtual directory:
+
=== Service Code ===
  
[[Image:IISM_add_jsonrpc_1.png]]
+
In Visual Prolog JSON is represented by the domain <vp>json::jsonValue</vp>:
  
Make sure that the application uses the desired Application Pool.  We don't intend to use the physical path, but the IIS will place a web.config file in that directory.  It could be a good idea to create an otherwise empty an unused directory for the purpose, but here we just use the example root directory:
+
<vip>domains
[[Image:IISM_add_jsonrpc_2.png]]
+
    jsonValue =
 +
        n; % null
 +
        f; % false
 +
        t; % true
 +
        r(real Number);
 +
        s(string String);
 +
        a(jsonValue* Array);
 +
        o(jsonObject Object).</vip>
  
With the application in place it is time to configure the ISAPI. Select '''Handler Mappings''':
+
There are only 7 kinds of JSON values null, false, true, a number, a string, an array and an object.
  
[[Image:IISM_handlerMapping_1.png]]
+
An array is a sequence of JSON values represented as a list in Visual Prolog.
  
And then add a "Script Map", matching everything and pointing to the ISAPI dll:
+
A JSON object represented by the <vp>jsonObject</vp> interface in Visual Prolog:
  
[[Image:IISM_handlerMapping_2.png]]
+
<vip>interface jsonObject supports mapM{string Key, jsonValue Value}
[[Image:IISM_handlerMapping_3.png]]
+
...
 +
end interface jsonObject</vip>
  
 +
The interface contains a number of convenience predicates/properties (which have been omitted above), but in essence a <vp>jsonObject</vp> is simply a map from Key's (names) to JSON values.
  
The JSON-RPC requests does not belong to "script" files on our disk so we have to switch off the switch off "Invoke handler only if request is mapped to''' setting in the '''Request Restrictions...''' :
+
{{Example|This code:
 +
<vip>J = jsonObject::new(),
 +
J:set_boolean("bbb", true),
 +
J:set_string("sss", "The String"),
 +
J:writeTo(stdio::outputStream),
 +
stdio::nl,
 +
jsonPrettyPrint::ppObject(stdio::outputStream, J)</vip>
 +
will produce this output:
 +
<source lang="text">{"bbb":true,"sss":"The String"}
 +
{
 +
    "bbb" : true,
 +
    "sss" : "The String"
 +
}</source>
 +
<vp>writeTo</vp> produces a condensed JSON representation suitable interprocess communication, etc.  <vp>jsonPrettyPrint</vp> can be used to pretty print the values (for example when debugging).
 +
}}
  
[[Image:IISM_handlerMapping_4.png]]
+
When creating an RPC service you will have to implement an object that supports the <vp>rpcService</vp> interface:
  
When OK-ing the changes you should also say OK to this dialog (if it appears):
+
<vip>interface testService supports rpcService
 +
end interface testService
  
[[Image:IISM_handlerMapping_5.png]]
+
class testService : testService
 +
end class testService</vip>
  
=== User Rights ===
+
In the implementation you will inherit from <vp>jsonRpcServiceSupport</vp> which will handle the parsing, writing, etc of JSON:
  
When the IIS run it will access various files as a certain Windows user, this user must have sufficient rights to the files.
+
<vip>implement testService
 +
    inherits jsonRpcServiceSupport
 +
    open core, json
 +
...
  
Which user is determined like this:
+
end implement testService</vip>
  
* If a '''Connect as..." user is set for the virtual directory or application then that user will access the files
+
You will register your procedures in the constructor:
* Otherwise (if '''Pass-through authentication is used):
 
** If annoymous access is allowed to the virtual directory/application then some IUSER_... is will access the files
 
** If some Windows authentification is used the windows user from the connection will will access the files
 
  
Anonymous/Windows access is controlled from the '''Authentification''' icon
+
<vip>clauses
 +
    new() :-
 +
        setProc_name("hello", hello),
 +
        setProc_name("calendar", calendar).</vip>
 +
 
 +
There are four kinds of procedures, with corresponding registration predicates:
 +
 
 +
* <vp>setProc_array</vp> for registering a procedure that uses positional arguments in an array
 +
* <vp>setProc_name</vp> for registering a procedure that uses named arguments in an object
 +
* <vp>setListener_array</vp> for registering a notification listener that uses positional arguments in an array
 +
* <vp>setListener_name </vp> for registering a notification listener that uses named arguments in an object
 +
 
 +
In the example there are two procedures "hello" and "calendar" that both uses named arguments.  The "hello" code looks like this:
 +
 
 +
<vip>constants
 +
    testService_error : jsonRpcError::serverErrorCode = jsonRpcError::serverError_last+1.
 +
    % grows up from "last", which is used for unspecified server error
 +
 
 +
predicates
 +
    hello : jsonProc_name.
 +
clauses
 +
    hello(_Context, ArgMap) = o(Result) :-
 +
        Result = jsonObject::new(),
 +
        Username = ArgMap:get_string("username"),
 +
        if "error" = Username then
 +
            raise_serviceError(testService_error, "'error' is an invalid user name")
 +
        elseif "userError" = Username then
 +
            exception::raise_user("'%Name%' is an invalid user name", [namedValue("Name", string(Username))])
 +
        elseif "definiteUserError" = Username then
 +
            exception::raise_definiteUser("'%Name%' is an invalid user name", [namedValue("Name", string(Username))])
 +
        else
 +
            Result:set_string("token", string::concat("Hello ", Username))
 +
        end if.</vip>
 +
 
 +
First the <vp>hello</vp> procedure creates a <vp>jsonObject</vp> for the <vp>Result</vp>.  Any kind of JSON value are legal as result, but <vp>hello</vp> is designed to return a JSON object with a <vp>"token"</vp> member.
 +
 
 +
Next it fetches the arguments from <vp>ArgMap</vp>.  It expects a string argument named <vp>"username"</vp>.
 +
 
 +
<vp>hello</vp> uses the value of <vp>Username</vp> to illustrate exception handling and success.
 +
 
 +
The <vp>"error"</vp> case illustrates how you can raise "service errors", that is errors that are specifically designed for the RPC service.  The "userError" and "definiteUserError" cases illustrates what will happen with other exceptions that occurs in the procedure call.
 +
 
 +
Notice that the client side code should be programmed to deal with errors.  On the server side you may of course also handle specific exceptions as in any other program, and perhaps you also want to log all errors somewhere.
 +
 
 +
If <vp>Username</vp> is not one of the error cases, we map "token" to a string in the <vp>Result</vp>.
 +
 
 +
{{Example|We recommend that your <vp>jsonProc</vp>'s only deal with argument extraction and result construction, and that you keep the real functionally in a separate business layer as illustrated by this code:
 +
<vip>predicates
 +
    hello : jsonProc_name.
 +
clauses
 +
    hello(_Context, ArgMap) = o(Result) :-
 +
        Result = jsonObject::new(),
 +
        Username = ArgMap:get_string("username"),
 +
        Token = businessLayer::hello(Username),  % actual functionality is in a separate business layer
 +
        Result:set_string("token", Token).</vip>}}
 +
 
 +
=== Stand-alone HTTP server ===
 +
 
 +
As illustrated by the <vp>jsonRpcService_httpApi</vp> program the <vp>testService</vp> (supporting the <vp>rpcService</vp>) can be used in a stand-alone HTTP server.  It does not require much code and it is quite simple to adjust the code to specific needs.  Nevertheless the complexity is quite large.
 +
 
 +
The server is based on Microsoft's [http://msdn.microsoft.com/en-us/library/windows/desktop/aa364510(v=vs.85).aspx HTTP Server API] which cooperates well with Microsoft Internet Information Services (IIS).  In fact, it seems that IIS is built on top of the HTTP Server API (or at least that both are built on top of some common base).
 +
 
 +
The HTTP Server api operates with the following entities:
 +
* Creator/Controller Process
 +
* Server Session
 +
* URL groups
 +
* Request queues
 +
* Worker Processes
 +
 
 +
The creator/controller process creates a server session, which defines URL groups and request queues.  The worker processes attach to the request queues and process the requests they receive.
 +
 
 +
In the example program the creator/controller process is also the only worker process.  And it is outside the scope of this tutorial to consider more complex scenarios.
 +
 
 +
The server session is the "owner"/"holder" of the URL groups and the request queues, which are the things we really need to consider.
 +
 
 +
An URL group defined by a set of URL patterns of the forms:
 +
:'''http:<nowiki />//<host>:<port>/<relativeURI>'''
 +
:'''https:<nowiki />//<host>:<port>/<relativeURI>'''
 +
 
 +
When a request arrives at the computer a matching mechanism will match it against the URL patterns in the active URL groups, the matching will either fail or determine an URL group that covers the URL.
 +
 
 +
Each URL group is attached to a request queue, and requests matched to a certain URL group is placed in the attached request queue. URL groups that are not attached to a request queue are inactive and does not participate in the matching.
 +
 
 +
Worker processes dequeue requests from the request queues and handle them.  A request queue can have several URL groups attached and each URL group can have several URL patterns.  A worker process dequeuing from a certain request queue will therefore receive requests that match some URL pattern in some URL group attached to that queue.
 +
 
 +
In the example server we create two request queues each having a single URL group attached, and each of these URL groups having one URL pattern. Effectively:
 +
* <vp>reqQueue_rpc</vp> receives requests of the form '''http:<nowiki />//localhost:5555/jsonrpc/<something>''', and
 +
* <vp>reqQueue_file</vp> receives requests of the form '''http:<nowiki />//localhost:5555/file/<something>''''
 +
 
 +
<vp>reqQueue_rpc</vp> is a <vp>requestQueue_rpc</vp> handles RPC requests by feeding them to our <vp>testService</vp>.
 +
 
 +
<vp>reqQueue_file</vp> is a <vp>requestQueue_file</vp> which uses <vp>fileMapper</vp> to map the requests to a file name and an associated <vp>ContentType</vp>, the <vp>requestQueue_file</vp> will then handle the transfer of the file.
 +
 
 +
==== Server configuration: netsh ====
 +
 
 +
The example runs smoothly because it uses port 5555 on localhost.  Normally you will however want to run applications on port 80 for http and 443 for https, and that will cause some security problems.
 +
 
 +
If you just change the port number and uses a public hostname you server program will get an 'Access is denied' exception when trying to add the URL patterns to the URL groups.
 +
 
 +
If you run the server "as Administrator" you will not get the access violation, but it is absolutely not advisable to run the program as administrator, because then it also have rights to do many other harmful things.
 +
 
 +
To make the program run in normal modes you will have to reserve the URL patterns in question for the user that run the server program.
 +
 
 +
This is done using the netsh.exe (net shell) program.  You will need to manipulate the http URL acl's.  For a detailed description see [http://msdn.microsoft.com/en-us/library/windows/desktop/cc307236(v=vs.85).aspx Netsh commands for HTTP].
 +
 
 +
{{Example|This will give an idea of what to do (in a cmd prompt):
 +
<source lang="text">>netsh http add urlacl http://www.something.com:80/file/ user=SMT\serviceuser
 +
>netsh http add urlacl http://www.something.com:80/jsonrpc/ user=SMT\serviceuser</source>
 +
'''Notice''' that the cmd prompt be run "as Administrator" to make modifications.
 +
If you just write "netsh" it will go into a prompt mode where change to http mode and then just write http commands
 +
<source lang="text">>netsh
 +
netsh>http
 +
netsh http>add urlacl http://www.something.com:80/file/ user=SMT\serviceuser
 +
netsh http>add urlacl http://www.something.com:80/jsonrpc/ user=SMT\serviceuser
 +
netsh http>exit</source>}}
 +
 
 +
It is possible to make the URL reservations from Visual Prolog, but in case of conflicts and problems it may be better to use the standard program netsh.
 +
 
 +
Notice that the example program is a console program.  In an larger scale context it would seem better to create a genuine windows service.  Visual Prolog has a template project for creating services, but that subject is outside the scope of this tutorial.
 +
 
 +
=== IISAPI plug-in ===
 +
 
 +
As illustrated by the <vp>jsonRpcService_isApi</vp> program the <vp>testService</vp> (supporting the <vp>rpcService</vp>) can also be used in an ISAPI dll, which can be used by the Internet Information Services to run the RPC requests.
 +
 
 +
<vp>jsonRpcService_isApi</vp> does not deal with file requests, because it is more natural to leave that to the IIS itself.  The dll code is actually rather trivial, the key issue is the IIS configuration.
 +
 
 +
The configuration of the IIS has been put into its own tutorial, see [[IIS configuration]].
 +
 
 +
=== Existing GUI program ===
 +
 
 +
In the '''webRPC''' example directory there is also a <vp>jsonRpcService_httpGUI</vp>, which illustrated how a GUI program can act like a stand-alone Web RPC Service.  You should notice that it is not recommended to let a real service program have a GUI, but it can be quite useful if you need to let other programs interoperate with the GUI program, i.e. in a normal user login session.  It can also be used as a "quick and dirty" way to expose already existing GUI code as a Web Service (perhaps as a test before creating the real service).
 +
 
 +
=== Multi-threading ===
 +
 
 +
No matter how you run the web service (stand-alone with or without GUI, or as ISAPI) you should notice that the HTTP requests run in parallel in several threads. So it is important to ensure that the request handling's are thread safe.  The <vp>requestQueue_rpc::synchronizer</vp> property can be used to synchronize the execution of the RPC's, the property is set to a predicate which will receive <vp>runnable</vp>s, when the runnable is executed the actual RPC call is performed.  The runnable's can for example be executed in a monitor ensuring that only one at the time is execute, or be put on a queue and executed sequentially from that or as <vp>jsonRpcService_httpGUI</vp> does be posted to the GUI queue and thus executed by the GUI thread.
 +
 
 +
=== Visual Prolog Client ===
 +
 
 +
This tutorial and the mentioned examples has focused on the Visual Prolog server side, using a web browser as client side. But sometimes a Visual Prolog program may need to access a web service as client.  The example <vp>jsonRpcClient</vp> illustrated how this can be done.  The method is quite similar to the one used in JavaScript:
 +
 
 +
* Create a <vp>jsonRpcRequest</vp> object, with method name and parameters
 +
* JSON "stringify" it using <vp>asString</vp>
 +
* POST it using an XMLHttpRequest object (an instance of <vp>xmlHTTP</vp> or <vp>xmlHTTP60</vp>)
 +
* JSON parse the response text  (using <vp>jsonObject::fromString</vp>)
 +
* Check for "error" member
 +
 
 +
In the example program you see a <vp>delayCall</vp>, whose purpose is to allow the GUI to be updated, without this call the GUI would not update until after the RPC have completed.  The call is however irrelevant for the RPC itself.
  
 
=== See also ===
 
=== See also ===
  
[[Web Services]]
+
* [http://www.json.org/ JSON]
 +
* [http://www.jsonrpc.org/specification JSON-RPC 2.0 Specification]
 +
* Microsoft's [http://msdn.microsoft.com/en-us/library/windows/desktop/aa364510(v=vs.85).aspx HTTP Server API]
  
 
[[Category:Tutorials]]
 
[[Category:Tutorials]]

Revision as of 11:54, 1 August 2014

This tutorial describes how to create a Web Application with a HTML/JavaScript front end that run in the users browser and a backend written in Visual Prolog.

The emphasis is on the Visual Prolog back-end, which implement a JSON-RPC Web Service. The HTML/JavaScript code is mainly included to make the example self-contained and to illustrate how font-end code can interact with a Visual Prolog web service.

It is described how the same service can:

  • Run as a stand-alone HTTP/HTTPS server
  • Run embedded as an ISAPI extension in Microsoft Information Services (IIS)

Notice that web services requires the commercial edition of Visual Prolog. Also notice that this tutorial will not consider SOAP at all.

The tutorial corresponds to the webRPC examples in the commercial edition (IDE: Help ->Install Examples...).

Overview

In this section we will try to describe the overall picture of the entire application and the way it works. What takes place is actually rather complex, but fortunately most of the complexity is handled automatically. An understanding of the overall process including knowledge of some of the complexity will however make it easier to understand what needs to be done and why.

The overall work method of the example application is as follows: The user browses (in a web browser) to the page http://<server>/file/test.htm (or http://<server>/file/calendar.htm), the server returns the corresponding file to the browser. The browser evaluates the contents of the file, which may result in additional requests for several cascading style scripts (css) files, JavaScript (js) files, images, etc. Furthermore, then html file contains embedded JavaScript code, which will make remote procedure calls (RPC) to the web server. The server will execute the corresponding procedures and return the result to the browser. Finally, the embedded JavaScript uses the returned data to update the HTML contents of the browser.

The communication between the client and server side is performed by means of the HTTP (or HTTPS) protocol. So on the server side we need an HTTP server, which can both return files and implement remote procedure calls. The program in jsonRpcService_httpApi runs the service as a stand alone HTTP server that both handles file requests and implements the remote procedure calls. The program in jsonRpcService_isApi implements the remote procedure calls as an ISAPI plug-in, the IIS (Microsoft Internet Information Services) is then configured to handle the file requests and to use the ISAPI for the remote procedure calls.

The stand alone HTTP server is implemented by means of Microsoft's HTTP Server API, which can share ports and URL's with the Internet Information Services (IIS), such that http://<server>/file, etc can go to our stand alone server while http://<server>/<something> can be handled by the IIS.

JSON & JSON-RPC

The remote procedure calls are implemented in accordance with the JSON-RPC 2.0 Specification specification, which implies that data is encoded in JSON (JavaScript Object Notation) format.

JSON syntax is a sub-set of JavaScript (hence the name): When evaluating a valid JSON text it will be come a corresponding JavaScript object. You should however always call a JSON parser instead of evaluating the text: evaluating text is a security risk, since the text could be "code" rather than "data".

It is not necessary to understand the JSON format in details, because on the client side you will work with JavaScript and on the server side you will work with an object/domain representation of JSON: parsing and writing is left to standard software.

When performing a JSON-RPC call the client side sends a JSON request object, and then server side will respond with a JSON response object, unless the request is a so called notification in which case there is no response.

Request object have four members:

  • jsonrpc A String specifying the version of the JSON-RPC protocol. MUST be exactly "2.0".
  • method A String containing the name of the method to be invoked.
  • params The method parameters (omitted if the method does not take parameters)
    • by-position: a JSON array, containing the values in the Server expected order.
    • by-name: a JSON object, with member names that match exactly, including case, to the method's expected parameters
  • id: an client created string or number used to identify the request. The id field is omitted for notifications.

A response object contains three members:

  • jsonrpc A String specifying the version of the JSON-RPC protocol. MUST be exactly "2.0".
  • id The value submitted by the client

The third member is one of these:

  • result (implying that the call succeeded) The result of the procedure call
  • error (implying that the call succeeded) An object describing the error which occurred

Client side

Example This code shows one way to make a procedure call on the client side:
function httpPOST(URL, method, params) {
    var ReqObj = { jsonrpc: "2.0", id: getId(), method: method, params: params };
    var Req = JSON.stringify(ReqObj);
    var xmlHttp = new XMLHttpRequest();
    xmlHttp.open("POST", URL, false);
    xmlHttp.setRequestHeader('Content-Type', 'application/json-rpc; charset=UTF-8');
    xmlHttp.send(Req);
    var resp = xmlHttp.responseText;
    var respObj = JSON.parse(resp);
    var error = respObj.error;
    if (error) {
        if (Data = error.data) {
            log("<pre>" + Data +"</pre>\n");
        }
        log("<p>error : "+ error.code +": " + error.message + "</p>\n");
    } else {
        // respObj.result contains the result
    }
}

In brief:

  • Create a request object
  • JSON "stringify" it
  • POST it using an XMLHttpRequest object
  • JSON parse the response text
  • Check for "error" member

There are many other ways to make client side code, typically you will use some standard JavaScript library packages, that handles errors and perform the calls asynchronously. calendar.htm uses such library code, but that is outside the scope of this tutorial.

Service Code

In Visual Prolog JSON is represented by the domain json::jsonValue:

domains
    jsonValue =
        n; % null
        f; % false
        t; % true
        r(real Number);
        s(string String);
        a(jsonValue* Array);
        o(jsonObject Object).

There are only 7 kinds of JSON values null, false, true, a number, a string, an array and an object.

An array is a sequence of JSON values represented as a list in Visual Prolog.

A JSON object represented by the jsonObject interface in Visual Prolog:

interface jsonObject supports mapM{string Key, jsonValue Value}
...
end interface jsonObject

The interface contains a number of convenience predicates/properties (which have been omitted above), but in essence a jsonObject is simply a map from Key's (names) to JSON values.

Example This code:
J = jsonObject::new(),
J:set_boolean("bbb", true),
J:set_string("sss", "The String"),
J:writeTo(stdio::outputStream),
stdio::nl,
jsonPrettyPrint::ppObject(stdio::outputStream, J)

will produce this output:

{"bbb":true,"sss":"The String"}
{
    "bbb" : true,
    "sss" : "The String"
}

writeTo produces a condensed JSON representation suitable interprocess communication, etc. jsonPrettyPrint can be used to pretty print the values (for example when debugging).

When creating an RPC service you will have to implement an object that supports the rpcService interface:

interface testService supports rpcService
end interface testService
 
class testService : testService
end class testService

In the implementation you will inherit from jsonRpcServiceSupport which will handle the parsing, writing, etc of JSON:

implement testService
    inherits jsonRpcServiceSupport
    open core, json
...
 
end implement testService

You will register your procedures in the constructor:

clauses
    new() :-
        setProc_name("hello", hello),
        setProc_name("calendar", calendar).
There are four kinds of procedures, with corresponding registration predicates:
  • setProc_array for registering a procedure that uses positional arguments in an array
  • setProc_name for registering a procedure that uses named arguments in an object
  • setListener_array for registering a notification listener that uses positional arguments in an array
  • setListener_name for registering a notification listener that uses named arguments in an object

In the example there are two procedures "hello" and "calendar" that both uses named arguments. The "hello" code looks like this:

constants
    testService_error : jsonRpcError::serverErrorCode = jsonRpcError::serverError_last+1.
    % grows up from "last", which is used for unspecified server error
 
predicates
    hello : jsonProc_name.
clauses
    hello(_Context, ArgMap) = o(Result) :-
        Result = jsonObject::new(),
        Username = ArgMap:get_string("username"),
        if "error" = Username then
            raise_serviceError(testService_error, "'error' is an invalid user name")
        elseif "userError" = Username then
            exception::raise_user("'%Name%' is an invalid user name", [namedValue("Name", string(Username))])
        elseif "definiteUserError" = Username then
            exception::raise_definiteUser("'%Name%' is an invalid user name", [namedValue("Name", string(Username))])
        else
            Result:set_string("token", string::concat("Hello ", Username))
        end if.

First the hello procedure creates a jsonObject for the Result. Any kind of JSON value are legal as result, but hello is designed to return a JSON object with a "token" member.

Next it fetches the arguments from ArgMap. It expects a string argument named "username".

hello uses the value of Username to illustrate exception handling and success.

The "error" case illustrates how you can raise "service errors", that is errors that are specifically designed for the RPC service. The "userError" and "definiteUserError" cases illustrates what will happen with other exceptions that occurs in the procedure call.

Notice that the client side code should be programmed to deal with errors. On the server side you may of course also handle specific exceptions as in any other program, and perhaps you also want to log all errors somewhere.

If Username is not one of the error cases, we map "token" to a string in the Result.

Example We recommend that your jsonProc's only deal with argument extraction and result construction, and that you keep the real functionally in a separate business layer as illustrated by this code:
predicates
    hello : jsonProc_name.
clauses
    hello(_Context, ArgMap) = o(Result) :-
        Result = jsonObject::new(),
        Username = ArgMap:get_string("username"),
        Token = businessLayer::hello(Username),  % actual functionality is in a separate business layer
        Result:set_string("token", Token).

Stand-alone HTTP server

As illustrated by the jsonRpcService_httpApi program the testService (supporting the rpcService) can be used in a stand-alone HTTP server. It does not require much code and it is quite simple to adjust the code to specific needs. Nevertheless the complexity is quite large.

The server is based on Microsoft's HTTP Server API which cooperates well with Microsoft Internet Information Services (IIS). In fact, it seems that IIS is built on top of the HTTP Server API (or at least that both are built on top of some common base).

The HTTP Server api operates with the following entities:

  • Creator/Controller Process
  • Server Session
  • URL groups
  • Request queues
  • Worker Processes

The creator/controller process creates a server session, which defines URL groups and request queues. The worker processes attach to the request queues and process the requests they receive.

In the example program the creator/controller process is also the only worker process. And it is outside the scope of this tutorial to consider more complex scenarios.

The server session is the "owner"/"holder" of the URL groups and the request queues, which are the things we really need to consider.

An URL group defined by a set of URL patterns of the forms:

http://<host>:<port>/<relativeURI>
https://<host>:<port>/<relativeURI>

When a request arrives at the computer a matching mechanism will match it against the URL patterns in the active URL groups, the matching will either fail or determine an URL group that covers the URL.

Each URL group is attached to a request queue, and requests matched to a certain URL group is placed in the attached request queue. URL groups that are not attached to a request queue are inactive and does not participate in the matching.

Worker processes dequeue requests from the request queues and handle them. A request queue can have several URL groups attached and each URL group can have several URL patterns. A worker process dequeuing from a certain request queue will therefore receive requests that match some URL pattern in some URL group attached to that queue.

In the example server we create two request queues each having a single URL group attached, and each of these URL groups having one URL pattern. Effectively:

  • reqQueue_rpc receives requests of the form http://localhost:5555/jsonrpc/<something>, and
  • reqQueue_file receives requests of the form http://localhost:5555/file/<something>'

reqQueue_rpc is a requestQueue_rpc handles RPC requests by feeding them to our testService.

reqQueue_file is a requestQueue_file which uses fileMapper to map the requests to a file name and an associated ContentType, the requestQueue_file will then handle the transfer of the file.

Server configuration: netsh

The example runs smoothly because it uses port 5555 on localhost. Normally you will however want to run applications on port 80 for http and 443 for https, and that will cause some security problems.

If you just change the port number and uses a public hostname you server program will get an 'Access is denied' exception when trying to add the URL patterns to the URL groups.

If you run the server "as Administrator" you will not get the access violation, but it is absolutely not advisable to run the program as administrator, because then it also have rights to do many other harmful things.

To make the program run in normal modes you will have to reserve the URL patterns in question for the user that run the server program.

This is done using the netsh.exe (net shell) program. You will need to manipulate the http URL acl's. For a detailed description see Netsh commands for HTTP.

Example This will give an idea of what to do (in a cmd prompt):
>netsh http add urlacl http://www.something.com:80/file/ user=SMT\serviceuser
>netsh http add urlacl http://www.something.com:80/jsonrpc/ user=SMT\serviceuser

Notice that the cmd prompt be run "as Administrator" to make modifications. If you just write "netsh" it will go into a prompt mode where change to http mode and then just write http commands

>netsh
netsh>http
netsh http>add urlacl http://www.something.com:80/file/ user=SMT\serviceuser
netsh http>add urlacl http://www.something.com:80/jsonrpc/ user=SMT\serviceuser
netsh http>exit

It is possible to make the URL reservations from Visual Prolog, but in case of conflicts and problems it may be better to use the standard program netsh.

Notice that the example program is a console program. In an larger scale context it would seem better to create a genuine windows service. Visual Prolog has a template project for creating services, but that subject is outside the scope of this tutorial.

IISAPI plug-in

As illustrated by the jsonRpcService_isApi program the testService (supporting the rpcService) can also be used in an ISAPI dll, which can be used by the Internet Information Services to run the RPC requests.

jsonRpcService_isApi does not deal with file requests, because it is more natural to leave that to the IIS itself. The dll code is actually rather trivial, the key issue is the IIS configuration.

The configuration of the IIS has been put into its own tutorial, see IIS configuration.

Existing GUI program

In the webRPC example directory there is also a jsonRpcService_httpGUI, which illustrated how a GUI program can act like a stand-alone Web RPC Service. You should notice that it is not recommended to let a real service program have a GUI, but it can be quite useful if you need to let other programs interoperate with the GUI program, i.e. in a normal user login session. It can also be used as a "quick and dirty" way to expose already existing GUI code as a Web Service (perhaps as a test before creating the real service).

Multi-threading

No matter how you run the web service (stand-alone with or without GUI, or as ISAPI) you should notice that the HTTP requests run in parallel in several threads. So it is important to ensure that the request handling's are thread safe. The requestQueue_rpc::synchronizer property can be used to synchronize the execution of the RPC's, the property is set to a predicate which will receive runnables, when the runnable is executed the actual RPC call is performed. The runnable's can for example be executed in a monitor ensuring that only one at the time is execute, or be put on a queue and executed sequentially from that or as jsonRpcService_httpGUI does be posted to the GUI queue and thus executed by the GUI thread.

Visual Prolog Client

This tutorial and the mentioned examples has focused on the Visual Prolog server side, using a web browser as client side. But sometimes a Visual Prolog program may need to access a web service as client. The example jsonRpcClient illustrated how this can be done. The method is quite similar to the one used in JavaScript:

  • Create a jsonRpcRequest object, with method name and parameters
  • JSON "stringify" it using asString
  • POST it using an XMLHttpRequest object (an instance of xmlHTTP or xmlHTTP60)
  • JSON parse the response text (using jsonObject::fromString)
  • Check for "error" member

In the example program you see a delayCall, whose purpose is to allow the GUI to be updated, without this call the GUI would not update until after the RPC have completed. The call is however irrelevant for the RPC itself.

See also