Friday, November 4, 2011

Architecture Part I


Good Lord!  Is that the time?

I have neglected this blog shamefully for most of a year.  At least, it would be shameful if anybody were reading it, but I don't suppose anybody is.  At any rate, here we go again.

How will this thing all fit together?

The application is intended to take data stored on a back-end (Domino server), present it in a front-end (browser), and handle communication between the two. First the data for the map, and later the game state, needs to be loaded up, interpreted, and rendered in the browser. Then any changes made in the browser need to be fed back to the server. We have two platforms, and two communication channels. Let me deal with each in turn. This will take several posts - bear with me.

The Domino server provides a number of services that I need (plus a bunch of others that I am not using at this stage).

First, there is storage. All of the data are stored in a single Notes .nsf database file. This allows me to have multiple copies of the database in existence, and bring them into synch when I choose. It also allows me to take advantage of the flexible design structure inherent in Notes, as well as the various languages it supports. I am going to skate over the details of the .nsf file format here, and of the various design elements that it can contain. If you are familiar with it, well and good. If not, I trust I can remain intelligible anyway. The three things you need to know about at this stage are documents, views, and forms.

A Notes document is essentially a database record - a set of name-value pairs (fields), where the values can be any of several data types. Any of these fields can be multi-value - that is, a single field could contain the value "Red":"Green":"Blue", while the same field in another document might contain just "Purple". The set of fields on a particular document is not rigidly determined by a pre-existing schema, and need not necessarily be the same as the set in another document that is ostensibly of the same "form". .nsf files will happily scale to hold millions of records, but a few tens or hundreds of thousands will be quite adequate for my purposes. For the purposes of mapping alone, I have defined two document types - they are:

Map: has a map title (e.g. "Europe") and a list of variants available ( e.g. dates 410, 1618, 1805, 1914). It contains also a number of other parameters such as the dimensions of the map and the graphic to be used by the editor as a guide in building the map in detail. A map may be defined as having up to 100 rows and columns, for a maximum of ten thousand individual hexes. Row and column identifiers are two digit decimal numeric values, running from 00 to 99.

Hex: describes an individual map location. Originally, I had a single document containing all information concerning a single hex on a particular map. I later changed that scheme to have potentially two documents per hex. The reason for this is that a single map may now support a number of variants. I visualise these as being used for different historical periods, but of course other uses may be found later. Some aspects of the map - the shape of coastlines, the location of rivers, mountain ranges and so on, will not change appreciably over a few thousand years. One hex document describes these more-or-less permanent features - the base, or lithosphere. The second document describes the features that change over historical time, which I am calling the biosphere (I am playing a little fast and loose with the term, I know, but I have to call it something). The biosphere document records forests, marshes and human habitations. If I later represent roads and railroads, they will go here too.

A view is a collection of documents, defined by a selection formula as belonging together. It may be indexed on one or more keys, and will typically display a subset of the data contained in its member documents. All of the documents in a view need not be all of the same form or internal structure, although they often are. The main function of views in this application is to present a set of documents to the browser. When the browser calls for a particular map variant to be loaded, it first retrieves the details of the map itself, then the set of base hexes (10,000 of them for a 100×100 map), then the biosphere hexes (a substantially smaller number - maybe 20% of the base count). Each view shows the required information in JSON format, so that having been retrieved by an AJAX call the whole can be turned into a Javascript object with a single line of code. Domino actually has JSON built into it. A view can be defined to show certain data, and can be called on with a special parameter in the URL "&Outputformat=JSON". I can see that being really useful if you want to retrieve into Javascript a view built for other purposes, but the way it formats the JSON is necessarily somewhat generic - Domino has no way of knowing what I am going to do with the data. To be sure, I can turn the view content into a Javascript object, but its internal structure is not terribly convenient. Since my views exist for no purpose other than delivering data in JSON format to the browser, I can be a little more targeted, and roll my own. I will go into that in more detail another time.

A form is mostly used to define the user interface of a document. The UI is united at runtime with the data in a single document to produce an on-screen representation of the document for human beings to read. In this case I am not using it for that purpose, since what is represented on screen is an amalgam of hundreds, or thousands, of documents. I am using a form to lay out the entire application, defining what it will look like in the browser, and calling on code to fill in the details.

The last element I need on the server side is one or more languages for processing stuff - accepting orders, for example, or map edits, and executing them to change the state of what is stored. Notes has four languages built into it that can help with this.

Function Language is a simple declarative language rather like that found in spreadsheet functions. It is used for building view data out of stored data, and in Notes has a number of other applications for it that I won't touch on here. It is no good, though, for intricate processing (although you might be surprised how far it will take you even in that direction).

LotusScript is the scripting language that first appeared in Notes 4 in the mid-90s, It runs both in the Notes Client and the Domino server, and in the client can interact with and manipulate objects in the user interface as well as in the back end. It is object oriented (although not fully featured as an OO language) and uses a syntax much like Visual Basic. LS has been a workhorse of Notes development for many years, and although it obviously does not work in a browser, it remains convenient enough that I am using it at this stage for all of the server-side scripting. Alternatively, I could write all the server-side stuff in...

Java, which has become the mainstream language of choice for many programmers in the last decade or so. Notes developers have been slow to take up Java, for what seems to me one good reason. The Great Thing about Java, the Thing that got everybody all excited, and made them want to re-write everything in that language, is that you can write a program on any platform, and have it run without modification on any other platform that supports Java. No more porting, no more recompiling, no more platform-specific pragmas... sounds great, right? Well, yes, it does, but Notes already had that since the '80s. If you are writing on the Notes platform, your app will already run without modification on any platform that supports Notes. That means pretty much any operating system at all, on both client and server side. That is not to say that Java offers no advantage to the Notes developer, but the single advantage that made Java compelling to the rest of the world... we already had, ten years earlier.

Finally, content for the browser must be in HTML. In Notes there are basically four ways to generate HTML. You can:
1) hand-write it and pass it complete to the browser.
2) have server-side code generate it and pass it to the browser.
3) have Domino generate it automatically on the fly based on a WYSIWYG foundation.
4) pass data to the browser and process it there (using Javascript) into HTML.
In this application I am not using method 2. If HTML is being generated automatically based on a data set, the HTML is nearly always more verbose than the data. I prefer, then, to pass the data, and have the code run on the browser side to render it (method 4). This also allows me to parse the data in more than one way - once into HTML for display, and once (or more) into other formats for manipulation. I do, on the form opened in the browser, have a mixture of hand-coded ("pass-through") HTML to set out the major screen divisions, static text and so on, and elements for which Domino generates the HTML before serving it up. The latter is mostly used for fields, which are easier and more manageable to do in Domino Designer as regular Notes fields than to hand-code in HTML.