Biography
Vincent Adcock works for the VLA (Veterinary Laboratories Agency) . He works in the Centre for Epidemiology and Risk Research that performs epidemiological research and risk analysis for both the government and private external organisations. As a part of his most recent work, he has been heading the development of a series of web based mapping solutions each of which involve the use of SVG (Scalable Vector Graphics) in conjunction with ASP (Active Server Pages) .Net
SVG has immense potential for Internet delivered maps, combining high quality visualisation and good user interactivity. These advantages make it overwhelmingly superior to raster based image maps, but the advantages of SVG are less when compared to "Internet-GIS" solutions such as ArcIMS and MapXtreme. These are also vector-based and scalable, and being relatively mature solution, provide a completely integrated technology. The relative benefit of SVG mapping compared to these products lies in its flexibility, low cost and potential to adapt easily and quickly solutions developed for other members of the XML (Extensible mark-up language) "family".
Nevertheless, practical implementation of SVG mapping poses many challenges. The most difficult of these is for the delivery of real-time dynamically generated maps, wherein data from an active database is rendered in the browser. This is the requirement when Internet mapping goes beyond the display of static maps, being used as a tool for data integration, visualisation and part of real-time decision support systems.
We faced such a requirement when asked to develop a system to assist the ministry of agriculture ( DEFRA (Department for the Environment, Farming and Rural Affairs) ) in Great Britain in the control of a disease of cattle (bovine tuberculosis), wherein data from a number of sources needs to be combined and delivered to field veterinarians. This requires a large number of maps to be generated, along with charts, meta-data, and guidance for interpretation. The solution we successfully developed uses a number of elements, of which while all are well known within the SVG community, less commonly have been successfully combined into a single complex application.
The development of this application will be described in detail in our session. In brief, it consists of
Currently, the only XML -based technology lacking is the use of XSLT (Extensible Style sheet Transformation Language) , which reflect the difficulties implementing this solution in a Microsoft environment. However, XSLT offers such overwhelming advantages for greater user inter-operability, that this will be the focus of our development effort in the coming months.
Introduction
Overview of the Application Architecture
Web Application
Web Server Architecture
Web Page Architecture
Databases
Spida_v1_1 Database
TBHistoric Database
SVG
Performance Issues
Pre-compilation
Compression
JavaScript
Charting
XML Metadata
Results
Acknowledgements
Bibliography
The overall purpose of this paper is to provide an overview of the Spida v1.1 web application that was designed to display historical animal disease information, specifically Bovine Tuberculosis (BTB) in England and Wales.
More specifically the document aims to detail:
Further details of the scientific justification can be found in the complimentary paper, “Use of Scalable Vector Graphics For A Web-delivered Interactive Digital Atlas of Bovine Tuberculosis” which was published as a part of the proceedings of the GisVet Conference 2004, see bibliography for details.
Spida (SVG Presented Interactive Disease Atlas) v1.1 operates around a combination of two key servers. The first of these is a web server used to host the Spida v1.1 web application. In order to run the application the server has version 1.1 of the .Net Framework installed along side Windows 2000 Professional Edition and Internet Information Services (IIS) 5.0
The application is located in a physical directory on the server and is accessed through a virtual directory provided by IIS. In order to allow access to the application by users across Defranet the access permissions have been set to allow anonymous access. This means that users who do not have an account on the server are still able to access the application. If this option was not set only those who have a user account in IIS set up for that server could obtain access.
The second main server is the database server. This server is also running under Windows 2000 Professional and has SQL Server 7.0 installed acting as a data store and provider.
Within the SQL Server instance two databases have been created. The first of these, Spida_v1_1, is the database used to store all the application data. It contains information used in the rendering of data such as vertices for England and Wales at both county and parish level, Ordinance Survey (http://www.ordnancesurvey.co.uk/) grid data and centre points used for the placement of on map charts and figures. In addition, it also holds details of the user accounts for the application and the levels of access granted to each.
The second database, TBHistoric, contains the data that is rendered by the system. It is accessed through a series of views and returned to the application via cross-referencing implemented in a series of stored procedures located in the Spida_v1_1 database. It is worth noting that both databases share an additional user account that was created to allow the system access.
In this case, the account created is named Application and has been granted select permission to all the required views in the TBHistoric table and exec permission to all the required stored procedures in the Spida_v1_1 database.
Outside of these two servers, the TBHistoric database receives periodic updates from a number of external sources. These include the VETNET data, Census data, TB50 data, and BRO data. Each of these is stored in separate locations outside the scope of this document. However, a slightly more in-depth description will be provided in the Databases section of this paper.
The last component of the system is the client PC. The only requirements for it is are that it has Internet Explorer (IE) 5.0 or higher installed and also has the Adobe SVG Viewer 3.0 or higher installed.
In order to explain the structure of the application we have divided it into four distinct segments. These are as follows:
There are three key sections to the web server:
Internet Information Services (IIS)
.Net Framework 1.1
Web Application – Spida v1.1
When a request is sent to the web server, it is first authenticated by IIS before being allowed access to the application. As anonymous access has been set, the user will be moved onto the next phase.
Rather than accessing the CLR directly, the call from the client will reference an object in the web application. This will then be passed through the CLR and content will be returned to the client.
Depending on the object requested external data might be required. In this case, a call will be sent by the CLR out to an external data source specified in the application code and data will be retrieved. This will then be processed and the dynamic content will be returned to the user (Figure 2).
As stated earlier, the data sources for the application fall to two main databases (Figure 4):
Access to these databases is handled exclusively through stored procedures. The aim of this is to prevent potential security threats such as sql injection, which can happen when using direct or parameterised sql in the page code.
In order to reduce overheads data is retrieved from the database and held in memory resident datasets for the duration of the majority of rendering operations. This means that the number of open connections at any one time is kept to a minimum reducing the overall delay caused by network communication.
It would be possible for the datasets to be pre-cached before hand and updated on a fixed time basis, however due to memory restrictions on the server this is not presently an appropriate solution and is under consideration for future versions of the application.
Once the data has been retrieved files are generated and stored on the server. These are stored in a temporary directory created in the Application Root/Files directory and are kept so long as the user remains logged in. When they log out of the application the files and directory are removed. This is performed to minimize space consumption on the server.
The Spida_v1_1 database is divided into three distinct groups of tables, each group related to a different section of the application.
The first group are tables relating to the rendering functions of the application and hold both coordinate data as well as lookup tables with the available data types, legend text and display colours.
The second group of tables relate to the administration of the application. These control user access to the application and the recording and handling of issues reports.
The third groups cover the other miscellaneous tables that do not server any direct function. These are generally used in the creation of data and are not used directly by the application.
As stated above retrieval and insertion of data into the database is handled entirely by stored procedures. Each one of these serves a different purpose and many of them are used by different segments of the application.
Due to the complexity of the TBHistoric database it is not appropriate to cover the entire table contained within here.
The requirement was to produce choropleth maps, proportional symbol maps, grid maps, and dot maps.
In addition, there are some of hybrid types but they will not be covered in the scope of this paper.
Choropleth, dot, and proportional symbol maps can be displayed at both countrywide and countywide levels. A countrywide area is composed of a series of counties while a countywide area is composed of a series of parishes. Navigation from countrywide to countywide level was achieved by moving the mouse of the desired area and left clicking, this then caused the page to reload now displaying a countywide view of the selected area. This was achieved by wrapping the path objects in and a tag and using an xref to link back the parent page with differing query data.
Grid maps however are only displayed at a country wide level using a grid divided into 10km square areas. No second level is presently assigned although it may be possible to use a 3.163km square grid if wishing to view data at a countywide level. Due to the non-standard shapes of both counties and parishes it was decided to represent them as Path objects. Coordinates are assigned from an initial point using the M directive in the d attribute and then each of the following coordinates is plotted using the L directive. Finally the z directive is used to close the shape so that its fill attribute can be changed. It should be noted that in order for the coordinates to be drawn correctly the order in which they are to be drawn must be extracted from the original shape file.
Pie charts are also represented using paths; this is because a circle segmented by lines cannot have attributes of the individual segments changed. Using the arc directive A it is possible to draw an arc from one point to another forming the outside of the circle. Presently the girds are also represented using path although rect could also be used. Finally dots are represented using the circle tag.
Initially all the SVG files were created dynamically causing the files to be generated by the server each time they were requested. This produces an extended load time to account for the network overhead and the server processing time.
In order to avoid incurring the overhead every time a page is requested pre-compilation was employed. As the data used by the system is historic and therefore static in nature it was possible to create the files the first time they were requested, keeping them resident on the physical storage of the server. In the case that it was dealing with dynamically changing data code could be put in place to delete the files and update them as required.
In order to successfully render detailed maps together with the associated information the volumes of data to be transmitted to the client machine were so large that the response times, particularly over slower networks, were in the order of five minutes. This was deemed to be unacceptable and so a solution was sought that would reduce the times to within reasonable levels, approximately one minute at maximum. The initial method employed was to reduce the resolution of the spatial data; by reducing the number of vertices plotted, it was possible to greatly reduce the file size and consequently the time required for the SVG file to be displayed. However, this still did not achieve the desired sub minute times.
Further research identified the ability of SVG viewers to render SVG files even when they are in a binary compressed form without the need for decompression. Following searching on the Internet we were able to locate a binary compression library for .Net written in C# capable of performing GZip compression. Because of the common architecture of the framework, despite the fact that the library was compiled in C# as it is still a part of .Net it can be called normally as if it were written in VB. The resulting SVG files were found to be in most cases less than
vTempFileName.Append(_ThemeId)
vTempFileName.Append("_" & _VisualisationId)
vTempFileName.Append("_" & _AreaId)
vTempFileName.Append("_" & HttpContext.Current.Session("Greyscale"))
vTempFileName.Append(".svg")
vFileName.Append(_ThemeId)
vFileName.Append("_" & _VisualisationId)
vFileName.Append("_" & _AreaId)
vFileName.Append("_" & HttpContext.Current.Session("Greyscale"))
vFileName.Append(".svgz")
If Not File.Exists(vFilePath & vFileName.ToString) Then
Dim MyStreamWriter As New StreamWriter(vFilePath & vTempFileName.ToString)
MyStreamWriter.Write(Content)
MyStreamWriter.Close()
Dim MyZipStream As New GZipOutputStream(File.Create(vFilePath & vFileName.ToString))
Dim MyFileStream As FileStream
MyFileStream = File.OpenRead(vFilePath & vTempFileName.ToString)
Dim vByte(MyFileStream.Length) As Byte
MyFileStream.Read(vByte, 0, CInt(MyFileStream.Length))
MyZipStream.Write(vByte, 0, vByte.Length)
MyZipStream.Close()
MyFileStream.Close()
If File.Exists(vFilePath & vTempFileName.ToString) Then
File.Delete(vFilePath & vTempFileName.ToString)
End If
End If
|
In order to accommodate the fact that objects may need to be changed dynamically, and therefore need to be capable of being uniquely referenced, the id attribute is no longer suitable for storing supplementary data. In order to accommodate this need an additional attribute was added, name. In order to display this additional data JavaScript functions had to be employed. A mouse over event was added to each of the objects with a listener placed in the web page where the SVG object would be embedded. When the event handler is fired, it picks up the attributes of the object and displays them on screen. A second mouse out event was also added to hide the data when the object was no longer selected.
The non-spatial data associated with each object spans a number of years. In order to display this data initially separate maps were required. To eliminate the need for loading additional pages JavaScript was employed to allow navigation between years. In order to accommodate this function it was necessary to include all the data relating to the currently selected spatial areas for the entire time period. Initially replicating the spatial objects approached this and simply using JavaScript to turn their visibility on and off, this proved to be inefficient as it drastically increases the file size. In order to resolve this problem a different approach was required.
It was determined that it would be more appropriate to define the spatial objects once and then use the JavaScript to adjust their attributes accordingly. Text objects were chosen as a suitable data container. The text objects are grouped into years inside a g object identified. When the JavaScript is called it receives a string value for the year from the main page and locates the corresponding g object then using a loop updates the attributes of the spatial objects to those contained in the corresponding text object. The final result of this was a near instantaneous change to the visually represented spatial data.
function initSvg(evt) {
SvgDoc=evt.target.parentNode;
};
function setYear(vYear) {
var vValues;
var vArray;
var vRow;
var vObject = SvgDoc.getElementById(vYear);
vValues = vObject.firstChild.data;
vArray = vValues.split('_');
for (i=0; i < vArray.length - 1; i++) {
vRow = vArray[i].split('~');
var vTempObject = SvgDoc.getElementById(vRow[0]);
TempObject.setAttribute('name',vRow[1]);
vTempObject.setAttribute('fill',vRow[2]);
}
};
function data_default(id) {
alert(id)
}
if(window.data==null) {
window.data=data_default;
};
|
Another supplementary benefit of development environment was identified when addressing the additional requirement of producing chart-based summaries of the attribute data; the preferred solution was to make use of the capabilities of SVG to draw them. The initial forays into producing these were unsuccessful and following some research a suitable proprietary solution was found.
GraPL (See http://www.grapl.com/) is a charting tool that can produce various forms of chart in SVG format. Following discussion with the developers at GraPL it was possible to secure a working C# version of the core libraries running under version 1.1 of the .Net Framework. By integrating this into Spida it became possible to produce the charts required dynamically in a similar fashion to the maps.
Furthermore, as GraPL’s output was also in SVG it was possible to employ the same compression routine to reduce the file sizes and avoid an increase in load times.
Dim MyChart As New GraPL()
…
With MyChart
.[New](0, 0, 400, 400)
.FrameStyle = "Wiped,Boxed"
.Style = "Relative,Forcezero,Stacked,Redraw"
.Margins = New Double() {50, 50, 50, 50}
.Heading = vTitle
.XLabels = vGroups_Y
.Axes = "Black,1,Solid"
.XCaption = "Years"
.YCaption = vYCaption
.XStyle = "Angled"
.YStyle = "Atend"
.CategoriseBy = vCategories
.CategoriseInto = vGroups_C
.Colours = vColours
.Patterns = "Solid"
.GroupBy = vYears
.GroupInto = vGroups_Y
.KeyFont = "times new roman, 8"
.Barchart(vData)
End With
Return CreateFile(Constants.SVGType.Thumbnail_Chart, MyChart.RenderSVG(True))
|
In order to provide some idea of where the data was sourced from XML based metadata was included. The information itself is stored in a series of XML files and then processed by the Framework’s built in XSLT processor when requested by the user. The XSL file included alters the format suitably and the resulting output is displayed on screen.
Examples of the maps and their associated SVG outputted by the completed application are shown below.
Choropleth Map:
Choropleth Map SVG:
<?xml version='1.0'?>
<svg onload='initSvg(evt)' xmlns='http://www.w3.org/2000/svg' xmlns:xlink = 'http://www.w3.org/1999/xlink' viewBox='134148 1182444 111894 106482' preserveAspectRatio='yes'>
<script>
var SvgDoc=null;
function initSvg(evt) {
SvgDoc=evt.target.parentNode;
};
function setYear(vYear) {
var vValues;
var vArray;
var vRow;
var vObject = SvgDoc.getElementById(vYear);
vValues = vObject.firstChild.data;
vArray = vValues.split('_');
for (i=0; i < vArray.length - 1; i++) {
vRow = vArray[i].split('~');
var vTempObject = SvgDoc.getElementById(vRow[0]);
vTempObject.setAttribute('name',vRow[1]);
vTempObject.setAttribute('fill',vRow[2]);
}
};
function data_default(id) {
alert(id)
}
if(window.data==null) {
window.data=data_default;
};
</script>
<a xlink:href=''>
<path d='M177271,1266401
…
L177271,1266401z' fill='#ffa500' name="BUDOCK | 2" stroke='black' stroke-width='1' id='7506' onmouseover='data(evt.target.getAttribute("name"));' onmouseout='data(" ");'>
<set attributeName='fill' attribueType='CSS' to='#87cefa' begin='mouseover' end='mouseout'/>
</path>
</a>
|
Proportional Symbol (Pie Charts) Map:
Proportional Symbol (Pie Charts) Map SVG:
<g id='Data_1987' visibility='visible'>
<circle cx='465789' cy='1126861' r='3370.99931231621' fill='#ffa500' stroke='black' stroke-width='1' id='11874_1987' name='Unconfirmed | 2/2' onmouseover='data(evt.target.getAttribute("name"))' onmouseout='data(" ")'><set attributeName='fill' attribueType='CSS' to='LightSkyBlue' begin='mouseover' end='mouseout'/></circle>
<path d='M190093,1235686 L170437,1235686 A19656.1348276724,19656.1348276724 0 0,1 203335,1221160 Z' fill='#ff0000' stroke='black' stroke-width='1' id='11879_1987' name='Confirmed | 25/68' onmouseover='data(evt.target.getAttribute("name"))' onmouseout='data(" ")'><set attributeName='fill' attribueType='CSS' to='LightSkyBlue' begin='mouseover' end='mouseout'/></path>
…
</g>
|
The author is grateful to DEFRA for funding this project (SB4013) and allowing for this application to be built.
Dr Peter Durr for writing the project and leading it as well as his guidance in writing this paper.
Dr Nigel Tait and Mr Ross McDonald for their assistance in obtaining spatial data from ArcGIS and other sources.
Mr Julian Greene for his work on the attribute database and the production of suitable views into the data.
Mr Stuart Eastland for his support in the initial stages of the development and guidance.
Adcock et al (2004)
GisVet Proceedings
XHTML rendition created by gcapaper Web Publisher v2.0, © 2001-3 Schema Software Inc.