Home Pagelogo
Geographical indexSearching

The DHP; How we make it

On this page, you will be able to read more about the way these pages are set up and what was needed to reach the current state of the `Home of the Netherlands'.

Contents

Hardware & Software

Hardware

The information you are now reading and all other pages from our server are coming to you from a Silicon Graphics Challenge-S workstation.

The Challenge has a 200 Mhz R4400 CPU, contains 64 MB of memory, 3 GB of harddisk, and two ethernet cards. With this machine, we are able to make sure that the Dutch Home Page will be accessible all the time (unless something funny happens to our network). The machine runs IRIX 5.3 with the Indigo Magic desktop. A joy to use.....

Software

We are using the Netsite Communications Server version 1.1 from Netscape Communications which is very easy to install and maintain. All setup actions and maintenance are done through your favorite WWW-browser.

v0.0: One Map Version

 The date of birth of the Dutch Home Page lies around November 1993. At that time, about 10 World-Wide-Web (WWW) servers (mostly universities) were active in the Netherlands. It seemed like a nice idea to keep track of the Dutch WWW-activities on a map of the country (as several other countries were doing by that time as well). This first map started to fill up during 1994 and by the second half of that year, measures had to be taken to keep the map functional. The first measure was to reduce the character size on the map.

v0.1: Four Maps Version

 By the beginning of 1995, the number of sites had grown to such a number that, if we wanted to keep the geographically oriented interface, the map had to be split up into smaller parts or into several maps showing sites within the same category. The first option was choosen and the main map was split up into four parts.

v0.2: Four Maps with City Maps

 From that time, the Internet and WWW has developed extremely rapid. This is reflected in the number of new sites which applied for a place on the map. One final measure was taken to enlarge the capacity of the page by setting up seperate pages for the four main cities Amsterdam, Rotterdam, Den Haag, and Utrecht. But the page's implementation could not grow any further.

V1.0: First Automated Version

 After three generations of pages, the demand for an automized way of drawing maps reached the level that we couldn't avoid it anymore; with three new sites a day, which takes about one hour to draw, and with again flooding pages, we had to take the big step to redefine the set-up of the pages.

The Requirements for a New Set-Up

 We decided to redesign the pages with the following targets:
  • the references to sites in all pages shall be produced fully automically from a database with the site-information;
  • everywhere a site is referenced, the language(s) available on that site will be displayed. Because many foreign users, which do not read Dutch, and Dutch users with poor knowledge of English, we want to distinguish between languages. We keep track of Dutch, English, and German;
  • to be able to draw more sites, the number of regions in which the country is split-up must be rather large;
  • there is a conflict between the beauty of satellite background images, with high resolution, and the data-size of the images. As well people with low-bandwidth communication, as people with high-bandwidth communication must be served with the best quality maps; and
  • to enable users to search for certain types of sites, selections of sites are put together on country maps. The selections are made on subject. For instance, Libraries, Educational, or Companies.
Very prestigious targets, we know.

Implementation

 There are many obstacles to overcome when switching from manual to automatic production of pages. Currently, about 15,000 lines of C-code are needed to produce the pages, and 3,500 for the database management.

Graphics Tools

 We decided to draw the sites on the background of a satellite image. A click on some location on the small image on the main page, is linked to a page which displays the sites in that region.
Unfortunatelly, some regions are very crowded with sites, other are near to empty. Some cities, especially Amsterdam, do contain so many sites that they would never fit on any region-map. Those sites have their own page, just displaying a long list of all sites in that specific city.

Happily, just before we started with the development of these pages, we found the tool gd1.1.1. This library is capable of manipulating gif images. This library was extended with modules to

  • enlarge and reduce the size of images avoiding the appearance of squares. The current source-satellite image is only 475x550 pixels in size, which is too small to be used as background;
  • write text in a proportional font. The gd library was only capable of 2 non-proportional fonts, which makes the texts written on the image too large. Gd is extended with a proportional font which resembles Helvetica Bold 12pt;
  • manage transparency. This had to be improved, because gd does not take the existence of colors dedicated to transparency into account when allocating colors; and
  • reduction the number of colors in an image. The best way to reduce the data-size of an image is by reducing the number of colors used in it. By setting a good threshold, the data-size of an image is often about halved!
Maybe, these extentions can be included in some next release of gd.

Drawing Images

 The hardest problem is drawing the sites and city-names on the images. As any good Computer-Scientist can tell you, this is an NP-complete algorithm. The implementation is not perfect yet, but basically works as follows.

At first, the location of the dots, representing the exact location of a city, and the border symbols are occupied. One city- or site-name should write over it. Then all sites are drawn, one after the other.

When a site is drawn, it tries to take one of the many predefined shapes. A shape is

  • up- or down-wards,
  • is left-aligned or right-aligned,
  • can have a stock (vertical distance between city's dot and first site),
  • permit some skips between the names of sites, when it conflicts with the space occupied by other image elements like other cities of border elements, and
  • start from a point horizonal from the cities dot, left or right.
As first try, the algorithm tries to take some of the predefined shapes. When an other city or site or border-symbol is already on the place where the shape should be, the shape is refused, and the next shape is tried.

If this does not result in a shape for the city, the selection is tried a bit harder: for each shape, a few conflicts with neighbour cities are tolerated. These neighbours are pushed softly, one after the other, to select one of the other predefined shapes. If they all succeed, the new shapes are taken. If one of the neighbours fails, they are all put back to the shape they had before the pushing started.

If this still does not produce a shape for the city, the neighbours are pushed harder: all neighbours have to try to form a new shape, even if they have to push some of their neighbours to achieve that. Only when this process is fully successful, the new situation is taken, otherwise the shape is refused.
As last resort, the city on the map of this region with the most sites is removed from the map to its own page. All cities have to start all over again to decide about their shape.

This algorithm produces rather quickly an acceptable result, although manual drawing by humans produces a much better result.

Pages

We distinguish a few types of pages:
The Home Page
is the main entrance point. This page connects you to the searching methods we offer you and the national information of other suppliers.
the Site List
is a textual browser. The main page shows
  • a full description of all new sites,
  • a reference to the 26 pages with full descriptions of all sites, sorted by starting letter of the city-name, and
  • a list of all references to sites. A very compact list of all knowledge in the database.
Region Pages
each display the sites in a certain part of the country. The regions overlap a little, and you can easily change from one region to the next. Each page also contains a list descriptions of the sites in that region.
Category Pages
display the location of sites which have some common interest on a map of the whole country. When the sites displayed in one category can be distiguished in more detail, they are represented by different symbols.
City Pages
are used as overflow for the region- and category-maps. We only have a few maps, hence welcome contributions for this.
All pages are in English only, because this is considered the international language.

Editions

 Editions had to be introduced to avoid problems with caching displayers.
For instance, NetScape caches the gif-files of regions, but not the related map-files (those are not transmitted at all). When new pages are produced, both the gifs and the maps are updated. Without precaution, the old region gif-file, which was cached on the user's machine, is used with the new map-file.

To ensure that gif and map are used in the correct combination, editions were introduced: the paths to gif and map contain the date of production. The gif is reloaded when the page which contains it is reloaded, and the old maps stays available for a few weeks.

Transition between editions are not disturbing the use of the DHP: without editions, there could appear some problems during the update of the pages.

For the DHP managers, it is easy to return to a former situation, when the newest edition shows problems.

Background Images

The satellite image was supplied by the Dutch National Aerospace Laboratories (NLR), but originally produced by the European Space Association (ESA). The image currently in use is only 475x550 pixels, 256 colors, but we have access to a false-color image of 13600x8800 pixels 24bits. The detail on that image is increadible, but may cause the data-size of the region-images to pass acceptable limits. More work has to be done to handle these 400MB of image-data.

Next to the satellite image, a drawn image of the country is used. This map is useful when the whole country has to be displayed on large: the satellite image would be too large.

The drawn-image sometimes has insufficient space to draw all sites. For this, a drawn map with an enlarge area is available.

Information

We don't have to write any program if no data is going in and no data is coming out. So the data, the information, plays the key-role in all projects, certainly in this one.

For this upgrade, we had to collect extra data from all sites already registered in the former verion of these pages. In addition to the information we had, all sites had to be checked for language and classified for categories of interest.

v2.0: First Full Redesign

The continuously growing number of sites stresses all limits of the capacity of pages. With this, also the amount of time spent to update the databases gets unacceptable for volunteers; the DHP screemed for a new construction with more capacity to ensure continuity.

Changed Requirements

 To improve the quality and capacity of the pages, the following requirements and targets were added:
  • The front-page will be split-up in three parts:
    • ways to look into the DHP information,
    • other resources for information about The Netherlands, and
    • sources of information about the rest of the World.
  • The layout of all pages, including the images used, will be designed by people with more graphical knowledge.
  • All pages will have a Dutch and an English version. The English version will only contain the sites which are not in Dutch only.
  • All sites will be revisited regularly to ensure that only high-quality sites are contained.
  • The available categories will be changed and reassigned.
  • The sites which are registered will be asked to add our logo to their main page.

New Implementation

 Major changes had to be made for the new implementation: about 90% of the code had to be rewritten to implement the new layout and to seperate English from Dutch language.

Currently under development; when more details are known, this chapter will be extented.

v2.1: New Search Facilities

(11 Feb 1997) Major software changes took place in version 2.1 to be able to provide extended search possibilities. More possibilities will follow.

The number of sites grows exponentially, which means a continuous effort to give all te sites a place on maps and in lists. Although this is automated, manual intervention is often required. For this version we decided to limit the geographical display to city-names only (so no sites on the maps anymore). The geographical maps are considerably reduced in size.

List of changes:

  • Two new ways to search the DHP: simple and step-wise (Dutch only).
  • No sites displayed on geographical maps.
  • Smaller geographical maps.
  • Other format of displaying a site's description.
  • Improvements on site-production to be able to produce the site's 400 pages in a better controled way.

v2.2: Speed-up and Category changes

(10 March 1997) The code is now using shared-memory to improve speed.

List of changes:

  • Faster access to the database, which means faster searching.
  • Improved search-facility messages.
  • Redesign of category pages.

v2.3: New category structure

(28 August 1997) The main change is a full redesign of the categories. All sites got a visit to establish their purpose.

List of changes:

  • Increased the number of categories from 98 to 225.
  • Changed the satimg interface not to use multipart-messages anymore: Micro IE was not able to display that.
  • Sites in lists ordered on full-name, and on maps in abbreviated form. Before, it always was in abbreviated order, even when the full-name was displayed.

v2.4: Geographical pages remade

The pages with sites geographically displayed were overfull, such that they needed to be split-up.

List of changes:

  • Breakup of the satellite image in three levels with different resolution. The regions are made when required, to cover the display of all cities.
  • Pages which show the regions made for each level.
  • Extensive use of USEMAPs.

Credits

 Currently, the DHP is being maintained and developed by the `Stichting The Dutch Home Page', which depends on the support of a few companies.