Last updated 3 June 2006

Introduction

It is very important that lawyers attempting to come to terms with the Internet have a good understanding of what the Internet is. Too many legal articles, and indeed judgements, are based upon a faulty understanding of the technical underpinings of the Internet.

Wikipedia defines the Internet as:

In the general sense, an internet (with a lowercase “i”, a shortened form of the original inter-network) is a computer network that connects several networks. As a proper noun, the Internet is the publicly available internationally interconnected system of computers (plus the information and services they provide to their users) that uses the TCP/IP suite of packet switching communications protocols. Thus, the largest internet is called simply “the” Internet. The art of connecting networks in this way is called internetworking.

In popular parlance, Internet often refers to the World Wide Web, electronic mail and online chat services operating on the Internet.

A highly recommended startingpoint in understanding the Internet is Roger Clarke’s Primer. Dr. Clarke defines the Internet as:

The Internet comprises a network of computer networks, which transmit messages to one another using a common set of communications protocols, or sets of operating rules. Networks comprise addressable devices or nodes (computers) connected by arcs (communication channels).

Technical Architecture

The key underlying technical aspects of the Internet are:

• A unified address space with IP addresses

• A standard packet-switched communications protocol (TCP/IP)

Other aspects are layered on top of this underlying transport mechanism. Common features are:

• Use of a client-server architecture

• Use of a human-readable addressing system called DNS - the Domain Name System

Packet Switched Network

The Internet, unlike a traditional phone system, is a packet switched network.

Read Roger Clarke’s explanation of this (in the form of a fairy tale) The Internet as a Postal Service: A fairy tale.
This is probably the most important single fact that must be understood before any position can be taken on Internet issues.

Fundamentally the Internet is what is known as a stupid network. A stupid network is where all the network does is shift packets - all other functionality is layered on top. Read the seminal paper (dating from 1997) The Rise of the Stupid Network.

Protocols

A protocol is a defined standard for communicating. In computer terms a protocol defines how one computer (or node on the network) communicated to another. Protocols are implemented in software. Different programs written by differing programmers can communicate if both programs implement a shared protocol correctly.

A protocol generally gains its authority from the network effect - that is, the more people who use the protocol, the more useful that protocol becomes. Sometimes large players in a marketplace can try with some success to influence or change a protocol, but this is the exceptions rather than the rule.

Examples of efforts to influence a standard in this way would be the ad-hoc changes made to the HTML standard by Netscape and Microsoft when their internet browsers where competing with each other. Now that Internet Explorer has over a 90% market share, the defacto HTML standard is set by it.

Internet Protocol Suite

The Internet has a considerable number of functions, such as the WWW, email, chat, P2P and so on. Underlying these obvious functions are a number of supporting functions.

Each of these functions has one or more protocols associated with it. The sum total of these protocols is refers to as the Internet Protocol Suite. Sometimes the Internet Protocol Suite is refered to by the names of the two most important protocols: TCP/IP

Examples of protocols in the Internet Protocol Suite and some of the software that implements them are:
• HTTP: Apache, Internet Explorer, Safari, Netscape, IIS
• SMTP: Outlook, Sendmail
• DNS: BIND
• TCP/IP: TCP/IP Stack
• Gnutella: BearShare, LimeWire, XoloX

IP Addresses

Generally speaking, every node on the Internet has a unique address. This address is a number between 0 and 4294967296, and is usually expressed as a number between 0.0.0.0 and 255.255.255.255. This unique identifier is called an IP Address.

IP Addresses are allocated by ICANN in blocks to 3 regional Network Information Centres, then to ISPs and others. There is a weak but real connection between an IP address and a real specific computer, and there are cases where an IP address has been traced back to a specific person. The rise of wireless, dynamic IP addressing and the ready availablility of software to forge an IP address means that this tracing is not always possible or accurate.

Due to historical reasons, IP addresses are not spread evenly throughout the world, but concentrated in the USA and other first world countries. Companies with Class A (more than 16 million addresses) include: Apple, Ford, Mercedes Benz and Stanford University. On the other hand, China has 22 million addresses for the whole country.

Solutions for this problem include sharing addresses (NAT), allocating addresses on a temporary basis from a pool (the way a normal dialup ISP operates) or creating more addresses (IPv6)

Domain Names

IP Addresses are not easy to remember. A large part of the widespread use of the internet in advertising and e-Commerce is due to the fact that it is possible to link a human readable name to a particular IP address. The system that does this is called the Domain Name System or DNS.

African, European and Asian countries already control the root of their trees. .uk. is maintained by www.nominet.org.uk, for example. (Well, more or less. It's a bit complicated. Technically ICANN claims the right to decide which organisation controls which country code. This has lead to small pacific islands finding that they have no control over their own country code, for example, becuase ICANN has given to a commercial US company the right to control and profit from it)

Individual organisations that maintain DNS servers (ISPs, large companies, universities etc) have links to the root DNS servers, which are controlled but not owned by ICANN. If an individual country was to go it alone, then they would have to convince those organisations to add their new alternative root DNS servers to the organisation's list of root servers. If ICANN didn't cooperate, then they would have to convince not only organisations in their own country, but organisations everywhere, worldwide. So far this has proved prohibitively difficult for people trying to set up alternative roots.

It's pure network effect that locks everyone in to ICANN. Everyone uses the DNS tree 'owned' by ICANN because, well, everyone uses it.

ICANN charges the countries for their .uk, .au etc country code top level domains (ccTLDs). It decides whether new top level domains are introduced, like .museum (eg icom.museum). Most of the criticism of this process so far has been that ICANN is neither open, transparent nor accountable.

Recently, ICANN started the process of breaking formal ties with the US Deparment of Commerce, which was generally welcomed. However at the same time ICANN was attempting to create a new top level domain for adult sites called .xxx.

The Deparment of Commerce has intervened on behalf of the Administration and declared that it would not relinquish control over the DNS system - read The Register - Bush Net Policy and now .xxx seemsdead in the water after conservative US lobbying against it.

ICANN should be small, limited to the maintenance of the existing top level domains and delegating the ccTLDs to the respective national governments and should not be dictated to by the US Adminsitration for political purposes and should not try to leverage itself into a supranational Internet governance body has it has been trying to do.

Resources

The Internet as a Postal Service: A fairy tale
Primer
Professor Graham Greenleaf’s Cyberspace: Structure, nature and forms of governance
How Stuff Works’ How Internet Infrastructure Works
How Stuff Works’ How Web Servers Work

Last Updated 28 July 2006

What is hypertext?

Hypertext is what you are using to find and navigate around this page and the information on it. Hypertext has been defined by various authors as `an approach to information management in which data is stored in a network of nodes connected by links' (Smith & Weiss) and as `screens of information not connected sequentially but connected by using associative links' (Stephen & Schreiber). At its simplest it is a technique of computerised cross-referencing, or what one writer called a 'generalised footnote'.

Other useful phrases used to describe hypertext are `non-linear text presentation' and `text navigation systems'.The essence of hypertext is that of a series of nodes of information, connected by links to form a network of information, as illustrated below. The result is that hypertext is non-sequential or non-linear: 'there is no single order that determines the sequence in which the text is to be read' (Jakob Nielsen Hypertext and Hypermedia Academic Press, San Diego, 1990 p3).

In the example below, it is possible for a user to navigate the nodes in the order A, D, C, E as well as in numerous other orders. Node F is equivalent to a table of contents.

A link is from an item of text within a node (an anchor) to another node (ie to the 'top' of that node), or to a target within the same node or within another node.

retrieval01

A small hypertext structure with 6 nodes and 13 links (following Nielsen 1990)

For more technical information, see the following definitions on Whatis.com, and browse some of the associated definitions connected to them (this is a very useful site for checking technical terms). Read as much as you need:

WhatIs 'hypertext'

WhatIs HTML (Hypertext Markup Language) - the standard for creating hypertext pages which can be read by web browsers

Distinguishing features of hypertext

What are the distinguishing features of hypertext? Here are some suggestions, influenced by Nielsen (1990), with some illustrations from AustLII and from the hypertext aspects of this Guide. It's incomplete, but a start:

• Non-linear navigation - Hypertext is a non-sequential (non-linear) means of accessing texts.
• Virtual structures - Hypertext presents different navigation options to the user ('virtual structures'), which may include some or all of the following:
• Associative or lateral structures - Links between text items determined by semantic relationships only. This is the essential type of hypertext link. (eg a link to the meaning of 'convention' in the Racial Discrimination Act 1975)
• Hierarchical structures - Tables of contents, menus and the like.
• Sequential structures - Where information does have a natural sequential order, it can be represented (eg the [Previous] and [Next] buttons to go back and forward between the sections of a statute).
• Overviews - Site maps and other structures to allow users to understand the structure of a whole site.
• 'Fisheye views' - Overviews where the current location is represented in most detail, with the detail receding as the context gets further away.
• Guided tours - Pre-defined useful paths through a site, for purposes such as teaching its use, or to demonstrate an argument.
• User control - Because of the previous features, hypertext allows the user to determine which nodes are accessed, and in which order, at the time of reading. In this way (without getting too post-modern about it) hypertext reduces the authority and control of the author/system designed, and increases that of the reader. But this should not be exaggerated - the user can still only browse within the structures enabled by the system designed (unless user-defined links are supported).
• Bearings - Hypertext structures provide various means to stop users getting lost ('bearings'). These may include:
• Landmarks - Pre-defined nodes which can be accessed at any time. These may be defined by the system (for example, the Netscape icon) or by the user (bookmarks), or may be particular to certain sets of web pages (eg the [Title] button at the top of this page).
• The user's backtrack path or history list - the ability to go back to where you came from (by one step or many), and to see that path represented visually (In Netscape, this is the 'Go' button on the menu bar).
• Current node identification - the ability to see the identity of the current node you are browsing (for example, the title at the top of a Netscape screen), and even better an indication of its context (such as where it sits in the hierarchy of nodes being browsed). (eg see the top line in yellow on any of Nielsen's web pages)
• User configurations - Hypertexts vary in the facilities they provide to increase user control, including:
• Bookmarks are user-created landmarks; creation and saving of bookmarks is now standard.
• User-added links between text items and nodes are one of the most powerful forms of user control (and essential to the ideas of Bush and Nelson discussed below). They are not yet standard on the web, but are available in many disk-based hypertext systems.
• User annotations to nodes / text (and links to and from them) are not yet standard either.
• Saved history lists in order to preserve a valuable path of research are often available.

Many of the items mentioned above are elements of the essentials of good hypertext design, discussed below.

There is now more general commercial interest in hypermedia, not hypertext per se, where hyper-structures are used to combine text with other media such as sound and image (moving and still). For the moment, the emphasis of legal applications is on hypertext, except in the litigation support context, where most aspects of hypermedia are very relevant.

History of hypertext

Vannevar Bush's 'memex'

Everyone traces the idea of hypertext back to Vannevar Bush's 1945 idea of the `memex`, which he said should operate by association, as the human mind does. He proposed a mechanical (non-digital) device - data on microfilm. See Vannevar Bush 'As We May Think' (Atlantic Monthly, 1945) - a fascinating article, worth a read.

Ted Nelson - 'hypertext' and Xanadu

The term `hypertext' was coined by Ted Nelson in the 1960s as a tern for 'non-sequential writing': see Elain Chain in 'Ted Nelson and Project Xanadu'.

In Ted Nelson ' A New Home for the Mind?' (Datamation Magazine, 1982), he described his `Xanadu' project. Parts of Xanadu were to include a hypertext corpus of the whole of English literature, which it involved users creating their own links or `trails' between documents, and the data was to be distributed across vast networks.

See Xanadu Australia for lots more, including the Xanadu archives, and material about the concept of 'transclusion'.

Ted Nelson's home page contains lots more, including a very detailed 1999 article surveying the history and purpose of Xanadu, his summary of which starts as follows:

Project Xanadu, the original hypertext project, is often misunderstood as an attempt to create the World Wide Web. It has always been much more ambitious, proposing an entire form of literature where links do not break as versions change; where documents may be closely compared side by side and closely annotated; where it is possible to see the origins of every quotation; and in which there is a valid copyright system-- a literary, legal and business arrangement-- for frictionless, non-negotiated quotation at any time and in any amount. The Web trivialized this original Xanadu model, vastly but incorrectly simplifying these problems to a world of fragile ever-breaking one-way links, with no recognition of change or copyright, and no support for multiple versions or principled re-use. Fonts and glitz, rather than content connective structure, prevail.

Much of Nelson's idea has this is now become reality - the World Wide Web (though Nelson is reported to have responded to the web as 'nice try').

Apple's Hypercard and the Hypertext '87 Conference (1987)

Englebart demonstrated a hypertext system in 1968, but there was little commercial interest for another decade, although there were various experimental systems (including Nelson's). High interest in hypertext only dates from Apple's 1987 release of Hypercard, and the Hypertext '87 Conference.

Jakob Nielsen reports on one of the early seminal conferences in Hypertext'2 Trip Report (York, U.K., 1989), including on the current state of Nelson's Xanadu.

Tim Berners-Lee and the World-Wide-Web

Meanwhile, the most significant practical development of hypertext was taking place at CERN, a particle physics laboratory in Switzerland, where Tim Berners-Lee (now Director of the World-Wide-Web Consortium - W3C) developed the basic ideas of the web from 1989.

See A Little History of the World Wide Web (1989-1995) at the W3C site, plus as much more About The World Wide Web as you can cope with.

Other references

Jakob Nielsen's Short History of Hypertext

Principles of hypertext design, and usability studies

Before considering how hypertext has been used with legal information, it is necessary to consider general aspects of what constitutes hypertext and good hypertext design.

Jakob Nielsen is a pioneer of hypertext design, and useit.com: Jakob Nielsen's Website is one of the best starting points on 'suable information technology'. Here are some key pieces by Nielsen on the design and usability of hypertext, and some general references:

Top Ten Mistakes in Web Design (Jakob Nielsen's Alertbox May 1996), subsequently revisited in 1999, and then updated by The Top Ten New Mistakes of Web Design (Jakob Nielsen's Alertbox, May 30, 1999) and Who commits them (Jakob Nielsen's Alertbox, May 16, 1999)

For contrast, read the Ten Good Deeds in Web Design (Jakob Nielsen's Alertbox, October 3, 1999)
'Why Frames Suck (Most of the Time)' - Jakob Nielsen's Alertbox (December 1996)
Report From a 1994 Web Usability Study, revisited in Changes in Web Usability Since 1994 (Jakob Nielsen's Alertbox for December 1, 1997)

Reviews of Neilsen's book Designing Web Usability: The Practice of Simplicity (2000) [the book is not on the web] make it clear that there are a lot of people out there who agree with Nielsen's approach.

• The Alertbox: Current Issues in Web Usability (All of Nielsen's bi-weekly columns)

For a discussion of how ideas similar to those of Nielsen are implemnented in a large-scale legal system, see Philip Chung, Daniel Austin, and Andrew Mowbray In defence of plain HTML for law: AustLII's approach to standards [1999] CompLRes 11 (Paper presented at AustLII's "Law Via The Internet '99" conference). Note in particular the Ten usability heuristics. (Also published as Chung P et al, 'A Defence of Plain HTML for Law:AustLII's Approach to Standards', 2000 (1).The Journal of Information, Law and Technology (JILT))

For a contrast to Neilsen's approach, see David Siegel Web Wonk - Tips for Writers and Designers - A typographer's approach to HTML, very different from that taken by Nielsen (and the antithesis of the approach taken by web sites such as AustLII).

For other references on hypertext design, see Elliot Chabot Thoughts on Web Design (1998) - by the designer of the US House of Representatives Internet Law Library; includes a lot of practical suggestions and links to tools and other valuable sites on web design, as well as comments on web design.

On Jakob Nielsen page there are 'Recommended other websites', an annotated index on web design

Where is the world-wide-web going? For some interesting (non-compulsory) discussion, see David Siegel The Balkanization of the Web (1996) - as well as being interesting this article contains some fascinating hypertext design features and Jakob Nielsen 'The Future of Hypertext' (1995)

Building large-scale legal hypertexts: Case studies

Lawyers were one of the first professions to make extensive use of hypertext, from the late 1980s, and they may have done so more in Australia than anywhere else. DiskROM's CDs of hypertext Australian legislation were available from the early 1990s. The Guide (UK) hypertext software was used for legal applications from the late 1980s.

Andrew Mowbray's 'Hype' component of the DataLex software was first used in a demonstration at the 1989 'Laws of Australia' inaugural conference - as a dial-up system using a text-based interface. The automation techniques used with Hype later became the basis of AustLII's hypertext markup.

There are three main approaches that can be taken to creating very large legal hypertext systems where rich interlinking of documents is desired:

Document creation: control: Where the creator of the legal source documents exercises a considerable degree of control over their creation according to standards that then allow automated creation of hypertexts. Examples are when offices of legislative drafting, and courts, mark up legislation or cases themselves using markup languages such as SGML or XML. An example is the Tasmanian government's legislation project.

Editorial control: Employ editorial staff to mark up by hand (as HTML, XML or SGML) legal documents that originate from other sources (including OPCs, courts, and authors). Commercial legal publishers (eg Butterworths Online) often do this. See also the LEXUM paper below concerning the Canadian Supreme Court.

Automated mark-up: In the absence of documents with mark-up at source, or the resources to employ large editorial teams, the only alternative is to automate the mark-up process by attempting to recongnise textual regularities in texts that you do not control, and automatically mark-up the data that is available to you.

In practice, few web publishers would use only one of these approaches. For example, AustLII encourages document creators to standardise their document creation as much as possible (while falling short of HTML or SGML tag insertion). Butterworths online employs some automated means as well as a high level of editorial control.

There is no available survey article which gives a history of hypertext in legal applications. However, three of the earliest large web-based systems, Cornell's Legal Information Institute (LII) created in 1992, the Australasian Legal Information Institute (AustLII), created in 1995 (but with a prior history back to 1985 as the DataLex Project), and Montreal's LEXUM, created in 1994 (now CanLII), are unusual in the extent to which they have documented their approach to building large-scale systems. They are all free access 'Public Legal Information Institutes'.

Cornell, AustLII and CanLII therefore provide three good case studies of the development of large-scale hypertext and retieval systems in a non-commercial environment.

Case study - Cornell LII

The following articles describe the current approach of the Cornell Legal Information Institute (LII):

A background article (with photos of Tom and Peter) from the Cornell Cronicle

Tom Bruce 'Tears Shed Over Peer Gynt's Onion: Some Thoughts on the Constitution of Public Legal Information Providers',2000 (2) The Journal of Information, Law and Technology (JILT)

Peter W. Martin 'Impermanent Boundaries - Imminent Challenges to Professional Identities and Institutional Competence' Keynote address given at the BILETA 2000 Conference, April 2000

Tom Bruce 'Public Legal Information: Focus and Future', 2000 (1) The Journal of Information, Law and Technology (JILT)

Peter Martin l Legal Information - A Strong Case for Free Content, An Illustration of How Difficult "Free" May Be to Define, Realize, and Sustain(2000) (Link appears to be broken)

Case study - AustLII

The Australasian Legal Information Institute is one of the world's largest free access law sites, and one of the most automated in its approach.

Background

For general overviews of AustLII, see the following:

G Greenleaf "How the Australasian Legal Information Institute (AustLII) achieved the free availability of legal information on the Internet"(Transcript of the 'Free the Law' Meeting, 8 November 1999, London) - This speech, with a linked 'slide show', gives a simple explanation of AustLII's philiosophy, technical basis and funding.

Tom McMahon Access to the Law in the Land of Oz (review of the 2nd AustLII Conference on Law via Internet '99) JILT - A Canadian view of AustLII and other Australian developments.

Automation of hypertexts

AustLII's web site now (1999) has over 28 M automatically created hypertext links between pages on the AustLII site, and automatic processing of the cases and legislation it receives into about 1 M separate web pages. It is therefore a leading example of the automated creation of large-scale legal hypertexts. It is still an unusual example of the degree of density in its internal hypertext linking.

The automation of hypertext links was addressed in the DataLex Project (1989-94), and this became the basis of the large-scale automation of hypertext creation on AustLII.

AustLII New Directions in Law via the Internet: The AustLII Papers' [1997] COL 2 explains the approach taken to hypertext automation - see particularly:

www.austlii.edu.au—1.html 'Rich' and automated hypertext - This is a functional description of why automated hypertext creation is necessary and what it attempts to achieve.

2.5. Hypertext markup (in Chaper 2 Managing large scale hypertext databases - AustLII's experience ) - This part describes in more technical detail the process of automated markup of links to legislation references on AustLII. A related process is used for markup of links to case references.

Austin D et al, 'Scalability of Web Resources for Law: AustLII's Technical Roadmap: Past, Present and Future', 2000 (1) The Journal of Information, Law and Technology (JILT). (Also available as Daniel Austin and Andrew Mowbray (AustLII) Inside AustLII: the scaleability of web resources for law [1999] CompLRes 7 (Paper presented at AustLII's "Law Via The Internet '99" conference)) - This paper gives the most extensive explanation of AustLII's technical basis and future directions.

Other articles concerning AustLII

Lord Justice Henry Brooke The BAILII Project (Address to the 15th Annual BILETA conference) 2000 (2) The Journal of Information, Law and Technology (JILT) - Discusses BAILII (the British and Irish Legal Information Institute) which was established by AustLII.

McCann et al, 'Reconciliation Online: Reflection and Possibilities', 2000 (2) The Journal of Information, Law and Technology (JILT) - Discusses AustLII's indigenous law resources

Chung P et al, 'A Defence of Plain HTML for Law:AustLII's Approach to Standards', 2000 (1).The Journal of Information, Law and Technology (JILT))

Case study - CanLII

CanLII is based at the University of Montreal in Québec, Canada. It is involved in a wide range of projects, some with its own software.

The following papers illustrate a variety of aspects of CanLII's approach:

Daniel Poulin, CanLII: How Law Societies and Academia Can Make Free Access to the Law a Reality, 2004 (1) The Journal of Information, Law and Technology (JILT).

Daniel Poulin (LexUM, University of Montreal, Canada) A neutral citation standard for case law [1999] CompLRes 38 (Paper presented at AustLII's "Law Via The Internet '99" conference) - see also material on Case Citation Standards.

Daniel Poulin, Alain Lavoie and Guy Huard 'Supreme Court of Canada's cases on the Internet via SGML' - (1997) E Law - Murdoch University Electronic Journal of Law, Vol 4, No 3 - this paper explains the advantages of SGML as a markup language for legal documents, and how its use is semi-automated in Canada by the LEXUM team. They also assume that 'chunking' (breaking source documents into manageable parts such as sections of Acts) and automated construction of tables of contents '(construction of tables of link end points') are easy. In fact, these are often the hardest parts of the conversion of poorly structured documents (which OPCs and Courts often supply). Automated methods of giving documents sensible names (to aid retrieval) is also not considered. However, this paper was written before many organisations had experience in this task.

Other approaches to large-scale hypertexts

There are few papers publshed on the creation of large-scale legal hypertexts. The following papers take a variety of approaches:

Bruce Mcleod (Spirit Consulting) 'Information overload equals pattern recognition': Providing relevant information to the Public from Government Funded Legal Web Sites [1999] CompLRes 31 (Paper presented at AustLII's "Law Via The Internet '99" conference) - Discusses innovations at Australia's other large free access legal web site, SCALEplus.

Trygve Harvold (Director, Lawdata, Norway) Commercial legal systems on the Web - Norway's experience [1997] CompLRes 7 (Paper presented at AustLII's "Law Via The Internet '97" conference) - The head of one of the longest running and most successful legal information systems explains how it balances commercial operations and free public access

Eric Schweighofer Part 8 'Hypertext' in 'The Revolution in Legal Information Retrieval or: The Empire Strikes Back' - 1999 (1) The Journal of Information, Law and Technology (JILT) - Provides a brief survey and mentions alternative approaches.

Val Haynes (NiuMedia Pacific Pty Ltd and University of Tasmania) Computerising the law of Papua New Guinea [1999] CompLRes 21 (Paper presented at AustLII's "Law Via The Internet '99" conference) - A very different perspective, the problems of computerising legislation and case law in a developing country.

Hypertext and Legislation systems

Point-in-time legislation

Arnold-Moore T et al, 'Connected to the Law: Tasmanian Legislation Using EnAct', 2000 (1). The Journal of Information, Law and Technology (JILT). (Also published as Jane Clemes (Department of Premier and Cabinet, Tasmania) and Timothy Arnold-Moore (Multimedia Database Systems, RMIT) Connected to the Law: Tasmanian legislation using EnAct [1999] CompLRes 6 (Paper presented at AustLII's "Law Via The Internet '99" conference)) The abstract of the paper states:

EnAct is a legislation drafting, management and delivery system that has been built to enable the Tasmanian Government to provide improved legislation information services to the community. EnAct provides the community with a facility that enables cost effective public access to reliable, up-to-date, searchable consolidated Tasmanian legislation. The 'point-in-time' capability allows users to search and browse the consolidated database as it was at any time since 1 February 1997. Tasmania achieved these goals by automating much of the legislative drafting and consolidation process. This paper discusses why the Tasmanian government implemented EnAct, the concepts behind EnAct, and the technology that makes all of this possible.

Ian Brightwell & Richard Dixon Hughes (DH4 Pty Ltd, NSW) Computerised legislative drafting and access to statutes [1999] CompLRes 8 (Paper presented at AustLII's "Law Via The Internet '99" conference)

Other papers concerning legislation systems

Bruce Mcleod (Spirit Networks, Canberra ) Information overload equals pattern recognition [1999] CompLRes 6 (Paper presented at AustLII's "Law Via The Internet '99" conference)

Michael Pronin (ParlInfo Manager, Parliament of Australia, Canberra) Towards 2001: The Australian Parliament's information on the internet (Paper presented at AustLII's " Law Via The Internet '97" conference)

Hypertext and Case law systems

Mark Burdack (NSW Attorney-General's Department) Automated publication of judicial decisions in New South Wales [1999] CompLRes 10 (Paper presented at AustLII's "Law Via The Internet '99" conference)

Bruce Kercher (Macquarie University) Where the future meets the past: Pre-1900 NSW case law on the web [1999] CompLRes 27 (Paper presented at AustLII's "Law Via The Internet '99" conference)

XML (Extensible Markup Language) - structured data on the web

The Extensible Markup Language (XML) is 'the universal format for structured documents and data on the Web'. Definition of XML (Extensible Markup Language) on Wikipedia.See also the related description of CSS (cascading style sheet), which is commonly implemented in XML. 'CSS gives more control over the appearance of a Web page to the page creator than to the browser designer or the viewer. ' Their use is explained by Jakob Nielsen in Effective Use of Style Sheets (Alertbox for July 1, 1997). W3C summarises XML in 10 Points (7, really) , the first of which is 'XML is a method for putting structured data in a text file'. (If you are still keen, the formal W3C also provides a further 4, 739 points)

Last Updated 28 July 2006

Introduction

What is text retrieval?

'Text retrieval' (also sometimes know as 'full text searching' or 'free text searching') refers to computerised systems which allow users to find particular combinations of words in large bodies of text whether or not those texts have any uniform structure.

Howard Turtle (see below) gives a more technical (and accurate) definition ' a retrieval system applies some matching function to the representation of the information needs and the representation of each document to determine which documents to retrieve'.

Definitions

For those unfamiliar with text retrieval, the following glossaries may be helpful, either as a starting point for reading, or to come back to later:

Search Engine Watch's Search Engine Glossary - simple but useful

I-Search Digest Search Engine Terms - A very comprehensive glossary with links to many of the resources it discusses. Includes brief analyses of all the main web search engines, as well as technical terms. If you read this glossary you will know just about every bit of jargon there is relating to web searching.

Basic principles - concordances and 'exact match models'

In the late 1950s, the basic technique underlying most computerised searching of large bodies of text was developed, variously know as the 'concordance', 'inverted file' or 'word occurrence index'. In summary, it involves the construction of an alphabetical list of every different word in each document in a set of documents, with the locations of each occurrence of that word recorded next to it. 'Searches' of the documents is in fact carried out over this 'concordance', not over the actual texts.

There seem to be few explanations of concordances or the underlying operation of search engines available on the web. The following reference, although it is old (1988), does explain the essentials of concordance-based searching:

Graham Greenleaf, Andrew Mowbray and David Lewis Retrieval Techniques (extract from Chapter 2 '`Basic principles of legal information retrieval' Australasian Computerised Legal Information Handbook Chapter 2, Butterworths, 1988).

The following points should be noted:

The relatonship between the concordance and the text files.

The relationship betwen the connectors and the concordance, and how the nature of the concordance determines what connectors (operators) are possible in a retrieval system.

The text retrieval system described in this book uses a relatively complex concordance five place concordance. Many text retrieval systems in use on the internet use far less complex concordances, sometimes only recording the number of a document in which a word is found, but not the location of the word within the document (much less the location of paragraphs or sentences).

Other references

For discussion of some basic concepts of text retrieval, and concepts used in indexing, see Dabney II. A Basic Model for Document Retrieval Systems' (in Dabney, cited below) [No longer on the web]

Howard Turtle 'Text retrieval in the legal world' - 5.1 Exact match models Artificial Intelligence & Law, (1995) Vol 3 Nos 1-2

Evaluation of text retrieval performance

How do you determine the effectiveness of a text retrieval system? Although Jon Bing says "The science of information retrieval lacks a comprehensive theoretical foundation', a lot of experimental effort has been made, with 'recall' and 'precision' as the two main measurements of quality of retrieval results.

Precision and recall

The effectiveness of full text retrieval is often measured in terms of precision and recall. The following table illustrates this. Assume that there are 100 documents in a collection being searched.The search retrieves 10 documents, only 4 of which (after inspection) are found to be relevant. However, after inspecting all the other 90 documents, we find that there are 2 relevant documents not found by the search.

'Precision' is the ratio of a to (a + b), or the proportion of relevant material retrieved to all material retrieved. Retrieval of irrelevant material (ie b) lowers this ratio. Here, the ratio is 4 / 10 = 40%

'Recall' is the ratio of a to (a + c), or the proportion of relevant material retrieved to all relevant material retrievable. Failure to retrieve relevant material (ie c) lowers this ratio. Here, the ratio i s 4 / 6 = 66%

Ideally, both precision and recall should be as close to 1 /1 ( ie 1 or 100%) as possible. We want to retrieve all relevant documents (perfect recall) and no irrelevant documents (perfect precision).

See Precision and recall (in Greenleaf Mowbray and Lewis cited above)

For a lengthier discussion, see Dabney IV. Key Ideas in the Evaluation of Document Retrieval Systems (in Dabney cited below)

Relationship between precision and recall - the retrieval problem

The principal problem in the science of information retrieval is caused by the simple fact that no information retrieval system delivers both perfect recall with perfect precision.

Many who have studied information retrieval have alleged that there is inverse relationship between precision and recall: the more we do to improve precision, recall drops; the more we do to increase recall, precision drops. Among others, this is asserted by Pao (reference below, p12), and suggested by Blair and Maron (reference below). Summarising 30 years of information retrieval research, Pao says `recall and precision are bound to lie within the 40 to 60 percent range'; but says that the inverse relationship is often noted but not proven.

This leads to two main issues:

How do we define the most desirable `balance' between precision and recall (if you can't have 100% of each)? . Does the answer depend on the type of documents, and the purpose of the search task?

What can we do to change retrieval systems so that both precision improves and recall improves?

`Recall devices' and `precision devices'

Bing classifies the search features of text retrieval systems as `recall devices' or `precision devices' (Jon Bing in an extract in LAWS 4609 Course Materials 1996 #3 p37 at the Law Reserve Desk) depending on whether their use enhance precision or recall. For example, truncations and thesaurii enhance recall, as does use of synonyms (and the OR connector), whereas use of the AND connector, and limiting a search over specified databases or parts of databases (eg headnotes) improves precision.

Further references

The following materials are not available on the internet but provide valuable discussion:

Howard Turtle 'Text retrieval in the legal world' - 3. Evaluation in IR Artificial Intelligence & Law, (1995) Vol 3 Nos 1-2 (see LAWS 4609 Course Materials 1996 #3 p46 at the Law Reserve Desk).

Miranda Pao `Evaluation and Measurement' extracts from Chapter 11, Concepts of Information Retrieval, Libraries Unlimited, Colorado, 1989 (see LAWS 4609 Course Materials 1996 #3 p34 at the Law Reserve Desk)

Arguments about text retrieval effectiveness in law

Blair and Maron's experiment

D Blair and M E Maron `An evaluation of retrieval effectiveness for a full-text document retrieval system' (1985) 28(3) Communications of the ACM 289 is the most famous and controversial article in the field of text retrieval and law. This is simply because if is one of very few large-scall empirical studies in text retrieval, and just about the only one related to legal information.

For a brief summary see Blair and Maron's experiment (in Greenleaf Mowbray and Lewis cited above)

for a brief review see R Crawford - Review of Blair and Maron 'An evaluation of retrieval effectiveness for a full-text document-retrieval system' - (from Computing Reviews)

For a lengthy description and critique see Daniel Dabney VII. The Experiment of Blair and Maron (in Dabney cited below) [No longer on the web]

Blair and Maron attacked conventional wisdom, by arguing that recall was much lower than the 75% recall that users of this system (lawyers) demanded and though they were getting (because that is when they stopped searching). They were in fact only getting 20% recall, but with 75% precision. This gives some support for the alleged inverse precision/recall ratio.

Some additional comments on the experiment:

It is one of few experiments in a very large real life legal database (40,000 docts, 350,000 pages) and shows the care and expense involved in serious information retrieval experiments (which is why there are relatively few).

Some aspects of the experiment are questionable. Relevance [both A and C] was determined by the users themselves (blind controls on samples) ( see p29). To determine recall they couldn't test relevance of all non-retrieved documents, so they only tested samples; It is unclear whether they only compared what was retrieved from the samples with what wasn't (p15), but whatever they did, this approach may have been flawed because (i) their `sample frames' that were tested were those `rich in relevant documents', which must skew the result against recall, if searches are over the whole database; and (ii) if they only compared part of a database they are using 'unrealistically small sample sizes', like others they criticise. However, they insist that their experiment 'gives a maximum value for recall for each result' (p15) - but we might question whether in fact it minimises recall.

Were the searches a fair test? They seem to assume that the lawyers / paralegals were being rational (from experience) in keeping their queries as they did. The searchers often used 5 concept terms connected by AND, because they thought that to broaden the search would risk `output overload' . However, if this is how they searched, then unless they used synonyms extensively, recall must be low. Blair and Maron's defence of the searchers used is essentially that these were the best and most experienced searchers available - if they can't use free text retrieval, who could?

They assert that recall also `decreases as the size of the database increases' (p31), but don't explain why. They refer to `output overload' as a `frequent problem of full-text retrieval systems' (though that is the exact reverse of their results!!) and that users counter it by reducing the retrieved set by using the AND operator to limit the number of documents retrieved.

They concludethat the assumptions underlying full text retrieval are wrong., and that earlier studies supporting the effectiveness of full text retrieval were wrong because they were not based on large databases.

It is possible that Blair & Maron's experiemnt is mainly significant in showing that that Boolean retrieval or `exactness' retrieval (as Bing calls it) without a relevance ranking method gives psychological encouragement to attempt to reduce large potentially relevant sets by inappropriate means. The use of relevance ranking to overcome the deficiencies of Boolean retrieval will be discussed in later parts of this topic.

What retrieval results do lawyers require?

There is considerable dispute, prompted to a large extent by Blair and Maron's experiment, over what lawyers do actually want and demand from computerised retrieval: do they prefer to maximise recall or precision. Dabney and Burson have contributed to this debate.

Daniel Dabney 'The Curse of Thamus' - see 'Ramifications for the Users of CALR Systems' in 'The Curse of Thamus: An Analysis of Full-Text Legal Document Retrieval' (1986) 78(5) Law Library Journal 5-40 (was on Yale Law School web site - No longer on the web)

Some notes on Dabney's approach (GG):

He accepts (like Blair &Maron) that lawyers want recall above all else, but unlike B&M he is talking about case research; but considers they will accept relatively low precision.

He gives reasons why case/statute systems may behave differently from litigation support systems in text retrieval performance, but the factors he gives go both ways. The most sensible conclusion is that another experiment is needed to prove anything significant about legal research systems (as opposed to litigation support systems.

He identifies 4 types of queries where computerised retrieval is most effective, particularly fact retrieval and citation retrieval;

He makes 5 suggestions how retrieval performance may be improved, but he takes a somewhat defeatist approach , and seems to largely accept B&M's conclusions, even for research systems.

Scott F Burson `A reconstruction of Thamus: comments on the evaluation of legal information retrieval systems' (1987) 79(134) law Library Journal (see LAWS 4609 Course Materials 1996 #3 p94 at the Law Reserve Desk)

Some notes on Burson's approach (GG):

He starts by interpreting Dabney as saying that Lexis and Westlaw probably only provide 20% recall - and that this `conjecture' is `almost certainly correct' In doing so he accepts similarilty of the use of text retreival for litiigation support (as by Blair and Maron) and for legal research - and this is a dubious assumption.

He says Dabney mistaken in arguing for improved recall, as this will degrade performance. He argues that lawyers want precison, not recall, in computerised tools -- but since these are not the only tools available, they can obtain recall by other means.

He argues high recall was necessary in the B&M study, as an information retrieval sysem was the only method of recall available. He then argues this isn't necessary in case/statute systems, simply because there are other (manual) research tools are also available which can complement the computerised tools;

He asserts that changing tools to increase recall will lower precision, but without evidence, and is against such changes. Iis he just assuming an immutable inverse ratio?

His opinion is that lawyers prefer high precision to high recall in computerised research tools, becasue they then have other tools available to obtain high recall (eg following up citations). In doing so he simply avoids the question of whether they think they are getting high recall, as in B&Ms study, or any evidence of whether they know what they can effectively get.

Again, this is a defeatist attitude, with no discussion of possible improvements to tools so as to improve recall without degrading precision, as this is assumed to be an impossible `will of the wisp'. Burson simply accepts low recall in computer-assisted legal research [cf litigation support] but isn't concerned.

As a result of these discussions, it is clear that the supposed inverse relationship between precision and recall poses a very difficult dilemma for the use of text retrieval systems in law. It may well be that the use of relevance ranking systems, either in conjunction with boolean retrieval or separately from it, provides the way out of this dilemma. This will be discussed in the subsequent readings on legal research via the internet, where the greatest use of relevance ranking systems has been made.

Further references

Daniel Dabney 'A Reply to West Publishing Company and Mead Data Central on The Curse of Thamus.' (1986) 78 Law Library Journal 349 (was on Yale Law School web site - no longer on the web)

Relevance ranking

One of the most important innovations in text retrieval, which can be used both as an alternative and an enhancement to boolean retrieval, is relevance ranking systems . They are also sometimes called 'best match' or 'probability models' (Turtle) or a 'statistical interface' (Feldman).

Inverse document frequency and within document frequency

The common feature of the various approaches to relevance ranking is the attempt to rank documents by the probability that they will be relevant to the query, such that the most relevant document is first in the list of retrieved documents, the next most relevant is displayed second, and so on.

The simplest relevance ranking system would be to count the number of times each search term occurred in a document, calculate the sum of all search terms occurring in the document, and list the document with the highest total as 'most relevant'. This would be a very crude measure.

Howard Turtle 'Text retrieval in the legal world' Parts 5.5 and 5.8 in Artificial Intelligence & Law, (1995) Vol 3 Nos 1-2 discusses the basic elements of relevance ranking systems.

The main question is how to calculate the 'significance' of a particular occurrence of a word in a document, as a means of assessing the relative likely relevance of the document in which it occurs. The second question is how to compound the measures of significance of each occurrence of each search term so as to give an overall measure of significance of a document.

Turtle says that two measures of word occurrence significance are used in most relevance ranking systems:

(i) Inverse document frequency = the number of occurrences of a term in whole database divided by the number of words in the whole database. This means (very roughly) that terms which occur in relatively fewer documents in the whole collection are given greater weight (also called the 'discrimination value' of the term).

(ii) Within document frequency = the number of occurrences of a term in a document divided by the number of words in the whole document. Therefore (again in rough terms) a word that occurs a lot in a short document gets a high score on this criterion.

One approach is then that the weighting to be given to an occurrence of a term in a particular document is the product of these measures for that term. As a result, for example, a term that occurs a lot in a short document (and so has high within document frequency), but doesn't occur very often in the whole database (and so has a high inverse document frequency) will get a very high overall weighting.

Finally, each occurrence of a search term in a document is multiplied by its appropriate weighting. The measure of relevance of the whole document would then be the sum of the weights of each occurrence of each search term in the document. So, a document which has many occurrences of search terms which have high overall weightings will be regarded as a document high in relevance.

There are many measures of relevance implemented in search engines more complex than this simple one.

Example in an internet system - AustLII's relevance ranking

AustLII implements relevance ranking in its SINO search engine in two ways (selected from the 'Find' setting), as both an alternative to boolean searching and as an enhancement to it:

'Any of these words' searches, which is searching based solely on the relevance ranking of search terms, with no boolean or proximity operators able to be used. The relevance ranking algorithm is described briefly in Freeform Search Help ('any of these words' used to be called 'Freeform' on AustLII)

'Boolean query' searches, are in fact not just boolean queries, but have results ordered by likely relevance. A boolean search is first carried out, and only those documents which satisfy the boolean search (ie an 'exact match' search) are then ranked into likely order of relevance.

The availability of relevance ranking can alter a user's search strategy using boolean retrieval. It is often wise to use a broad boolean search in order to maximise recall, relying upon the relevance ranking to supply the precision that the boolean search lacks.

There is a brief discussion of how popular internet search engines rank web pages in How Search Engines Rank Web Pages (on Search Engine Watch).

Effect of relevance ranking on evaluation of search results

If a search engine uses relevance ranking in any way, it is no longer possible to evaluate its results using simple measures of recall and precision. For example, if the first 50 documents retrieved by a relevance ranked search are all or most of the relevant documents in a database, it doesn't much matter that a search retrieves 100 documents but the last 50 were not very relevant, because the user can just keep working down the list until the degree of relevance is no longer high enough to warrant their continuing reading additional items.

Turtle (see 3.2 in his article) suggests methods of constructing a precision / recall curve, which shows recall values at pre-defined recall points (eg from 0.1 to 1.0 of all relevant documents). One way of thinking about such a curve is that it measures such things as 'if a user stops browsing when precision drops below 50% (ie only one in two documents is regarded as relevant), does this occur when recall of only 30% of relevant documents has occurred, or not until 90% of relevant documents have been recalled?

Examples in pre-internet systems - WestLaw's WIN and Lexis 'Freestyle'

Westlaw's WIN and Lexis' 'Freestyle' were two pre-internet examples of so-called 'natural language' searching, which were early implementations of relevance ranking to legal documents. In the LAWS 4609 Course Materials #3 at the UNSW Law Reserve Desk there are extracts from search manuals - WestLaw 'Retrieving documents with WIN - WestLaw is natural' (Chapter 5 of Introduction to WestLaw (1993), WestLaw) at p105 and Lexis 'Freestyle' searching (Information from Lexis help screens) at p110.

For discussion of evaluations, see:

Sheilla Desert, "Westlaw's Natural v. Boolean Searching: A Performance Study" 85 Law Library Journal 713 (Was on Yale Law School web site - no longer on the web)

Daniel Dabney 'A Reply to West Publishing Company and Mead Data Central on The Curse of Thamus.' (1986) 78 Law Library Journal 349 (Was on Yale Law School web site - no longer on the web)

Integration of hypertext and text retrieval

The purpose of these extracts is to act as an introduction to how hypertext and text retrieval may be combined in unusual ways to produce a result where 'the sum is greater than the parts'. In section 5 following, dealing with legal research on the internet, there is a similar theme in how it is necessary to use 'intellectual indexing' (which are essentially structures created using hypertext) in addition to full text retrieval, in order to get the best results.

The 'DataLex' approach to integrating hypertext and text retrieval

The main reading for this section is extracts from Graham Greenleaf, Andrew Mowbray and Peter van Dijk 'Representing and using legal knowledge in integrated decision support systems: DataLex WorkStations' Artificial Intelligence and Law Kluwer, Vol 3, Nos 1-2, 1995, 97-124; This pre-AustLII paper outlines an approach to the integration of hypertext and text retrieval in legal applications, much but not all of which has subsequently been implemented on AustLII. In a later reading guide, the integration of both these technologies with inferencing systems is discussed.

Relative costs of different technologies

In Paquin, Blanchard, and Thomasset `Loge-expert: From a legal expert system to an information system for non-lawyers' Proceedings of the 4th International Conference on Artificial Intelligence and Law, ACM Press 1991, p254 (copy available at UNSW Law Reserve desk in LAWS 4609 Materials #3 (1996) Theories of computerising law: hypertext and text retrieval at p20 - ignore the expert systems aspects for the moment).

This article contains one of the few attempts to compare the benefits and costs of using text retrieval, hypertext and expert systems to computerise legal information.

retrieval02

Diagram of relative costs and effectiveness of different forms of computerisation of law (following Paquin, Blanchard, and Thomasset 1991)

By 'noise' they mean the opposite of precision (items retrieved but not relevant). By 'silence' they mean the opposite of recall (items relevant but not retrieved).

These authors developed their theory (in pre-internet times) before the widespread use of relevance ranking (which makes arguments about the high level of 'noise' in text retrieval much harder to sustain), and before it was clear that large-scale automation of the creation of hypertext links was possible (which dramatically reduces the costs of creation of hypertexts).

Additional reading

M. A. Pasha and P. J. Soper 'Combining the Strengths of Information Management Technologies to Meet the Needs of Legal Professionals' (1996) 2 The Journal of Information, Law and Technology (JILT) - discusses limitations of hypertext and the need to integrate artificial intelligence and hypertext techniques ('knowledge-based hypermedia').

Other approaches to text retrieval in law

There are many enhancements to basic boolean ('exact match') retrieval systems, and quite a few alternative approaches to boolean retrieval. Some have been tested for legal information retrieval.

Two surveys of the various approaches that have been taken to text retrieval in law are:

Howard Turtle's survey - 'Text retrieval in the legal world'

Howard Turtle 'Text retrieval in the legal world' Artificial Intelligence & Law, (1995) Vol 3 Nos 1-2 (see LAWS 4609 Course Materials #3 at the UNSW Law Reserve Desk) is one of the most systematic surveys of pre-internet approaches to legal information retrieval. It is often quite technical but is recommended highly. Howard Turtle played an important role in developing the WIN (WestLaw is Natural) relevance ranking retrieval system for WestLaw.

Erich Schweighofer's survey - 'The Revolution in Legal Information Retrieval'

Eric Schweighofer ''The Revolution in Legal Information Retrieval or: The Empire Strikes Back' - 1999 (1) The Journal of Information, Law and Technology (JILT). Schweighofer's paper is for the most part less technical than Turtle's. Among the many projects summarised and discussed, Schweighofer includes summaries of:

Vector Space Models where documents and search queries are represented as vectors, and the similarity between a search vector and documents vectors is calculated. This approach is related to the Flexicon project (see below).

Approaches to relevance ranking (which he calls 'inference networks').

Connectionist Information Retrieval - the use of neural networks and sub-symbolic reasoning for retrieval

'Automatic Acquisition of Legal Knowledge (Machine Learning) and Summarising of Documents' - which is also related to Flexicon (see later)

What makes internet research different

Some of the factors that make legal research on the internet different from legal research using a CD-ROM, a single dial-up system (eg Lexis), or a litigation support system are:

That research may be across multiple sites on the web (often world-wide).

Wide use of relevance ranking systems (see earlier).

The role of web spiders or robots to build searchable collections of remote documents.

The increasingly complex relationship between directories (hypertext-based structured sets of links) and search engines.

The (potential) role of meta-tags to provide some control over search engine behaviour to the developers of web pages.

This Part first discusses internet-wide search and indexing tools, and then moves to a discussion of specialist tools dealing with legal information.

Two general resources referred to often in this Part are:

Graham Greenleaf the Internet for Asian Law - Project DIAL - referred to below as 'Greenleaf 1998';

Greenleaf et al Solving the Problems of Finding Law on the Web: World Law and DIAL', 2000 (1) The Journal of Information, Law and Technology (JILT) - referred to below as 'Greenleaf et al 1999'.

Web spiders - the basis of internet search engines

Internet-wide search engines are based on the interaction of at least three programs: a web spider (also known as web robot or web crawler), a program that creates a concordance (index) for searching, and a search engine. Here are some explanations of how they interact, as background to later more detailed discussion of search engines:

How Search Engines Work (on Search Engine Watch) - for much more detail see Search Engine Features For Webmasters, and connected pages (on Search Engine Watch).

Angela Elkordy Web Searching, Sleuthing and Sifting Lesson Three: What's next? (Search Engines and Web Indexes) (1999) The part headed 'What's a bot?' gives another explanation .

Greenleaf 2.4. Automated indexes and Internet-wide search engines - (Greenleaf 1998) This part also discusses some limitations of search engines.

Why is internet legal research difficult?

There is relatively little research as yet about the difficulties of legal research on the internet, as distinct from internet research generally (see below), on which much has been written.

See 2. The Problem - Internet Legal Research is Difficult (in Greenleaf et al 1999) for a 1999 discussion of the difficulties of using internet-wide indexes and search engines and indexes for legal research.

For an older (1998) but in some respects more detailed discussion, see Chapter 2.- Legal research via the Internet - Potential and problems' (Greenleaf 1998). The parts of this Chapter that are most useful here are:

2.2. Finding law on the net - Why is it so difficult?

2.3. Intellectual indexes (directories) for law - Reviews the limitations of current internet indexes for law.

2.4. Automated indexes and Internet-wide search engines - Discusses some of the basics features of internet-wide search engines, and why they are often not effective for legal research.

There is a summary in 2.8. Conclusions - Reasons supporting the development of Project DIAL.

In Greenleaf et al 1999, these problems of general search engines for legal research were summed up as:

They are not comprehensive - see Lawrence and Giles 1999, discussed below

expansion to 800M web pages is core problem

best coverage is 16% (Northern Light)

coverage declined from 33% in 1997 to 16% in 1999

Even meta-search engines could give 42% at best

Many other reasons why they are not comprehensive

Diminishing economic returns in increasing coverage?

General search engines are often out of date

General search engines contain too much `noise'

General search engines are biased in favour of 'popular' pages

General search engines do not provide unbiased world-wide coverage

Some are selling priority places in their relevance ranking

Are these reasons still correct, six years later?

General research on Internet-wide search engines and directories

There is a great deal of published research on the effectiveness of various internet-wide research tools (ie those which are not restricted to legal information).

Some of the more useful sources are discussed below, with the most recent information first::

Search Engine Showdown (2006)

Greg Notess Search Engine Showdown site appears to do attempt an independent evaluation of the claims of coverage of search engines, as seen from the following pages (as at 4 August 2006):

Database Total Size Estimates as of 2002

'Relative Size Showdown' - This test in 2002 of total hits from 33 searches places iWon and Google as the two search engines providing the largest numbers of hits in a comparison of searches. [However, we should question whether that is a particularly meaningful statistic - read the report.]

The percentage of dead links in search results in 2000 was as high as 13.7% (Alta Vista), but only 2.3% for Fast.

A commonly quoted estimate of the total number of web pages as at mid-2000 was 1 billion web pages, which if correct would means that the best search engines were only covering about 30-35% of the total web. In Lawrence and Giles 1999 study (below), the estimate was that the best search engine then covered 15% of a total 800M web pages.

Search Engine Watch (2000)

One of the most extensive general resources is Search Engine Watch edited by Danny Sullivan. It's a huge site and easy to get lost, so you might like to start with the guide to first-time visitors. It is updated constantly, and has a very informative monthly newsletter to which anyone who is seriously interested in search engines is likely to subscribe.

Aspects of the site that you might wish to look at include:

Search Engine Status Reports - an extraordinary array of comparisons of the different search engines.

The report on Search Engine Sizes As at its 7 July 2000 report, Search Engine Watch accepted an estimate of 1 billion web pages at at February 2000 (up from 8000 million estimated by Lawrence and Giles in early 1999). It states that Google had the broadest coverage, with 56% claimed coverage (560M web pages), with WebTop and Inktomi second with 500M pages each, and AltaVista fourth with 350M (35% coverage). ( In mid-1999 his report indicated that Alta Vista was clearly the search engine with the broadest coverage, followed by Inktomi (Hot Bot) and Northern Light, and with the others well behind. See, in contrast, Lawrence and Giles below).

His graphs also show the vastly increased claims of coverage by all of the major search engines over the previous 12 months, with all admitting coverage no greater than 150M pages in June 1999 (consistent with Lawrence and Giles claim of no more than 15% coverage), but with claims in June 2000 of over 350Mpages, and claims by Google and Inktomi in July 2000 of even more rapid (almost instant) escalation to over 500M pages, these claims must be treated with some scepticism.

The Major Search Engines - a valuable comparison and explanation of the major internet-wide search engines.

Lawrence and Giles study (1999) - 15% coverage only?

A study by Lawrence and Giles (Lawrence S and Giles C L (1999) 'Accessibility of information on the Web' Nature, Vol 400, 8 July 1999, pgs107-9 (Macmillan, UK)), argued that the best search engines (in early 1999) only covered 15% of the estimated 800M web pages, after adjustment for bad links. Northern Light, Snap and AltaVista were roughly equal. It is important that this research was based on the author's independent assessments of what percentage of web pages each search engine covered, not the search engines' self-reporting assessments of what they cover.

The article is not available in full for free on the web, but is discussed in 2.3 General Purpose Search Engines Alone are Insufficient (Greenleaf et al 1999). A summary by Greg Notess (who runs Search Engine Showdown) , where he concludes that the article generally agrees with his own tests.

Feldman's survey (1998)

Susan Feldman Web search services in 1998: Trends and challenges (1998) Searcher Vol 6 No 6 - A superb survey which includes criteria for evaluation of search engines, trends, and a comparative table of major search engines.

Some items to note from Feldman's paper.:

No single search engine goes close to making the whole world-wide-web searchable: 'Results show little overlap between search engines, so use more than one.' She cites evidence that 'the overlap is at best about 34 percent.'.

'Most sites recognize that a combination of browsing categorized pages and direct query serves users best.'. So it seems some combination of intellectual indexing and free-text searching is being accepted as providing the best results.

' Most of the search companies have developed a precise search feature which allows Boolean searching, in addition to their standard statistical interface for the broad query.' (A 'statistical interface' refers to pure relevance ranking without boolean operators, whereas the reference to 'Boolean searching' is to boolean searching where the results are then ranked for relevance.)

Development of multi-language search capacities and various forms of on-line translation capacities is an active field of research at present.

'Metatags' - 'descriptors' or subject headings or indexing terms which are not visible to the user of a page but can be read by search engines - are not utilised by search engines as much as we might expect. One reason given is that although about 30% of web pages have metatags, 90% of those are 'spam' (also called 'metatag stuffing'), 'misleading terms or repeated terms to make their site come up in the top 10 listed, or in completely irrelevant search results'. Many search engines therefore simply ignore metatags.

Among the various means of funding the operation of search engines, 'Charges for preferential listings' is listed as an increasingly popular option. This means that some search engine companies sell the highest ranking positions in their (so-called) relevance ranked lists of search results (ie the rest of the results might genuinely be ranked in order, but the top-ranking spots are simply sold). Amazon.com has been subjected to heavy criticism for a similar practice. The ethics/transparency of relevance ranking.

These conclusions seem generally consistent with what is argued in the Project DIAL Report.

Feldman's Search Engine Feature Chart(at the end of the article) compares the features of eight search engines (and associated directories).

Targeted web spiders

The problems of internet-wise research tools have led to a search for discipline-specific approaches to finding internet-wide information. One solution has been to combine an internet index for a discipline such as law with a web spider that only goes to sites listted in that index and does not 'wander off' to other sites. This is variously called a 'limited area search engine' (LASE) or a 'targeted web spider'

3.5 Other Limited Area Search Engines (Greenleaf et al 1999) discusses a number of such LASEs.

3.1 A New Technical Solution - A Limited Area Search Engine for Law (Greenleaf et al 1999) explains the approach taken in AustLII's World Law since 1997.

SOSIG Law Gateway

The Institute of Advanced Legal Studies in London and the University of Bristol Law Library have since late 1999 started the SOSIG Law Gateway as part of the UK' s Social Science Information Gateway (SOSIG) project:

The SOSIG Law Gateway provides guidance and access to global legal information resources on the Internet.The service aims to identify and evaluate legal resource sites offering primary and secondary materials and other items of legal interest.'

The most detailed description of the Law Gateway is Steven Whittle A National Law Gateway : developing SOSIG for the UK Legal Community, Commentary 2000 (2) The Journal of Information, Law and Technology (JILT).

retrieval03

Some noteable aspects of the Law Gateway:

It aims to evaluate, not just to identify and link.

It aims for global coverage;

Its email notification service ('current awareness service').

Its indexing standards.

Whittle's article states that:

A Social Science Search Engine is also provided - this is a limited area search engine drawn from a supplementary database of over 50,000 links gathered by a web robot which visits each of the quality websites featured in the Catalogue (including all the Law section sites) and follows any links it finds for those pages and automatically indexes the content.

It seems that the way this works is that, if a search using the search engine on each page of the Law Gateway fails to find a record in the catalog, only then does the system offer another search over the full text of law (and other) sites to which the web robot has been sent. For example, try a search for 'domain names', and it gets no results by searching the catalog, but finds four items from the full text search.

AustLII's World Law / Project DIAL

World/ World Law is the global index and search facility on AustLII. Project DIAL was a part of World Law (the Asian Development Bank funds the legislation and Asian emphasis, and training in some Asian countries) but there is no signigicant technical difference between World Law and DIAL.

Graham Greenleaf et al Solving the Problems of Finding Law on the Web: World Law and DIAL', 2000 (1) The Journal of Information, Law and Technology (JILT) gives a detailed (1999) account of World Law.

retrieval04

There is a User Guide which helps explain how it works.

retrieval05

The main features of World Law that are worth noting are:

Global coverage - but for some countries this is mainly through an embedded search.

Each site added to the index is usually catalogued under multiple nodes: by country; by source or type of document; and by a subject index. This gives considerable flexibility in how the data may be accessed, and in the search combinations that are possible.

Searches from any point in World Law return two sets of results: (i) all web pages where the search terms are found, by a full text search of all sites to which the web spider has been sent; and (ii) all directories in the index (catalog) where the search terms are found. So, both the catalog are searchable, and the full text of sites to which the web spider is sent.

The scope of a full text search of the web spider materials is limited by where you search from (the context). - limited scope searches.

A 'Search this site' facility allows the full text of individual web sites to be searched.

The index also contains stored searches - see 6. Storing Searches to Create a Self-Maintaining Index

There are an increasing number of outside contributors to maintainig the index - see 3.3 International Indexing Partnerships - A Necessary Element (in Greenleaf 1999)

Law sections of ODP (Open Directory Project)

The Open Directory Project (ODP) explains in About the Open Directory Project that its goal is 'to produce the most comprehensivedirectory of the web, by relying on a vast army of volunteer editors' because 'As the web grows, automated search engines and directories with small editorial staffs will be unable to cope with the volume of sites.'

Features of the ODP are:

Everyone is invited to become editors:

Like any community, you get what you give. The Open Directory provides the opportunity for everyone to contribute. Signing up is easy: choose a topic you know something about and join. Editing categories is a snap. We have a comprehensive set of tools for adding, deleting, and updating links in seconds. For just a few minutes of your time you can help make the Web a better place, and be recognized as an expert on your chosen topic.

The resulting data is available for anyone else to re-use in other projects, in whole or part. See the Free use license for the Open Directory data

ODP does not have its own search engine, but ODP data is being used with other commercial search engines, including Netscape, Lycos, HotBot, and others - See Sites using Open Directory data.

The Law subdirectories of ODP have over 11,000 entries (as at August 2000). The best way to gain an appreciation of whether ODP is likely to be effective to produce an effective legal research tool is to look at a selection of ODP entries and gain an idea of the range and quality of its coverage. Is it genuinely international? What do most of the 11,000 entries concern?

retrieval06

Library of Congress' GLIN (Global Legal Information Network)

The US Library of Congress' GLIN (Global Legal Information Network) is not a limited area search engine, but is the most ambitious centralised collection of laws yet undertaken other than the commercial LEXIS system. Its home page explains its purpose:

The Global Legal Information Network (GLIN) provides a database of national laws from contributing countries around the world accessed from a World Wide Web server of the U.S. Library of Congress. The database consists of searchable legal abstracts in English and some full texts of laws in the language of the contributing country. It provides information on national legislation from more than 35 countries, with other countries being added on a continuing basis.

See 2.7.1. Global Legal Information Network (GLIN) (Greenleaf 1998) for a summary of GLIN (as at 1998).

retrieval07

Web Law

www.weblaw.edu.au "is is a cooperative subject gateway to Internet resources for Australian legal researchers. It is a distributed cataloging system, based on a central database model. Law librarians from the participating organisations have taken responsibility for a particular subject area or areas and provide annotated records for quality Internet legal resources in this area. An academic or practitioner with expertise in the subject is involved in peer assessment of the listings. The sites are checked and updated each month. Quality control is assured because participating organisations adhere to common selection criteria and quality guidelines, as well as to the designated metadata and thesaurus schema."

Semantic Gap

Difference between syntax and semantics, meaning and expression

Introduction to Metadata

Metadata is one way to overcome the semantic gap, by providing "Data about data". It includes unique identifiers that identify concepts, documents and other objects within the scope of the system. It often also contains unique identifiers which refer to specific relationships between other resources.

Citation Systems

Citation systems are an example of metadata

Uses of Metadata

Searching

Inferencing

Semantic Web

Semantic web is a set of standards and software to create, publish and inference over structured data on the Internet.

Triples

Ontologies

SchemaWeb.info

Is a generalised law ontology possible?

OpenCyc

Agents

RDF

RDF is a standard for expressing metadata for the Internet and Semantic Web

Dublin Core

Dublin Core is a metadata standard for published articles

Licencing Systems

Such as creative commons

AEShareNet

Problems with Metadata

Oldstyle Metadata Tags and their problems

Meta tags were widely abused - trust problems

Legal Uses

AustLII - Use of citations - Usermark

Caselaw NSW

Weblaw

Introduction

The aim of this outline is to help you obtain an understanding of the possibilities and limitations of knowledge-based technologies when applied to law.

Particular areas of coverage include:

The history of Artificial Intelligence and 'expert systems', and the main models of computerised inferencing (drawing inferences) that have emerged.

The relationship between legal theory (jurisprudence) and computerisation of legal tasks.

Rule-based reasoning and its applications to statute-based law.

Case-based reasoning and the problems of applying it (or neural networks or anything else) to case law.

The relationships between inferencing and the other technologies of hypertext and text retrieval.

The special issues in web-based legal inferencing, and the opportunities for greater integration with hypertext and text retrieval.

Automated legal document generation systems - why so little progress?

Web resources

The most substantial collections of research papers available for free access can be found at:

The Foundation for Legal Knowledge Systems in the Netherlands; this site contains 100+ papers from the JURIX Conferences since 1989. Many of the papers are highly technical (particularly those concerning various types of logics), but many are also accessible to those new to the field.

Computerisation of Law Resources (CompLRes) database on AustLII - contains papers from AustLII's 'Law via Internet' Conferences, as well as other papers. Includes quite a few papers on inferencing systems.

Links to other research papers, home pages of researchers etc can be found at:

WorldLII Catalog:Computerisation of Law:Inferencing - includes many of the resources which will be used in this Study Guide.

It is also worth checking the Other Indexes listed in the WorldLII Catalog.

Print resources

Here are the main starting points:

Proceedings of the (biennial) International Conference on Artificial Intelligence and Law (ICAIL) since 1987 are the single most substantial resource in the field - The web version of the Conference Proceedings is hosted by the ACM Virtual Library, but not all of the papers are here, and all are password protected and only accessible to ACM members. The programmes for each of the Conferences - and therefore the table of contents for proceedings are at the ICAIL home page.

The journal Artificial Intelligence and Law (Kluwer) is the main refereed journal in the fieldsee The articles are not on the web.The journal's home page contains the tables of contents of previous issues.

John Zeleznikow and Dan Hunter, Building Intelligent Legal Information Systems: Representation and Reasoning in Law Kluwer, 1994 is the most approachable textbook to give an overview of the field, and is recommended to those commencing study of this area.

See the selected 'Computers and Law' monographs in the UNSW Library for further print resources. Most monographs in this field are very specialised and technical.

Background and history of 'AI & law'

For a brief introduction to the history of AI and law, and the components of a 'legal expert system', see Graham Greenleaf `Legal expert systems - robot lawyers?' (1989) Australian Legal Convention. . It's rather old, but still useful, particularly on the components of a 'legal expert system'.

Bob Moles and Bibi Sangha Computer Systems - and Legal Reasoning? (on Uniserve Law) - the introductory part 'The artificial intelligence project ' gives another view. There is also An Overview of Different Approaches to Expert Systems

Andrew Greinke 'Legal Expert Systems: A Humanistic Critique of Mechanical Legal Inference' - includes a useful and critical introduction to underlying computing techniques in 'AI and law' (in E-Law)

Rule-based inferencing systems in law

Can rule-based systems be used in law?

A controversial starting point for arguments about the extent to which rule-based reasoning systems are useful in law is Robert Moles 'Logic Programming - An Assessment of Its Potential for Artificial Intelligence Applications In Law'Journal of Law and Information Science (1991) vol 2 no 2 (now on UniServe Law) Moles' argument is explicitly a critique of the body of work known as `logic programming in law', but is a more general attack on (at least) all attempts to create rule-based expert systems in law. In effect, he concludes that the value of any such research is an open question.

I have listed below some of Moles' principal arguments (but without doing them justice in a brief description), followed by a brief comment on how such an argument could be answered.

Isomorphism - Moles gives a damning paraphrase of Bench-Capon's description of how isomorphism is maintained in the Alvey Project, and rightly concludes that it shows considerable insensitivity to the nature of legal interpretation. Answer?: Even if Bench-Capon's is a poor example of isomorphism, how is this a criticism of isomorphism as an ideal? Can it be done better? (see the next section of this guide for more extensive discussion)

Statutes are interdependent, so it is useless to represent in isolation; if we represented all possibly relevant rules, inferencing engines couldn't cope. Answer?: A good point, there is a practical limitation to the comprehensiveness of any expert system. But the same criticism applies to a textbook, but doesn't stop them being written. Is this a double standard? Isn't it a question of expert judgment as to what other rules must be added;? Also, doesn't the user still need to use discretion or an over-ride capacity ? (This is the `its only a tool' argument.)

Statutes and the common law are interdependent , so its no use just representing statutes. Moles gives superb example concerning the British National Act (the subject of a famous early legal expert system by Sergot et al) - to understand the Act it is necessary to understand a statue 250 years old and the case law on it at the time. Answer?: There is no way around this but to rely on expert judgement to add some rules (or opportunities for users to make decisions?) reflecting these interpretative issues, as there is no possibility of comprehensiveness!

Consequentialism - Courts [always?] apply consequentialist reasoning (ie consider the consequences of reasoning in deciding whether the reasoning is adequate), so it is impossible to separate the knowledge representation from the inferencing engine. He give an excellent example of the DIY shop case. Answer?: Perhaps there isn't any, unless you hypothesise meta-rules like the one he quotes; also, although he (via MacCormick) claims that such reasoning is encountered `at every turn', it must be something of an empirical question as to how much this threatens the viability of legal expert systems.

Words are ambiguous - He correctly chides the logic programmers for silly statements about choosing unambiguous interpretations of legal sources; and for conducting surveys to determine meanings of words. Answers?: He wrongly conflates Susskind's misuse of the `survey' approach with what the logic programmers do, and doesn't seem to appreciate Susskind's `stop here' approach to ambiguity. The answer to this can only be that the ambiguity of language can only be dealt with by leaving some interpretative decisions in the operation of an expert system to the user (while attempting to assist the user to find the correct interpretation), and by attempting to make it explicit where interpretations have been built in to the knowledge-base (part of the idea of isomorphic representation).

Rules [or groups of them] are not atomistic entities, so isomorphism's claimed advantages in terms of maintainability are fictitious. Answer?: There is a major error of understanding in Moles' thinking that to say that to change one rule is to change the meaning of other rules is somehow a criticism of isomorphism - this is precisely what isomorphism aims to achieve!

The view that there is no need for legal expertise is obvious rubbish - Moles suggest that if the logic programmers (in the British Nationality Act example) had obtained expert advice it would have destroyed their project, and makes the acerbic comment that `They have used logic programming to model the logical consequences of their own untutored assumptions as to how a statute, dealing with an area about which they have no experience, would be read and used by someone about whom they have no understanding'. No answer needed, Moles is correct that legal expertise is needed.

Moles concludes that there has been no progress by anyone in the previous 20 years! He sees the need to approach with an `open mind, the basic question of the suitability of the legal domain for he development of expert systems'

It is worth asking 'what is Moles really criticising?' - Is his model of 'logic programming' a straw man? If he is criticising the quest for artificial intelligence in the strict sense, then his pessimism is justified, but not if he is purporting to generalise this into a more overall critique of inferencing systems as useful tools for lawyers. If he is, then the relevant comparison may be a textbook, not a human mind. Perhaps this is a mole-hill not a mountain.

Isomorphism and knowledge representation

'Isomorphism' in legal knowledge representation, used in the sense of 'creating a well defined correspondence between source documents and the representation of the information they contain [that is] used in the system' [Bench-Capon and Coenen, 1992], has been shown to provide many advantages for the operation of legal inferencing systems (despite dissents such as that of Moles).

Arguments for the use of an isomorphic approach and a 'quasi natural language' representation are discussed in Greenleaf, Mowbray and van Dijk 5.5. Isomorphism and the Value Of 'english-Like' Representations (in 'Representing and using legal knowledge in integrated decision support systems: DataLex Workstations' [1995] CompLRes 1).

The approach taken in this paper is very similar to that taken by SoftLaw (see following section), who have applied an isomorphic and English-like representation to their development of large commercial inferencing systems.

Additional reading

Russell Allen (1997), 'The Use of Isomorphism in Modelling the Common Law' [1997] CompLRes 3 - This essay considers the difficult issue of whether the concept of isomorphism is relevant to knowledge-representation involving case law.

Other papers and resources on rule-based inferencing in law

Andrew Greinke 'Legal Expert Systems: A Humanistic Critique of Mechanical Legal Inference' - Argues that legal expert systems are of limited use in dealing with real world complexity due to limitations of their pattern-matching and rule application strategies. (in E-Law)

Robert N. Moles Surendra Dayal 'There Is More To Life Than Logic' Journal of Law and Information Science (1992) vol 3 no 2 pp 188-218 (on UniServe Law)

For further references, see AustLII Index:Computerisation of Law:Inferencing:Rule-Based

Case study of legislation-based inferencing systems: Ruleburst

Ruleburst (was Softlaw) is an Australian company which is one of the world leaders in the field of large-scale legal inferencing systems. Ruleburst was founded in 1989, based on expert systems work by Mead and Johnson from the mid-1980s, and has grown to over 50 staff. Its clients include some of the largest public sector agencies in Australia, and some overseas. It has won many national and international awards.

Browse the Ruleburst pages, and read the following to obtain an introductory idea of their approach:

The Softlaw approach

Softlaw and Government and the links from that page.

What is Statute Expert?

There are also some demonstration versions of Statute Expert which are available from the site, which you might want to look at. Go to the sample facts, then do the sample applicationi

4.1. Papers concerning SoftLaw's work

There are a number of papers about the Work of Softlaw:

Surend Dayal and Peter Johnson, SoftLaw Corporation A web-based revolution in Australian public administration? [1999] CompLRes 13 (Paper presented at AustLII's "Law Via The Internet '99" conference)

Philip Kellow (Director, Centre for Legal Process, Sydney) Legal expert systems via the Internet [1997] CompLRes 9 (Paper presented at AustLII's "Law Via The Internet '97" conference). This project (no longer proceeding) applied Softlaw's technology to development of a free access application via the Internet: "The first eLAPS module prepares an application for review to the Commonwealth Administrative Appeals Tribunal (AAT). This Tribunal reviews decisions made by Government agencies under a range of Commonwealth statutes. "

The following papers are available to students enrolled in the UNSW law subject "Computerisation of Law":

Peter Johnson Knowledge management, knowledge based systems and the transformation of government Leading People into 2000 Conference, August 1999

P Johnson Electronic Service Delivery: Achieving Accuracy and Consistency in Complex Transactions (1999)

P Johnson and S Dayal Best Practice in the Use of Rule-Based Systems (1998) - very detailed analysis of the precise advantages of rule-based systems compared with traditional administrative responses

The following older papers concerning SoftLaw are in the printed volume of Developing Computer Applications to Law (B) subject materials Vol # 5 - Case Study: SoftLaw available at the UNSW Law Library Reserve Desk.

Peter Johnson and David Mead `Legislative knowledge base systems for public administration - Some practical issues' Proc. 3rd International Conference on Artificial Intelligence and Law, ACM Press, 1991

Surendra Dayal, Peter Johnson & David Mead `Natural language - An appropriate knowledge representation scheme for the administrative domain' SoftLaw Corporation, 1993

Surendra Dayal et al `Beyond knowledge representation - Commercial uses for legal knowledge bases' Proceedings 4th International Conference on Artificial Intelligence & Law, Amsterdam, 1993, ACM Press

Softlaw Corporation - `STATUTE Corporate - Component Technologies Guide' (Chapter on STATUTE Expert) Softlaw Corporation, Canberra, 1991

Case-based reasoning systems in law

There are many attempts to develop case-based reasoning systems in law, but little information seems to be available on the web. The following notes only discuss one example of a case-based reasoning system applied to law.

Tyree's FINDER

These notes deal with various implementations of FINDER, Alan Tyree's implementation of case-based reasoning using 'precedent analysis by nearest-neighbour decision analysis' (PANNDA). FINDER is actually the particular example application using the 'finder's cases', whereas the general approach is more properly called PANNDA. However, the general approach is also referred to below as 'FINDER'.

Alan L Tyree 'FINDER: an expert system' (1987??) - The original paper on the FINDER case-based reasoning system, but unfortunately it does not contain a detailed explanation. However, it should be read as an introduction to FINDER.

FINDER (The Finder's Cases) , is an implementation of FINDER using the wysh software, with a rule added about trespassers .

These notes are derived from Alan Tyree Expert Systems in Law Prentice Hall 1989 Ch 7, pgs 137-44, and from discussions with Alan Tyree. They are, however, my interpretations (GG).

`Similarity' is the basis of FINDER - `The concept [of similarity] is at the heart of any well-defined notion of precedent.' (eg the maxim `like cases should be decided alike'); PANNDA attempts to model similarity in a `direct' fashion, by a mathematical measure of the presence/absence of key attributes of each case.

The key assumptions in FINDER are:

that a group of cases can be identified as relevant to deciding a legal issue;

that it is impossible to deduce any rule(s) of general application from the cases;

that it is, however, possible to identify attributes of those cases which are relevant to the Courts' decisions [although exactly how they are relevant can't be reduced to a rule]; for PANNDA, these attributes must also be reduced to Yes / NO form]

`that we have exercised care and skill in representing the attributes and values of each of the cases at an appropriate level of generality' (Tyree p138) -- ie the application of expertise.

This approach could be relevant where any statute or case simply lists factors which are to be taken into account or `weighed' in reaching a conclusion.

A summary of the method used in FINDER (in the 'finder's cases' example):

each of 10 factors identified by expert given a 1/0 ie present/absent in case;

low variance factors = where there are a preponderance of 1s or 0s in the 8 cases; high variance = where a 4/4 or 3/5 split; they ranged from 7/1 (low) for A4 and A9 to 4/4 (high) for A2; the low variance factors (7/1 etc) are considered to be more significant than the high variance ones ; therefore they are given a lower weighting (VAR) (eg .109), whereas high variance ones have higher weighting (eg .250);

for each Q, each factor in the test case is compared with the case; if the factors are the same, the score is 0; if different, the score is 1/VAR; the total distance between each case and the test case is the sum of these 10 scores ; so, the more zeros, the lower the scores; and the more difference on low variant factors, the higher the scores.

the case with the lowest score is regarded as the closest to the test case; those with higher scores progressively further away. ie not simply a matter of which one has the highest no of 0s and 1s the same as the test case.

The assumption behind the measure of variance (which determines the weight to be given to attributes) is that factors which almost all cases share, but 1 or 2 don't (ie 7/1/ or 6/2) are likely to be the most significant factors determining the outcome of a set of cases. Therefore, if a case differs from the test case in respect of one of these, this should mean it is `further away' than if it merely differed on one factor which is spread uniformly b/w all the cases (ie 4/4/ or 3/5).

The measure of variance is not correlated with the outcomes of the cases; ie if only two cases in a set of 10 have a Y value for variable A, A is regarded as more important than B which has a 50/50 Y/N split -- but the system doesn't know if there is a significant correlation b/w a Y answer for variable A and a particular outcome for the precedent. Tyree explains (verbally) that there are usually too few cases to make such a correlation in a way that will be statistically significant [otherwise, wouldn't we just represent such a correlation as a rule??]. The justification for his approach is that the cases, and the factors described within them, are not drawn at random, but are selected by the expert because of their significance. Therefore, variables with 4/4 or 5/3 splits actually separate the cases into groups more effectively, so `why would an expert include a 7/1 variable unless it is particularly significant'. He agrees this is contentious.

The outcome of a FINDER consultation is that a report is generated with identifies the 'nearest' case (to be followed) and the nearest case with the opposite outcome (to be distinguished), and the similar and distinguishing facts for both nearest case and nearest opposite case. It is a simulation of a legal argument from a statistical process. There is an attempt at recognition of hard cases, when it identifies that there are two 'nearest' cases (ie the same distance away from the test case) with opposite outcomes.

Some imitations of FINDER:

It is limited to true / false values (the wysh version also includes 'uncertain')

It makes no distinction between the hierarchical precedential value of cases. It is questionable whether this is a criticism -- perhaps it should only be used where there are a `flat' set of cases, where no particular case is regarded as more authoritative than others.

It does not emulate human legal reasoning in reaching its conclusions [as far as we know]. Tyree attempts to show that it does produce a similar result, however.

Other papers about FINDER

James Popple SHYSTER - a case-based legal expert system derived from FINDER and extending it

Greenleaf, Mowbray and van Dijk '6.2. 'Examples' and the Inference Engine' (in 'Representing and using legal knowledge in integrated decision support systems: DataLex Workstations' [1995] COL 1) - discusses the PANNDA implementation of FINDER in the DataLex Workstation Software.

JustSys: 'Split-up' and related systems (Stranieri and Zeleznikow)

One of the best-known current Australian research projects in this area is in the domain of property settlements in family law, legal decision support system built by the Database Research Laboratory at Melbourne's LaTrobe University Computing Institute.

'Family Law by Computer' (a discussion of 'Split-Up'), transcript of the Law Report, ABC Radio National, 16 July 1996 (Susanna Lobez with Andrew Stranieri and John Zeleznikow )

www.justsys.com.au is their website. You can try the sample applications there.

Additional reading

For papers generally concerning case-based reasoning in law see:

WorldLII Catalog:Computerisation of Law:Inferencing:Case-Based Reasoning

Argumentation and legal reasoning (an index of articles and web pages) on CBR-Web ( the Case-Based Reasoning homepage at the University of Kaiserslautern)

Map-based systems in law

www.grayske.com

Integration of legal inferencing systems with other technologies

The DataLex Project

A principal aim of the DataLex Project (1985-1994) was to explore the integration of legal inferencing technologies with hypertext and text retrieval.

Greenleaf, Mowbray and van Dijk (1995) 'Representing and using legal knowledge in integrated decision support systems: DataLex Workstations' [1995] COL 1; This article gives the pre-internet background to AustLII's inferencing research.
The following Parts set out the arguments in favour of integration of technologies:

Part 5. ntegrating inferencing with hypertext and text retrieval

Part 6. Multi-modal inferencing and integration

One of the earliest papers on the DataLex project was Tyree, Mowbray and Greenleaf The DataLex Project, presented at the first AI & Law Conference in Boston, 1987 (it describes the earliest DataLex 'shells', much earlier than those described in the above paper).

AustLII and legal inferencing via the web

AustLII is a proponent of legal inferencing systems via the internet, partly because of the potential it creates for legal inferencing systems to be integrated with large online collections of primary legal materials (like AustLII) which are being updated constantly.

Russell Allen et al 'With a wysh and a prayer: An experiment in cooperative development of legal knowledgebases' 2000 (2) The Journal of Information, Law and Technology (JILT)

Part 5. New Legal Services via the Web - AustLII's Research on Legal Inferencing (in Greenleaf Mowbray and King (1997) The AustLII Papers - This paper discusses these additional advantages.

AustLII's Legal Inferencing Project Home Page