-<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.1//EN"
- "http://www.oasis-open.org/docbook/xml/4.1/docbookx.dtd"
+<?xml version="1.0" standalone="no"?>
+<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
+ "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"
[
<!ENTITY % local SYSTEM "local.ent">
%local;
<!ENTITY % entities SYSTEM "entities.ent">
%entities;
- <!ENTITY % common SYSTEM "common/common.ent">
- %common;
+ <!ENTITY % idcommon SYSTEM "common/common.ent">
+ %idcommon;
]>
-<!-- $Id: pazpar2_conf.xml,v 1.7 2007-01-26 18:53:55 quinn Exp $ -->
<refentry id="pazpar2_conf">
<refentryinfo>
<productname>Pazpar2</productname>
<productnumber>&version;</productnumber>
+ <info><orgname>Index Data</orgname></info>
</refentryinfo>
+
<refmeta>
<refentrytitle>Pazpar2 conf</refentrytitle>
<manvolnum>5</manvolnum>
+ <refmiscinfo class="manual">File formats and conventions</refmiscinfo>
</refmeta>
<refnamediv>
</refsynopsisdiv>
<refsect1><title>DESCRIPTION</title>
- <para>
- The pazpar2 configuration file, together with any referenced XSLT files,
- govern pazpar2's behavior as a client, and control the normalization and
- extraction of data elements from incoming result records, for the
- purposes of merging, sorting, facet analysis, and display.
- </para>
-
- <para>
- The file is specified using the option -f on the pazpar2 command line.
- There is not presently a way to reload the configuration file without
- restarting pazpar2, although this will most likely be added some time
- in the future.
- </para>
+ <para>
+ The Pazpar2 configuration file, together with any referenced XSLT files,
+ govern Pazpar2's behavior as a client, and control the normalization and
+ extraction of data elements from incoming result records, for the
+ purposes of merging, sorting, facet analysis, and display.
+ </para>
+
+ <para>
+ The file is specified using the option -f on the Pazpar2 command line.
+ There is not presently a way to reload the configuration file without
+ restarting Pazpar2, although this will most likely be added some time
+ in the future.
+ </para>
</refsect1>
-
+
<refsect1><title>FORMAT</title>
- <para>
- The configuration file is XML-structured. It must be valid XML. All
- elements specific to pazpar2 should belong to the namespace
- "http://www.indexdata.com/pazpar2/1.0" (this is assumed in the
- following examples). The root element is named 'pazpar2'. Under the
- root element are a number of elements which group categories of
- information. The categories are described below.
+ <para>
+ The configuration file is XML-structured. It must be well-formed XML. All
+ elements specific to Pazpar2 should belong to the namespace
+ <literal>http://www.indexdata.com/pazpar2/1.0</literal>
+ (this is assumed in the
+ following examples). The root element is named "<literal>pazpar2</literal>".
+ Under the root element are a number of elements which group categories of
+ information. The categories are described below.
+ </para>
+
+ <refsect2 id="config-threads"><title>threads</title>
+ <para>
+ This section is optional and is supported for Pazpar2 version 1.3.1 and
+ later . It is identified by element "<literal>threads</literal>" which
+ may include one attribute "<literal>number</literal>" which specifies
+ the number of worker-threads that the Pazpar2 instance is to use.
+ A value of 0 (zero) disables worker-threads (all work is carried out
+ in main thread).
</para>
+ </refsect2>
+ <refsect2 id="config-server"><title>server</title>
+ <para>
+ This section governs overall behavior of a server endpoint. It is identified
+ by the element "server" which takes an optional attribute, "id", which
+ identifies this particular Pazpar2 server. Any string value for "id"
+ may be given.
+ </para>
+ <para>The data
+ elements are described below. From Pazpar2 version 1.2 this is
+ a repeatable element.
+ </para>
+ <variablelist> <!-- level 1 -->
+ <varlistentry>
+ <term>listen</term>
+ <listitem>
+ <para>
+ Configures the webservice -- this controls how you can connect
+ to Pazpar2 from your browser or server-side code. The
+ attributes 'host' and 'port' control the binding of the
+ server. The 'host' attribute can be used to bind the server to
+ a secondary IP address of your system, enabling you to run
+ Pazpar2 on port 80 alongside a conventional web server. You
+ can override this setting on the command line using the option -h.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>proxy</term>
+ <listitem>
+ <para>
+ If this item is given, Pazpar2 will forward all incoming HTTP
+ requests that do not contain the filename 'search.pz2' to the
+ host and port specified using the 'host' and 'port'
+ attributes. The 'myurl' attribute is required, and should provide
+ the base URL of the server. Generally, the HTTP URL for the host
+ specified in the 'listen' parameter. This functionality is
+ crucial if you wish to use
+ Pazpar2 in conjunction with browser-based code (JS, Flash,
+ applets, etc.) which operates in a security sandbox. Such code
+ can only connect to the same server from which the enclosing
+ HTML page originated. Pazpar2s proxy functionality enables you
+ to host all of the main pages (plus images, CSS, etc) of your
+ application on a conventional webserver, while efficiently
+ processing webservice requests for metasearch status, results,
+ etc.
+ </para>
+ </listitem>
+ </varlistentry>
- <refsect2 id="config-server"><title>server</title>
+ <varlistentry>
+ <term>relevance / sort / mergekey</term>
+ <listitem>
+ <para>
+ Specifies character set normalization for relevancy / sorting
+ and the mergekey - for the server. These definitions serves as
+ default for services that don't have these given. For the meaning
+ of these settings refer to the "relevance" element inside service.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>settings</term>
+ <listitem>
<para>
- This section governs overall behavior of the client. The data
- elements are described below.
+ Specifies target settings for the server.. These settings serves
+ as default for all services which don't have these given.
+ The settings element requires one attribute 'src' which specifies
+ a settings file or a directory . If a directory is given all
+ files with suffix <filename>.xml</filename> is read from this
+ directory. Refer to
+ <xref linkend="target_settings"/> for more information.
</para>
- <variablelist> <!-- level 1 -->
- <varlistentry>
- <term>listen</term>
- <listitem>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>service</term>
+ <listitem>
+ <para>
+ This nested element controls the behavior of Pazpar2 with
+ respect to your data model. In Pazpar2, incoming records are
+ normalized, using XSLT, into an internal representation.
+ The 'service' section controls the further processing and
+ extraction of data from the internal representation, primarily
+ through the 'metadata' sub-element.
+ </para>
+ <para>
+ Pazpar2 version 1.2 and later allows multiple service elements.
+ Multiple services must be given a unique ID by specifying
+ attribute <literal>id</literal>.
+ A single service may be unnamed (service ID omitted). The
+ service ID is referred to in the
+ <link linkend="command-init"><literal>init</literal></link> webservice
+ command's <literal>service</literal> parameter.
+ </para>
+
+ <variablelist> <!-- Level 2 -->
+ <varlistentry><term>metadata</term>
+ <listitem>
+ <para>
+ One of these elements is required for every data element in
+ the internal representation of the record (see
+ <xref linkend="data_model"/>. It governs
+ subsequent processing as pertains to sorting, relevance
+ ranking, merging, and display of data elements. It supports
+ the following attributes:
+ </para>
+
+ <variablelist> <!-- level 3 -->
+ <varlistentry><term>name</term>
+ <listitem>
<para>
- Configures the webservice -- this controls how you can connect
- to pazpar2 from your browser or server-side code. The
- attributes 'host' and 'port' control the binding of the
- server. The 'host' attribute can be used to bind the server to
- a secondary IP address of your system, enabling you to run
- pazpar2 on port 80 alongside a conventional web server. You
- can override this setting on the command lineusing the option -h.
+ This is the name of the data element. It is matched
+ against the 'type' attribute of the
+ 'metadata' element
+ in the normalized record. A warning is produced if
+ metadata elements with an unknown name are
+ found in the
+ normalized record. This name is also used to
+ represent
+ data elements in the records returned by the
+ webservice API, and to name sort lists and browse
+ facets.
</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>proxy</term>
- <listitem>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry><term>type</term>
+ <listitem>
<para>
- If this item is given, pazpar2 will forward all incoming HTTP
- requests that do not contain the filename 'search.pz2' to the
- host and port specified using the 'host' and 'port'
- attributes. This functionality is crucial if you wish to use
- pazpar2 in conjunction with browser-based code (JS, Flash,
- applets, etc.) which operates in a security sandbox. Such code
- can only connect to the same server from which the enclosing
- HTML page originated. Pazpar2s proxy functionality enables you
- to host all of the main pages (plus images, CSS, etc) of your
- application on a conventional webserver, while efficiently
- processing webservice requests for metasearch status, results,
- etc.
+ The type of data element. This value governs any
+ normalization or special processing that might take
+ place on an element. Possible values are 'generic'
+ (basic string), 'year' (a range is computed if
+ multiple years are found in the record). Note: This
+ list is likely to increase in the future.
</para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>service</term>
- <listitem>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry><term>brief</term>
+ <listitem>
<para>
- This nested element controls the behavior of pazpar2 with
- respect to your data model. In pazpar2, incoming records are
- normalized, using XSLT, into an internal representation (see
- the <link
- linkend="config-retrievalprofile">retrievalprofile</link> secion.
- The 'service' section controls the further processing and
- extraction of data from the internal representation, primarily
- through the 'metdata' sub-element.
+ If this is set to 'yes', then the data element is
+ includes in brief records in the webservice API. Note
+ that this only makes sense for metadata elements that
+ are merged (see below). The default value is 'no'.
</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry><term>sortkey</term>
+ <listitem>
+ <para>
+ Specifies that this data element is to be used for
+ sorting. The possible values are 'numeric' (numeric
+ value), 'skiparticle' (string; skip common, leading
+ articles), and 'no' (no sorting). The default value is
+ 'no'.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry><term>rank</term>
+ <listitem>
+ <para>
+ Specifies that this element is to be used to
+ help rank
+ records against the user's query (when ranking is
+ requested). The value is an integer, used as a
+ multiplier against the basic TF*IDF score. A value of
+ 1 is the base, higher values give additional
+ weight to
+ elements of this type. The default is '0', which
+ excludes this element from the rank calculation.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry><term>termlist</term>
+ <listitem>
+ <para>
+ Specifies that this element is to be used as a
+ termlist, or browse facet. Values are tabulated from
+ incoming records, and a highscore of values (with
+ their associated frequency) is made available to the
+ client through the webservice API.
+ The possible values
+ are 'yes' and 'no' (default).
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry><term>merge</term>
+ <listitem>
+ <para>
+ This governs whether, and how elements are extracted
+ from individual records and merged into cluster
+ records. The possible values are: 'unique' (include
+ all unique elements), 'longest' (include only the
+ longest element (strlen), 'range' (calculate a range
+ of values across all matching records), 'all' (include
+ all elements), or 'no' (don't merge; this is the
+ default);
+ </para>
+ </listitem>
+ </varlistentry>
- <variablelist> <!-- Level 2 -->
- <varlistentry><term>metadata</term>
- <listitem>
- <para>
- One of these elements is required for every data element in
- the internal representation of the record (see
- <xref linkend="data_model"/>. It governs
- subsequent processing as pertains to sorting, relevance
- ranking, merging, and display of data elements. It supports
- the following attributes:
- </para>
-
- <variablelist> <!-- level 3 -->
- <varlistentry><term>name</term>
- <listitem>
- <para>
- This is the name of the data element. It is matched
- against the 'type' attribute of the 'metadata' element
- in the normalized record. A warning is produced if
- metdata elements with an unknown name are found in the
- normalized record. This name is also used to represent
- data elements in the records returned by the
- webservice API, and to name sort lists and browse
- facets.
- </para>
- </listitem>
- </varlistentry>
+ <varlistentry><term>mergekey</term>
+ <listitem>
+ <para>
+ If set to '<literal>required</literal>', the value of this
+ metadata element is appended to the resulting mergekey if
+ the metadata is present in a record instance.
+ If the metadata element is not present, the a unique mergekey
+ will be generated instead.
+ </para>
+ <para>
+ If set to '<literal>optional</literal>', the value of this
+ metadata element is appended to the resulting mergekey if the
+ the metadata is present in a record instance. If the metadata
+ is not present, it will be empty.
+ </para>
+ <para>
+ If set to '<literal>no</literal>' or the mergekey attribute is
+ omitted, the metadata will not be used in the creation of a
+ mergekey.
+ </para>
+ </listitem>
+ </varlistentry>
- <varlistentry><term>type</term>
- <listitem>
- <para>
- The type of data element. This value governs any
- normalization or special processing that might take
- place on an element. Possible values are 'generic'
- (basic string), 'year' (a range is computed if
- multiple years are found in the record). Note: This
- list is likely to increase in the future.
- </para>
- </listitem>
- </varlistentry>
+ <varlistentry><term>setting</term>
+ <listitem>
+ <para>
+ This attribute allows you to make use of static database
+ settings in the processing of records. Three possible values
+ are allowed. 'no' is the default and doesn't do anything.
+ 'postproc' copies the value of a setting with the same name
+ into the output of the normalization stylesheet(s). 'parameter'
+ makes the value of a setting with the same name available
+ as a parameter to the normalization stylesheet, so you
+ can further process the value inside of the stylesheet, or use
+ the value to decide how to deal with other data values.
+ </para>
+ <para>
+ The purpose of using settings in this way can either be to
+ control the behavior of normalization stylesheet in a database-
+ dependent way, or to easily make database-dependent values
+ available to display-logic in your user interface, without having
+ to implement complicated interactions between the user interface
+ and your configuration system.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist> <!-- attributes to metadata -->
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>relevance</term>
+ <listitem>
+ <para>
+ Specifies ICU tokenization and transformation rules
+ for tokens that are used in Pazpar2's relevance ranking.
+ The 'id' attribute is currently not used, and the 'locale'
+ attribute must be set to one of the locale strings
+ defined in ICU. The child elements listed below can be
+ in any order, except the 'index' element which logically
+ belongs to the end of the list. The stated tokenization,
+ transformation and charmapping instructions are performed
+ in order from top to bottom.
+ </para>
+ <variablelist> <!-- Level 2 -->
+ <varlistentry><term>casemap</term>
+ <listitem>
+ <para>
+ The attribute 'rule' defines the direction of the
+ per-character casemapping, allowed values are "l"
+ (lower), "u" (upper), "t" (title).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry><term>transform</term>
+ <listitem>
+ <para>
+ Normalization and transformation of tokens follows
+ the rules defined in the 'rule' attribute. For
+ possible values we refer to the extensive ICU
+ documentation found at the
+ <ulink url="&url.icu.transform;">ICU
+ transformation</ulink> home page. Set filtering
+ principles are explained at the
+ <ulink url="&url.icu.unicode.set;">ICU set and
+ filtering</ulink> page.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry><term>tokenize</term>
+ <listitem>
+ <para>
+ Tokenization is the only rule in the ICU chain
+ which splits one token into multiple tokens. The
+ 'rule' attribute may have the following values:
+ "s" (sentence), "l" (line-break), "w" (word), and
+ "c" (character), the later probably not being
+ very useful in a pruning Pazpar2 installation.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ <para>
+ From Pazpar2 version 1.1 the ICU wrapper from YAZ is used.
+ Refer to the <ulink url="&url.yaz.yaz-icu;">yaz-icu</ulink>
+ utility for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>sort</term>
+ <listitem>
+ <para>
+ Specifies ICU tokenization and transformation rules
+ for tokens that are used in Pazpar2's sorting. The contents
+ is similar to that of <literal>relevance</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>mergekey</term>
+ <listitem>
+ <para>
+ Specifies ICU tokenization and transformation rules
+ for tokens that are used in Pazpar2's mergekey. The contents
+ is similar to that of <literal>relevance</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
- <varlistentry><term>brief</term>
- <listitem>
- <para>
- If this is set to 'yes', then the data element is
- includes in brief records in the webservice API. Note
- that this only makes sense for metadata elements that
- are merged (see below). The default value is 'no'.
- </para>
- </listitem>
- </varlistentry>
+ <varlistentry>
+ <term>settings</term>
+ <listitem>
+ <para>
+ Specifies target settings for this service. Refer to
+ <xref linkend="target_settings"/>.
+ </para>
+ </listitem>
+ </varlistentry>
- <varlistentry><term>sortkey</term>
- <listitem>
- <para>
- Specifies that this data element is to be used for
- sorting. The possible values are 'numeric' (numeric
- value), 'skiparticle' (string; skip common, leading
- articles), and 'no' (no sorting). The default value is
- 'no'.
- </para>
- </listitem>
- </varlistentry>
+ <varlistentry>
+ <term>timeout</term>
+ <listitem>
+ <para>
+ Specifies timeout parameters for this service.
+ The <literal>timeout</literal>
+ element supports the following attributes:
+ <literal>session</literal>, <literal>z3950_operation</literal>,
+ <literal>z3950_session</literal> which specifies
+ 'session timeout', 'Z39.50 operation timeout',
+ 'Z39.50 session timeout' respectively. The Z39.50 operation
+ timeout is the time Pazpar2 will wait for an active Z39.50/SRU
+ operation before it gives up (times out). The Z39.50 session
+ time out is the time Pazpar2 will keep the session alive for
+ an idle session (no operation).
+ </para>
+ <para>
+ The following is recommended but not required:
+ z3950_operation (30) < session (60) < z3950_session (180) .
+ The default values are given in parantheses.
+ </para>
+ </listitem>
+ </varlistentry>
- <varlistentry><term>rank</term>
- <listitem>
- <para>
- Specifies that this element is to be used to help rank
- records against the user's query (when ranking is
- requested). The value is an integer, used as a
- multiplier against the basic TF*IDF score. A value of
- 1 is the base, higher values give additional weight to
- elements of this type. The default is '0', which
- excludes this element from the rank calculation.
- </para>
- </listitem>
- </varlistentry>
+ </variablelist> <!-- Data elements in service directive -->
+ </listitem>
+ </varlistentry>
+
+ </variablelist> <!-- Data elements in server directive -->
+ </refsect2>
- <varlistentry><term>termlist</term>
- <listitem>
- <para>
- Specifies that this element is to be used as a
- termlist, or browse facet. Values are tabulated from
- incoming records, and a highscore of values (with
- their associated frequency) is made available to the
- client through the webservice API. The possible values
- are 'yes' and 'no' (default).
- </para>
- </listitem>
- </varlistentry>
+ </refsect1>
+
+ <refsect1><title>EXAMPLE</title>
+ <para>Below is a working example configuration:
+ <screen><![CDATA[
+ <?xml version="1.0" encoding="UTF-8"?>
+ <pazpar2 xmlns="http://www.indexdata.com/pazpar2/1.0">
+
+ <threads number="10"/>
+ <server>
+ <listen port="9004"/>
+ <service>
+ <metadata name="title" brief="yes" sortkey="skiparticle"
+ merge="longest" rank="6"/>
+ <metadata name="isbn" merge="unique"/>
+ <metadata name="date" brief="yes" sortkey="numeric"
+ type="year" merge="range" termlist="yes"/>
+ <metadata name="author" brief="yes" termlist="yes"
+ merge="longest" rank="2"/>
+ <metadata name="subject" merge="unique" termlist="yes" rank="3"/>
+ <metadata name="url" merge="unique"/>
+ <relevance>
+ <icu_chain id="relevance" locale="el">
+ <transform rule="[:Control:] Any-Remove"/>
+ <tokenize rule="l"/>
+ <transform rule="[[:WhiteSpace:][:Punctuation:]] Remove"/>
+ <casemap rule="l"/>
+ </icu_chain>
+ </relevance>
+ <settings src="mysettings"/>
+ <timeout session="60"/>
+ <service>
+ </server>
+ </pazpar2>
+ ]]></screen>
+ </para>
+ </refsect1>
- <varlistentry><term>merge</term>
- <listitem>
- <para>
- This governs whether, and how elements are extracted
- from individual records and merged into cluster
- records. The possible values are: 'unique' (include
- all unique elements), 'longest' (include only the
- longest element (strlen), 'range' (calculate a range
- of values across al matching records), 'all' (include
- all elements), or 'no' (don't merge; this is the
- default);
- </para>
- </listitem>
- </varlistentry>
- </variablelist> <!-- attributes to metadata -->
+ <refsect1 id="config-include"><title>INCLUDE FACILITY</title>
+ <para>
+ The XML configuration may be partitioned into multiple files by using
+ the <literal>include</literal> element which takes a single attribute,
+ <literal>src</literal>. The of the <literal>src</literal> attribute is
+ regular Shell like glob-pattern. For example,
+ <screen><![CDATA[
+ <include src="/etc/pazpar2/conf.d/*.xml"/>
+ ]]></screen>
+ </para>
+ <para>
+ The include facility requires Pazpar2 version 1.2.
+ </para>
+ </refsect1>
- </listitem>
- </varlistentry>
- </variablelist> <!-- Data elements in service directive -->
- </listitem>
- </varlistentry>
- </variablelist> <!-- Data elements in server directive -->
- </refsect2>
+ <refsect1 id="target_settings"><title>TARGET SETTINGS</title>
+ <para>
+ Pazpar2 features a cunning scheme by which you can associate various
+ kinds of attributes, or settings with search targets. This can be done
+ through XML files which are read at startup; each file can associate
+ one or more settings with one or more targets. The file format is generic
+ in nature, designed to support a wide range of application requirements. The
+ settings can be purely technical things, like, how to perform a title
+ search against a given target, or it can associate arbitrary name=value
+ pairs with groups of targets -- for instance, if you would like to
+ place all commercial full-text bases in one group for selection
+ purposes, or you would like to control what targets are accessible
+ to users by default. Per-database settings values can even be used
+ to drive sorting, facet/termlist generation, or end-user interface display
+ logic.
+ </para>
+
+ <para>
+ During startup, Pazpar2 will recursively read a specified directory
+ (can be identified in the pazpar2.cfg file or on the command line), and
+ process any settings files found therein.
+ </para>
+
+ <para>
+ Clients of the Pazpar2 webservice interface can selectively override
+ settings for individual targets within the scope of one session. This
+ can be used in conjunction with an external authentication system to
+ determine which resources are to be accessible to which users. Pazpar2
+ itself has no notion of end-users, and so can be used in conjunction
+ with any type of authentication system. Similarly, the authentication
+ tokens submitted to access-controlled search targets can similarly be
+ overridden, to allow use of Pazpar2 in a consortial or multi-library
+ environment, where different end-users may need to be represented to
+ some search targets in different ways. This, again, can be managed
+ using an external database or other lookup mechanism. Setting overrides
+ can be performed either using the
+ <link linkend="command-init">init</link> or the
+ <link linkend="command-settings">settings</link> webservice
+ command.
+ </para>
+
+ <para>
+ In fact, every setting that applies to a database (except pz:id, which
+ can only be used for filtering targets to use for a search) can be overridden
+ on a per-session basis. This allows the client to override specific CCL fields
+ for searching, etc., to meet the needs of a session or user.
+ </para>
+
+ <para>
+ Finally, as an extreme case of this, the webservice client can
+ introduce entirely new targets, on the fly, as part of the
+ <link linkend="command-init">init</link> or
+ <link linkend="command-settings">settings</link> command.
+ This is useful if you desire to manage information
+ about your search targets in a separate application such as a database.
+ You do not need any static settings file whatsoever to run Pazpar2 -- as
+ long as the webservice client is prepared to supply the necessary
+ information at the beginning of every session.
+ </para>
+
+ <note>
+ <para>
+ The following discussion of practical issues related to session and settings
+ management are cast in terms of a user interface based on Ajax/Javascript
+ technology. It would apply equally well to many other kinds of browser-based logic.
+ </para>
+ </note>
+
+ <para>
+ Typically, a Javascript client is not allowed to directly alter the parameters
+ of a session. There are two reasons for this. One has to do with access
+ to information; typically, information about a user will be stored in a
+ system on the server side, or it will be accessible in some way from the server.
+ However, since the Javascript client cannot be entirely trusted (some hostile
+ agent might in fact 'pretend' to be a regular ws client), it is more robust
+ to control session settings from scripting that you run as part of your
+ webserver. Typically, this can be handled during the session initialization,
+ as follows:
+ </para>
+
+ <para>
+ Step 1: The Javascript client loads, and asks the webserver for a new Pazpar2
+ session ID. This can be done using a Javascript call, for instance. Note that
+ it is possible to submit Ajax HTTPXmlRequest calls either to Pazpar2 or to the
+ webserver that Pazpar2 is proxying for. See (XXX Insert link to Pazpar2 protocol).
+ </para>
+
+ <para>
+ Step 2: Code on the webserver authenticates the user, by database lookup,
+ LDAP access, NCIP, etc. Determines which resources the user has access to,
+ and any user-specific parameters that are to be applied during this session.
+ </para>
+
+ <para>
+ Step 3: The webserver initializes a new Pazpar2 settings, and sets user-specific
+ parameters as necessary, using the init webservice command. A new session ID is
+ returned.
+ </para>
+
+ <para>
+ Step 4: The webserver returns this session ID to the Javascript client, which then
+ uses the session ID to submit searches, show results, etc.
+ </para>
+
+ <para>
+ Step 5: When the Javascript client ceases to use the session, Pazpar2 destroys
+ any session-specific information.
+ </para>
- <refsect2 id="config-queryprofile"><title>queryprofile</title>
+ <refsect2><title>SETTINGS FILE FORMAT</title>
+ <para>
+ Each file contains a root element named <settings>. It may
+ contain one or more <set> elements. The settings and set
+ elements may contain the following attributes. Attributes in the set node
+ overrides those in the setting root element. Each set node must
+ specify (directly, or inherited from the parent node) at least a
+ target, name, and value.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term>target</term>
+ <listitem>
<para>
- At the moment, this directive is ignored; there is one global
- CCL-mapping file which governs the mapping of queries to Z39.50
- type-1. This file is located in etc/default.bib. This will change
- shortly.
+ This specifies the search target to which this setting should be
+ applied. Targets are identified by their Z39.50 URL, generally
+ including the host, port, and database name, (e.g.
+ <literal>bagel.indexdata.com:210/marc</literal>).
+ Two wildcard forms are accepted:
+ * (asterisk) matches all known targets;
+ <literal>bagel.indexdata.com:210/*</literal> matches all
+ known databases on the given host.
</para>
- </refsect2>
-
- <refsect2 id="config_retrievalprofile"><title>retrievalprofile</title>
<para>
- Note: In the present version, there is a single retrieval
- profile. However, in a future release, it will be possible to
- associate unique retrieval profiles with different targets, or to
- generate retrieval profiles using XSLT from the ZeeRex description of
- a target.
+ A precedence system determines what happens if there are
+ overlapping values for the same setting name for the same
+ target. A setting for a specific target name overrides a
+ setting which specifies target using a wildcard. This makes it
+ easy to set defaults for all targets, and then override them
+ for specific targets or hosts. If there are
+ multiple overlapping settings with the same name and target
+ value, the 'precedence' attribute determines what happens.
</para>
-
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>name</term>
+ <listitem>
<para>
- The following data elements are recognized for the retrievalprofile
- directive:
+ The name of the setting. This can be anything you like.
+ However, Pazpar2 reserves a number of setting names for
+ specific purposes, all starting with 'pz:', and it is a good
+ idea to avoid that prefix if you make up your own setting
+ names. See below for a list of reserved variables.
</para>
-
- <variablelist>
- <varlistentry><term>requestsyntax</term>
- <listitem>
- <para>
- This element specifies the request syntax to be used in queries. It only
- makes sense for Z39.50-type targets.
- </para>
- </listitem>
- </varlistentry>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>value</term>
+ <listitem>
+ <para>
+ The value of the setting. Generally, this can be anything you
+ want -- however, some of the reserved settings may expect
+ specific kinds of values.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>precedence</term>
+ <listitem>
+ <para>
+ This should be an integer. If not provided, the default value
+ is 0. If two (or more) settings have the same content for
+ target and name, the precedence value determines the outcome.
+ If both settings have the same precedence value, they are both
+ applied to the target(s). If one has a higher value, then the
+ value of that setting is applied, and the other one is ignored.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ By setting defaults for target, name, or value in the root
+ settings node, you can use the settings files in many different
+ ways. For instance, you can use a single file to set defaults for
+ many different settings, like search fields, retrieval syntaxes,
+ etc. You can have one file per server, which groups settings for
+ that server or target. You could also have one file which associates
+ a number of targets with a given setting, for instance, to associate
+ many databases with a given category or class that makes sense
+ within your application.
+ </para>
+
+ <para>
+ The following examples illustrate uses of the settings system to
+ associate settings with targets to meet different requirements.
+ </para>
+
+ <para>
+ The example below associates a set of default values that can be
+ used across many targets. Note the wildcard for targets.
+ This associates the given settings with all targets for which no
+ other information is provided.
+ <screen><![CDATA[
+<settings target="*">
- <varlistentry><term>nativesyntax</term>
- <listitem>
- <para>
- This element specifies the native syntax and encoding of the
- result records. The default is XML. The following attributes
- are defined:
- </para>
- <variablelist>
- <varlistentry><term>name</term>
- <listitem>
- <para>
- The name of the syntax. Currently recognized values are
- 'iso2709' (MARC), and 'xml'.
- </para>
- </listitem>
- </varlistentry>
+ <!-- This file introduces default settings for pazpar2 -->
- <varlistentry><term>format</term>
- <listitem>
- <para>
- The format, or schema, to be expected. Default is
- 'marc21'.
- </para>
- </listitem>
- </varlistentry>
+ <!-- mapping for unqualified search -->
+ <set name="pz:cclmap:term" value="u=1016 t=l,r s=al"/>
- <varlistentry><term>encoding</term>
- <listitem>
- <para>
- The encoding of the response record. Typical values for
- MARC records are 'marc8' (general MARC-8), 'marc8s'
- (MARC-8, but maps to precomposed UTF-8 characters, more
- suitable for use in web browsers), 'latin1'.
- </para>
- </listitem>
- </varlistentry>
+ <!-- field-specific mappings -->
+ <set name="pz:cclmap:ti" value="u=4 s=al"/>
+ <set name="pz:cclmap:su" value="u=21 s=al"/>
+ <set name="pz:cclmap:isbn" value="u=7"/>
+ <set name="pz:cclmap:issn" value="u=8"/>
+ <set name="pz:cclmap:date" value="u=30 r=r"/>
- <varlistentry><term>mapto</term>
- <listitem>
- <para>
- Specifies the flavor of MARCXML to map results to.
- Default is 'marcxml'. 'marcxchange' is also possible, and
- useful for Danish DANMARC records.
- </para>
- </listitem>
- </varlistentry>
- </variablelist> <!-- parameters to nativesyntax directive -->
- </listitem>
- </varlistentry>
- </variablelist> <!-- sub-elements in retrievalprofile -->
- </refsect2>
+ <!-- Retrieval settings -->
- </refsect1>
-
- <refsect1><title>EXAMPLE</title>
- <para>Below is a working example configuration:
- <screen><![CDATA[
-<?xml version="1.0" encoding="UTF-8"?>
-<pazpar2 xmlns="http://www.indexdata.com/pazpar2/1.0">
+ <set name="pz:requestsyntax" value="marc21"/>
+ <set name="pz:elements" value="F"/>
-<server>
- <listen port="9004"/>
- <proxy host="us1.indexdata.com"/>
+ <!-- Query encoding -->
+ <set name="pz:queryencoding" value="iso-8859-1"/>
- <service>
- <metadata name="title" brief="yes" sortkey="skiparticle" merge="longest" rank="6"/>
- <metadata name="isbn" merge="unique"/>
- <metadata name="date" brief="yes" sortkey="numeric" type="year" merge="range"
- termlist="yes"/>
- <metadata name="author" brief="yes" termlist="yes" merge="longest" rank="2"/>
- <metadata name="subject" merge="unique" termlist="yes" rank="3"/>
- <metadata name="url" merge="unique"/>
- </service>
-</server>
+ <!-- Result normalization settings -->
-<queryprofile/> <!-- Like a CCL profile++ . Can optionally refer to XSLT to
- convert ZeeRex into queryprofile. Multiple profiles can exist. -->
+ <set name="pz:nativesyntax" value="iso2709"/>
+ <set name="pz:xslt" value="../etc/marc21.xsl"/>
-<retrievalprofile>
- <requestsyntax>marc21</requestsyntax>
- <nativesyntax name="iso2709" format="marc21" encoding="marc8s" mapto="marcxml"/>
- <map type="xslt" stylesheet="marc21.xsl"/>
-</retrievalprofile>
+</settings>
-</pazpar2>
-]]></screen>
+ ]]></screen>
</para>
- </refsect1>
+
+ <para>
+ The next example shows certain settings overridden for one target,
+ one which returns XML records containing DublinCore elements, and
+ which furthermore requires a username/password.
+ <screen><![CDATA[
+<settings target="funkytarget.com:210/db1">
+ <set name="pz:requestsyntax" value="xml"/>
+ <set name="pz:nativesyntax" value="xml"/>
+ <set name="pz:xslt" value="../etc/dublincore.xsl"/>
+
+ <set name="pz:authentication" value="myuser/password"/>
+</settings>
+ ]]></screen>
+ </para>
+
+ <para>
+ The following example associates a specific name/value combination
+ with a number of targets. The targets below are access-restricted,
+ and can only be used by users with special credentials.
+ <screen><![CDATA[
+<settings name="pz:allow" value="0">
+ <set target="funkytarget.com:210/*"/>
+ <set target="commercial.com:2100/expensiveDb"/>
+</settings>
+ ]]></screen>
+ </para>
+
+ </refsect2>
+
+ <refsect2><title>RESERVED SETTING NAMES</title>
+ <para>
+ The following setting names are reserved by Pazpar2 to control the
+ behavior of the client function.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term>pz:cclmap:xxx</term>
+ <listitem>
+ <para>
+ This establishes a CCL field definition or other setting, for
+ the purpose of mapping end-user queries. XXX is the field or
+ setting name, and the value of the setting provides parameters
+ (e.g. parameters to send to the server, etc.). Please consult
+ the YAZ manual for a full overview of the many capabilities of
+ the powerful and flexible CCL parser.
+ </para>
+ <para>
+ Note that it is easy to establish a set of default parameters,
+ and then override them individually for a given target.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry id="requestsyntax">
+ <term>pz:requestsyntax</term>
+ <listitem>
+ <para>
+ This specifies the record syntax to use when requesting
+ records from a given server. The value can be a symbolic name like
+ marc21 or xml, or it can be a Z39.50-style dot-separated OID.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>pz:elements</term>
+ <listitem>
+ <para>
+ The element set name to be used when retrieving records from a
+ server.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>pz:piggyback</term>
+ <listitem>
+ <para>
+ Piggybacking enables the server to retrieve records from the
+ server as part of the search response in Z39.50. Almost all
+ servers support this (or fail it gracefully), but a few
+ servers will produce undesirable results.
+ Set to '1' to enable piggybacking, '0' to disable it. Default
+ is 1 (piggybacking enabled).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>pz:nativesyntax</term>
+ <listitem>
+ <para>
+ Specifies how Pazpar2 shoule map retrieved records to XML. Currently
+ supported values are <literal>xml</literal>,
+ <literal>iso2709</literal> and <literal>txml</literal>.
+ </para>
+ <para>
+ The value <literal>iso2709</literal> makes Pazpar2 convert retrieved
+ MARC records to MARCXML. In order to convert to XML, the exact
+ chacater set of the MARC must be known (if not, the resulting
+ XML is probably not well-formed). The character set may be
+ specified by adding:
+ <literal>;charset=</literal><replaceable>charset</replaceable> to
+ <literal>iso2709</literal>. If omitted, a charset of
+ MARC-8 is assumed. This is correct for most MARC21/USMARC records.
+ </para>
+ <para>
+ The value <literal>txml</literal> is like <literal>iso2709</literal>
+ except that records are converted to TurboMARC instead of MARCXML.
+ </para>
+ <para>
+ The value <literal>xml</literal> is used if Pazpar2 retrieves
+ records that are already XML (no conversion takes place).
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>pz:queryencoding</term>
+ <listitem>
+ <para>
+ The encoding of the search terms that a target accepts. Most
+ targets do not honor UTF-8 in which case this needs to be specified.
+ Each term in a query will be converted if this setting is given.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>pz:negotiation_charset</term>
+ <listitem>
+ <para>
+ Sets character set for Z39.50 negotiation. Most targets do not support
+ this, and some will even close connection if set (crash on server
+ side or similar). If set, you probably want to set it to
+ <literal>UTF-8</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>pz:xslt</term>
+ <listitem>
+ <para>
+ Is a comma separated list of of files that specifies
+ how to convert incoming records to the internal representation.
+ </para>
+ <para>
+ The suffix of each file specifies the kind of tranformation.
+ Suffix "<literal>.xsl</literal>" makes an XSL transform. Suffix
+ "<literal>.mmap</literal>" will use the MMAP transform (described below).
+ </para>
+ <para>
+ The special value "<literal>auto</literal>" will use a file
+ which is the <link linkend="requestsyntax">pz:requestsyntax's</link>
+ value followed by
+ <literal>'.xsl'</literal>.
+ </para>
+ <para>
+ When mapping MARC records, XSLT can be bypassed for increased
+ performance with the alternate "MARC map" format. Provide the
+ path of a file with extension ".mmap" containing on each line:
+ <programlisting>
+ <field> <subfield> <metadata element></programlisting>
+ For example:
+ <programlisting>
+ 245 a title
+ 500 $ description
+ 773 * citation</programlisting>
+ To map the field value specify a subfield of '$'. To store a
+ concatenation of all subfields, specify a subfield of '*'.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>pz:authentication</term>
+ <listitem>
+ <para>
+ Sets an authentication string for a given server. See the section on
+ authorization and authentication for discussion.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>pz:allow</term>
+ <listitem>
+ <para>
+ Allows or denies access to the resources it is applied to. Possible
+ values are '0' and '1'. The default is '1' (allow access to this resource).
+ See the manual section on authorization and authentication for discussion
+ about how to use this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>pz:maxrecs</term>
+ <listitem>
+ <para>
+ Controls the maximum number of records to be retrieved from a
+ server. The default is 100.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>pz:id</term>
+ <listitem>
+ <para>
+ This setting can't be 'set' -- it contains the ID (normally
+ ZURL) for a given target, and is useful for filtering --
+ specifically when you want to select one or more specific
+ targets in the search command.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>pz:zproxy</term>
+ <listitem>
+ <para>
+ The 'pz:zproxy' setting has the value syntax
+ 'host.internet.adress:port', it is used to tunnel Z39.50
+ requests through the named Z39.50 proxy.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>pz:apdulog</term>
+ <listitem>
+ <para>
+ If the 'pz:apdulog' setting is defined and has other value than 0,
+ then Z39.50 APDUs are written to the log.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>pz:sru</term>
+ <listitem>
+ <para>
+ This setting enables
+ <ulink url="&url.sru;">SRU</ulink>/<ulink url="&url.solr;">SOLR</ulink>
+ support.
+ It has four possible settings.
+ 'get', enables SRU access through GET requests. 'post' enables SRU/POST
+ support, less commonly supported, but useful if very large requests are
+ to be submitted. 'srw' enables the SRW (SRU over SOAP) variation of
+ the protocol.
+ </para>
+ <para>
+ A value of 'solr' anables SOLR client support. This is supported
+ for Pazpar version 1.5.0 and later.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>pz:sru_version</term>
+ <listitem>
+ <para>
+ This allows SRU version to be specified. If unset Pazpar2
+ will the default of YAZ (currently 1.2). Should be set
+ to 1.1 or 1.2.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>pz:pqf_prefix</term>
+ <listitem>
+ <para>
+ Allows you to specify an arbitrary PQF query language substring.
+ The provided string is prefixed the user's query after it has been
+ normalized to PQF internally in pazpar2.
+ This allows you to attach complex 'filters' to queries for a given
+ target, sometimes necessary to select sub-catalogs
+ in union catalog systems, etc.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>pz:pqf_strftime</term>
+ <listitem>
+ <para>
+ Allows you to extend a query with dates and operators.
+ The provided string allows certain substitutions and serves as a
+ format string.
+ The special two character sequence '%%' gets converted to the
+ original query. Other characters leading with the percent sign are
+ conversions supported by strftime.
+ All other characters are copied verbatim. For example, the string
+ <literal>@and @attr 1=30 @attr 2=3 %Y %%</literal>
+ would search for current year combined with the original PQF (%%).
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>pz:sort</term>
+ <listitem>
+ <para>
+ Specifies sort criteria to be applied to the result set.
+ Only works for targets which support the sort service.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>pz:recordfilter</term>
+ <listitem>
+ <para>
+ Specifies a filter which allows Pazpar2 to only include
+ records that meet a certain criteria in a result. Unmatched records
+ will be ignored. The filter takes the form name[~value] , which
+ will include only records with metadata element (name) that has the
+ substring (value) given. If value is omitted all records with the
+ metadata present will be included.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ </refsect2>
+
+ </refsect1>
+ <refsect1><title>SEE ALSO</title>
+ <para>
+ <citerefentry>
+ <refentrytitle>pazpar2</refentrytitle>
+ <manvolnum>8</manvolnum>
+ </citerefentry>
+ <citerefentry>
+ <refentrytitle>yaz-icu</refentrytitle>
+ <manvolnum>1</manvolnum>
+ </citerefentry>
+ <citerefentry>
+ <refentrytitle>pazpar2_protocol</refentrytitle>
+ <manvolnum>7</manvolnum>
+ </citerefentry>
+ </para>
+ </refsect1>
</refentry>
<!-- Keep this comment at the end of the file
Local variables: