X-Git-Url: http://lists.indexdata.dk/cgi-bin?a=blobdiff_plain;f=doc%2Frecordmodel-domxml.xml;h=2c899769505e9474d9af761a2092d9d273b73895;hb=bd797d70f666280cdf941337d86b438a8d2506fc;hp=8dfcdb6cfb49102c770f5752eb80d7b6015a2724;hpb=799d116b6a50e44e9eeccf17ed81ffb70220b4f4;p=idzebra-moved-to-github.git diff --git a/doc/recordmodel-domxml.xml b/doc/recordmodel-domxml.xml index 8dfcdb6..2c89976 100644 --- a/doc/recordmodel-domxml.xml +++ b/doc/recordmodel-domxml.xml @@ -1,5 +1,5 @@ - + &dom; &xml; Record Model and Filter Module @@ -216,50 +216,107 @@
Extract pipeline + + The <extact> pipeline takes documents + from any common &dom; &xml; format to the &zebra; specific + indexing &dom; &xml; format. + It may consist of zero ore more + ]]> + &xslt; transformations, and the outcome is handled to the + &zebra; core to drive the proces of building the inverted + indexes. See + for + details. +
Store pipeline -
+ The <store> pipeline takes documents + from any common &dom; &xml; format to the &zebra; specific + storage &dom; &xml; format. + It may consist of zero ore more + ]]> + &xslt; transformations, and the outcome is handled to the + &zebra; core for deposition into the internal storage system. +
Retrieve pipeline - - All named stylesheets defined inside - schema element tags - are for presentation after search, including - the indexing stylesheet (which is a great debugging help). The - names defined in the name attributes must be - unique, these are the literal schema or + Finally, there may be one or more + <retrieve> pipeline definitions, each + of them again consisting of zero or more + ]]> + &xslt; transformations. These are used for document + presentation after search, and take the internal storage &dom; + &xml; to the requested output formats during record present + requests. + + + The possible multiple + <retrieve> pipeline definitions + are distinguished by their unique name + attributes, these are the literal schema or element set names used in &srw;, &sru; and - &z3950; protocol queries. + &z3950; protocol queries.
-
- &dom; filter internal record representation - When indexing, an &xml; Reader is invoked to split the input - files into suitable record &xml; pieces. Each record piece is then - transformed to an &xml; &dom; structure, which is essentially the - record model. Only &xslt; transformations can be applied during - index, search and retrieval. Consequently, output formats are - restricted to whatever &xslt; can deliver from the record &xml; - structure, be it other &xml; formats, HTML, or plain text. In case - you have libxslt1 running with E&xslt; support, - you can use this functionality inside the &dom; - filter configuration &xslt; stylesheets. +
+ Canonical Indexing Format + + + &dom; &xml; indexing comes in two flavors: pure + processing-instruction governed plain &xml; documents, and - very + similar to the Alvis filter indexing format - &xml; documents + containing &xml; <record> and + <index> instructions from the magic + namespace xmlns:z="http://indexdata.dk/zebra-2.0". + + +
+ Processing-instruction governed indexing format + + The output of the processing instruction driven + indexing &xslt; stylesheets must contain + processing instructions named + zebra-2.0. + The output of the &xslt; indexing transformation is then + parsed using &dom; methods, and the contained instructions are + performed on the elements and their + subtrees directly following the processing instructions. + + + For example, the output of the command + + xsltproc dom-index-pi.xsl marc-one.xml + + might look like this: + + + + + + 11224466 + + How to program a computer + + ]]> +
-
- &dom; Canonical Indexing Format +
+ Magic element governed indexing format + The output of the indexing &xslt; stylesheets must contain certain elements in the magic - xmlns:z="http://indexdata.dk/zebra/xslt/1" + xmlns:z="http://indexdata.dk/zebra-2.0" namespace. The output of the &xslt; indexing transformation is then parsed using &dom; methods, and the contained instructions are performed on the magic elements and their @@ -267,30 +324,34 @@ For example, the output of the command - - xsltproc xsl/oai2index.xsl one-record.xml + + xsltproc dom-index-element.xsl marc-one.xml might look like this: - <?xml version="1.0" encoding="UTF-8"?> - <z:record xmlns:z="http://indexdata.dk/zebra/xslt/1" - z:id="oai:JTRS:CP-3290---Volume-I" - z:rank="47896" - z:type="update"> - <z:index name="oai_identifier" type="0"> - oai:JTRS:CP-3290---Volume-I</z:index> - <z:index name="oai_datestamp" type="0">2004-07-09</z:index> - <z:index name="oai_setspec" type="0">jtrs</z:index> - <z:index name="dc_all" type="w"> - <z:index name="dc_title" type="w">Proceedings of the 4th - International Conference and Exhibition: - World Congress on Superconductivity - Volume I</z:index> - <z:index name="dc_creator" type="w">Kumar Krishen and *Calvin - Burnham, Editors</z:index> - </z:index> - </z:record> + + + 11224466 + + How to program a computer + + ]]> +
+ + +
+ Semantics of the indexing formats + + + Both indexing formats are defined with equal semantics and + behaviour in mind. + + + This means the following: From the original &xml; file one-record.xml (or from the &xml; record &dom; of the same form coming from a splitted input file), the indexing @@ -310,24 +371,30 @@ insert, update, and delete. - In this example, the following literal indexes are constructed: + + + In these examples, the following literal indexes are constructed: - oai_identifier - oai_datestamp - oai_setspec - dc_all - dc_title - dc_creator + any:w + control:w + title:w + title:p + title:s - where the indexing type is defined in the - type attribute - (any value from the standard configuration - file default.idx will do). Finally, any + where the indexing type is defined after the + literal ':' charaacter. + Any value from the standard configuration + file default.idx will do. + Finally, any text() node content recursively contained - inside the index will be filtered through the + inside the <z:index> element, or any + element following a index processing instruction, + will be filtered through the appropriate charmap for character normalization, and will be - inserted in the index. + inserted in the named indexes. + + Specific to this example, we see that the single word oai:JTRS:CP-3290---Volume-I will be literal, @@ -387,6 +454,9 @@ filter configuration files involves in this process, and that the literal index names are used during search and retrieval. + +
+