3 IPV6 updates. Allow IPv6 addresses for database hosts and IPv6 address
4 for HTTP server. By default the HTTP server uses IPv4 only, but that can be
5 changed by setting "host" attribute for the "listen" element.
7 Change semantics of pz:extendrecs. Allow for repeated fetches .
8 pz:extendrecs is now the number of extra records to fetch (was total
13 Fix hang or memory violation if show occurred before a search. (not that
14 it makes much sense to perform a show before a search).
18 pz2.js: Element_parseChildNodes concatenates all Text/CDATA nodes, instead
23 New pz:metadata attribute, empty="empty-value" for Pazpar2's
24 internal representation. With this attribute, Pazpar2 treates an empty
25 pz:metadata type as having the value for "empty" - if empty.
27 New setting, pz:extendrecs, which triggers extended fetch of records
28 for a database beyond pz:maxrecs for a show command.
30 Fix warning that was falsely issued for "missing limitmap".
32 Log message for Pazpart start/stop changed. Now using same style as
33 Metaproxy, ie Pazpar2 start SHA1 / Pazpar2 stop .
37 Fix hang of 2nd command=show with esn/syntax given.
41 New merge attribute type: 'first', which takes all metadata fields
42 from first target that returns the particular field.
46 Extend info command with hostname and YAZ SHA1
47 Indent results for both command stat and info.
49 Allow limit on merged content. The new configuration metadata
50 element, limitcluster, configures that a metadata element (name) be used
51 as limit name for search. Applies to the whole service (ie all targets),
52 unlike pz:limitmap which is configured per-target (database).
54 New feature: limitmap local:* matches against all metadata fields.
56 Allow repeated list in limitmap spec . Separated by comma. For
57 example: value="local:title,rpn:@attr 1=4".
59 New element <message> in bytarget response. Holds diagnostic message
60 of code (say 'Unsupported Use Attribute' for Bib-1 114).
62 Improved logging for record ingestion failures.
64 Avoid using struct icu_chain in non-YAZ_HAVE_ICU mode. In the rare case,
65 when YAZ is compiled without ICU support.
69 Requires YAZ-4.2.40 to support native solr support.
71 Fix and improve logic handling whether or not to re-do search on sort
72 order changes. A sort order with change in ascending/descending only
73 whould not trigger a new search, which is required for targets with
74 native sorting capabilities. Each client is now checked if instructions
75 (sortmap) exist for native sorting and only client that does requires
76 it is researched. Other clients is just re-ingesting the records,
77 they already have. The resultset is now cleared if any researching
80 Connection sharing between session has broken since version 1.6.8 with
81 introduction of logic that would minimize searching if pazpar2 could
82 detect this based on same query and limits and partly sort order.
83 This could lead to segementations violations.
85 Added a chapter in the manaul about relevance ranking.
89 Rank tweak: follow=number will increase mult by number if two terms
90 occur next to each other; number-1 if they are one term apart , .. 0
91 if they are number a part (all in order). Default is 0 (following
94 Rank tweak: lead=k will divide mult by 1 + log2(1+k*l) where k is
95 value given by lead and l is length from beginning of field where
96 term occurs (l=0 for first term, l=1 for second term, ..). Default
99 Rank tweak: length=strategy. length="linear" if mult is to be divided
100 by length (existing, default behavior), length="log" if mult is to be
101 divided by log2(1+length), length="none" if mult is not to be affected
104 --- 1.6.20 2012/09/21
106 Rank algorithm details may be printed as part of show response in
107 element <relevance_info>.. This is only printed if <rank debug="yes"/>
110 Record as returned by show/record command have a minimal indentation
111 which makes things human-readable.
113 New configuration of default sorting criteria (sort-default) in
114 service definition. If no criteria is defined it will be as before
117 Search command now supports sort parameter just as the show command.
118 If no parameter is give, it will use the service sort-default value.
120 --- 1.6.19 2012/09/18
122 Rank algorithm skips strings that gets normalized to empty string.
123 For example, & and ! could map to the empty string. The weight for
124 those terms is now 0 (as if they were not part of the query).
126 Rank algorithm does not use CCL from limitmap; only from the query
127 parameter (user query).
131 --- 1.6.18 2012/09/17
133 Rank algorithm configurable by 'rank' element inside service. So far
134 only, attribute 'cluster' is recognized. If cluster="yes", multiple
135 records inside a cluster boosts higher than single records. This
136 is default behavior and existing behavior. cluster="no" takes the
137 average score of each record in a cluster.
139 --- 1.6.17 2012/09/05
141 Fix bad re-use of connections (connections with changing proxy should not
144 --- 1.6.16 2012/08/22
146 Fix a bug introduce in 1.6.15 around the position sorting. It resetted the
147 resultset and sorting when the sort order is position. However this will be done on
148 every client poll, which will make pazpar2 continuing reset and fetching.
149 It should only be done on FIRST request where the sort order change.
151 Fix an issue on suggestion option: Also disable suggestions on empty string.
153 Clean up in turbo marc stylesheet.
155 Remove the hardcoded size of termlists.
157 --- 1.6.15 2012/06/27
159 New facility: ccldirective may be given in service definition. Allows
160 CCL parsing to be customized a bit, such as defining names of operators
163 New facility: raw record by checksum, rather than offset. The record
164 command optionally takes checksum which identifies certain record from
167 New facility: per field ranking. Rank may be given as M [F N] where
168 M is default rank and N is rank for CCL terms from field F.
170 --- 1.6.14 2012/06/04
172 Fix for IE7/8 in pz.js
174 Applied patch from Giannis Kosmas on keepAlive, which also adds keepAlive to init response.
176 Lower log level some places.
178 Remove some invalid test results.
180 --- 1.6.13 2012/05/23
182 Introducing a version=2 parameter for show, termlist and bytarget commands.
183 This enables pazpar2 to return approximation on hit and count count when
184 doing record filtering using the limit parameter on search and a
185 limitmap with a value of "local:"
187 Setting pz:xslt may embed local XSLT as an alternative to referring
189 Value is not CDATA but XML nodes embedded, so escaping is not necessary
190 but a root element *must* be present. For example:
191 <settings target="target="z3950.indexdata.com/marc">
200 Metadata field rank may given by XML internal document (pz:xslt
201 result). If rank is not given, the rank from service description is
204 Metadata field can now configured a default limitmap and facetmap.
205 Setting limitmap to "local:" would work for all kind of targets, but would
206 prob. not be the optimal solution. But at least better than the default behavior
207 of pazpar2 where no filtering is done.
209 A service definition can now also contains <set/> that defines service-wide
210 settings. These will override server-wide sets and will be overridded by
213 New setting, pz:present_chunk, that specifies number of records to fetch
214 at a time. Zero will disable chunkation; will fetch max_records at once.
216 --- 1.6.12 2012/03/14
218 Revert the format change in termlist response, that could break
219 some clients / UIs since they were expecting an (empty) element
220 if no facet values was found.
222 --- 1.6.11 2012/03/07
224 Revert the behavior of returning errors when unable to block
225 on termlist, bytarget and search, when unable to block due to
226 other block. The client will now receive a regular response,
227 but it will be logged in the server. A parameter (report) is
228 added to change behavior to return error response or WARNING
229 status message. Consider this "API" as private, as it is mostly
230 untested and could be changed in future releases.
232 Fix spell error in pz2.js fix in 1.6.10.
234 New Marc2TurboMarc.xsl (contribution from Sven Porst).
235 Can solve the missing marc21.xsl updates in some cases.
237 tmarc.xsl: Simplify the 6xx to subject-long and fix 1-based
238 substring (contribtion from Sven Porst)
240 marc21.xsl: fix 1-based substring call
242 tmarc.xsl and marc21.xsl: use 856$a as last option for electronic-text.
244 Add test_termlist_block to test suite
246 --- 1.6.10 2012/02/23
248 Fix SEGV for invalid PQFs and SRU/SOLR targets
249 Also refactor a bit the code that converts from PQF to SRU/SOLR queries.
251 Fix pz2.js: "null object" due to change in in bytarget result XML.
253 Fixes in tmarc.xsl: Subject-long shorten for extra commas only.
254 Added this normalization to the other subject-long fields (d6xx),
255 where it was missing.
257 Fixes in marc21.xsl: Updated with most of the new tmarc.xsl.
258 Still differences around medium and holdings. marc21.xsl is not
259 longer active used by Index Data, and should be considered unsupported.
260 Use tmarc.xsl instead.
264 Fix SEGV that could occur for failed connections.
268 Fix bug for command sort that could return no results for active clients
269 (from previous search). This bug was present in 1.6.6-1.6.7.
271 Fix bug in results that could include results that should have been
272 filtered out. This bug was present in 1.6.6-1.6.7.
276 Fix bug introduced in 1.6.6 where a connection re-use could stall
279 Local filtering may now specify a local metadata field, eg.
280 pz:limitmap:somefield[t]=local:otherfield
284 For search, when limit and or filtering is in place and search
285 is identical to previous search, the result set is re-used and the
286 target is not searched.
288 Limits may work perform local filtering as well, by using "local:"
293 Updated bytarget command to contain a suggestions element with misspelled
294 words and suggestions to these. pz2.js has been updated to deliver this
295 onwards as well. Only target that currently delivers this is the solr
296 client in YAZ 4.2.18.
300 New service definition element, xslt, that allows an embedded stylesheet
301 to be defined. This can be referred to from pz:xslt as an alternative to
304 New pz:sortmap:field setting for specifying hints on how to make
305 a target natively sort on a field. This is used for command=show in
306 conjunction with sort.
308 New pz:url setting for specifying the actual URL for a target. When
309 this is used the target ID is not used as URL anymore and the target ID
310 may be almost any string (not including []).
312 command=termlist without name parameter returns all termlists/facets.
313 Previously if name parameter was omitted, only "subject" was returned.
317 Make termlist sorting stable. Terms with same frequency are now sorted by
318 their display name. This makes a pretty display and improves our
319 regression test because qsort is not a stable sort.
321 New sort parameter value 'position'. The 'position' sorts merged records
322 by their original position from the remote target. This is primarily useful
323 for debugging and may be used for targets that already perform some kind
324 of relevance ranking. Note that sort by default is decreasing; so to get
325 records in their original order sort=position:1 must be used.
329 tmarc.xsl: yet another 773$g fix. Was broken in 1.6.1 as well.
331 Facility to change working directory for pazpar2 daemon. Option -wdir
332 sets working directory to dir. This facility is useful if core dumps
333 must be saved. In this case, the current working directory must be
334 writable by the running user, such as "nobody".
338 New configuration element <icu_chain> for <server>/<service> which
339 allows a named ICU rule (chain) to be defined. The names relevance,
340 sort, mergekey and facet are used for those operations. The definition
341 <icu_chain id="sort" locale="en"> .. </icu_chain>
343 <sort> <icu_chain locale="en> ... </icu_chain> </sort>
344 And so on.. for relevance, mergekey and facet as well. The latter
345 style is deprecated. The facet terms are normalized by the facet
346 rule by default. This may be changed on a metadata field basis by
347 defining the new attribute 'facetrule' for the metadata element.
349 <icu_chain id="myrule" locale="en"> ... </icu_chain>
350 <metadata name="author" termlist="yes" facetrule="myrule"/>
352 Preserve rorder for merged metadata. Fixes issue as reported by Sven
353 Porst: http://lists.indexdata.dk/pipermail/yazlist/2011-July/003230.html
355 tmarc.xsl: set journal-subpart to 773$ only.
359 Modify the behavior for the limit parameter (first defined in 1.5.7).
360 Mapping of limit searches are now defined by the new configuration item
361 pz:limitmap. Fix a dead-lock problem with the limit parameter.
363 Extend tmarc.xsl to extract 773$g data (OpenURL).
367 ICU default maps remove backquote (`).
369 Command 'search' takes limit parameter (optional). The limit parameter
370 allows a search to be limited one or more facets and the corresponding
371 values. This is for server side filtering.
373 Configure tweak: Use -lm for log(3) if needed
377 Fix a problem with skiparticle sortkey that could be completely
378 ignored (and reduced to "").
380 Fix dependency problem in pazpar2 RPM package (did not require
381 libyaz4 as it should).
385 Fix memory leak that occurred for command=termlist&name=xtargets .
387 Pazpar2 may save HTTP requests. Enabled by option -R.
391 Experimental support for DTIC DADS target. New dads-pz2.xsl.
393 Support for query_syntax (overrides the default for SRU | Z39.50)
395 Support for extraArgs (ZOOM "extraArgs" option) for targets
397 New commands: status-server and status-session
401 Fix for threaded runs: Client now have a copy of the database URL,
402 which can used after the database has been release from the client.
403 This makes the logging in the connection idle timeout of the client nicer (no NOURL) and should be thread-safe.
405 tmarc.xsl: Add journal-title-abbrev and full text.
407 cf.xsl: new fields: isbn, issn, journaltitle, volume, issue
409 Fix for cmd=record before search.
411 Session Logging clean up.
413 Fix wrong termlist factor when maxrecs is different from 100.
417 Fix missing pz:termlist_term_factor in settings.c messed up pz:preferred.
418 Term factor is default enabled but can be diseabled by
419 pz:termlist_term_factor=0
423 Add scaling of facet count. Currently always enabled, needs fixing.
424 Allow user-defined info for target suffix. This has no meaning in
425 Pazpar2 except to distinguish targets from each other. The suffix
426 data begins with #. For example z3950.indexdata.com/gils#Mydata
428 Added exact-match recordfilter; format name=value
432 SOLR support. Pazpar2 may operate as web service client for SOLR.
436 Fix for show command and block=1 (dead lock). Bug was introduced in
441 New RPM packages: pazpar2, pazpar2-js, pazpar2-doc. These have been
442 tested on CentOS 5.5 only.
446 Fix problem with result sets being removed from a client session
447 if the connection for it was resused by another session. Bug #3489.
449 New iphone UI for Pazpar2 (www/iphone).
453 Fixes for threaded operation.
455 New stylesheets for TurboMARC: tmarc.xsl and opac_turbomarc.xsl.
457 New example services in etc/services in source. In the Debian packages
458 these are located in /etc/pazpar2/services-available
460 Threaded mode operational on Windows. Requires Windows 7 or Windows
463 Default value of setting pz:max_connections is 0 which means that there
464 is no limit on number of connections.
468 Pazpar2 may operate in threaded mode. Enabled by element threads in
469 the configuration. See pazpar2_conf for details.
471 New setting setting: pz:max_connections. Setting pz:max_connections is
472 a limit of number of sockets to a host. When this limit is reached,
473 Pazpar2 will wait up to 5 seconds for a connection to becomes available.
474 The client will be marked Client_Error when it can not be searched
475 (other clients in a session may work). If pz:max_connections is not set
476 for a target, a value of 30 will be used. Note: the pz:max_connections
477 will only work in threaded mode.
479 pz2.js: JSON support for show.
481 Debian package: Enable default service, default.xml, before starting
482 Pazpar2 only if there is no service already in /etc/pazpar2/services-enabled.
486 Debian version depends on on libyaz4. Note that Pazpar2 will still
487 compile from source with YAZ 3.
489 Split services into separate files. The example configuration file
490 pazpar2.cfg.dist now includes a default service default.xml (part of
491 etc). And default.xml includes settings/edu.xml. The default.xml file,
492 not to be confused with settings/defaults.xml, is a template for jsdemo
493 and other services. The Debian package installs /etc/pazpar2/server.xml
494 which is now the main pazpar2 configuration (used to be called pazpar2.cfg).
495 server.xml includes services from /etc/pazpar2/services-enabled/*.xml .
496 The default.xml (from etc) is installed in /etc/pazpar2/services-available
497 and a symlink to it is created from services-enabled. The default.xml
498 service is unnamed and, thus, will be used by jsdemo and test1.
500 New setting pz:negotiation_charset. Patch from Andrei V. Toutoukine. The
501 new setting pz:negotiation_charset specifies character set for Z39.50 Init.
505 Support for additional fields in cf.xsl and pazpar2.conf.dist:
506 publisher, available, due, location (=locallocation), callno
507 (=callnumber), thumburl and score.
509 Describe pz:xslt and the auto setting.
511 Move mergekey definition away from the normalization stylesheets and
512 define a mergekey common for all target types in pazpar2.cfg.
514 Code update: Use the Odr_int type for hit counts. This is part of
515 YAZ 3.0.47 and later and so configure checks for that.
519 Metadata attribute 'skiparticle' also works for ICU based
520 normalization. (was only working for the non-ICU/ASCII before).
522 Command bytarget with argument settings=1 will show settings per
523 target.. This is to be able to verify correct settings and be able to
524 test that they are correct. The database settings array size is now
525 also stored.. Problems with database settings array is that if not
526 careful it will be too small (smaller than dictionary per-service
529 Make record list sorting stable by comparing mergekey for records if
530 relevance/title or other sorting criteria all match. This is merely to
531 ensure that our regressions tests works (reproducible output).
533 Relevance calculation changes: use a different denominator (length) for
534 per-field relevance scoring.. Instead of length of all ranked fields we
535 now use length of individual fields (as if they were individual "free"
536 text fields). This will ensure that documents with a long field with no
537 match (say description) will not "hurt" a title match.
539 Diagnostic member was not set on connection error. Fixed
543 Command search takes two optional parameters, startecs and maxrecs,
544 that specifies the start offset (0, 1, ...) and maximum number of records
545 to fetch for each target.
547 XSLTs + MARC maps are cached within a session so we don't re-parse
548 them over and over again. Even for a session with a single search
549 there's much to be gained because many targets use the same
552 The metadata attribute 'mergekey' now takes one of three values 'no',
553 'required', 'optional' . And the resulting mergekey from metadata
554 is now ordered in the same way as metadata in the service definition.
555 Older Pazpar2 version use the order in which metadata appeared in a
558 The search argument 'filter' now offers a new operator ~ which does a
559 substring match. The = operator works as before: string match for
560 anything but pz:id, or target match for pz:id.
562 New setting pz:recordfilter. The value of this setting takes the
563 form name[~value]. This setting makes Pazpar2 ignore all retrieved
564 records that do not have the metadata element name with value substring
567 Pazpar2 allows YAZ log level to be set (option -v).
571 For WS responses Pazpar2 creates XML header. Exception: raw record.
573 Setting XML files are now stored in etc/settings instead of etc. This
574 reflects the layout with the Debian package layout.
576 Settings may be posted for command=settings. The POSTed settings must
577 have root element 'settings' like regular setting files. In order to be
578 recognized, the POST request must use Content-Type=text/xml.
580 A service may be posted for command=init. This service will be used
581 during the session. The service may have its own target settings,
582 ICU config, timeout, etc. In order to be recognized, the POST request
583 must use Content-Type=text/xml.
585 Timeout values may be given per-service. That's element 'timeout'
586 which takes three attribute values (a subset may be given): 'session',
587 'z3950_operation', 'z3950_session'. Option -T is no longer supported
588 - used to specify session timeout.
590 Option -t tests the Pazpar2 configuration and returns exit code
591 (0=success, non-zero=failure). In previous version of Pazpar2, -t
592 specified local settings.
594 In version 1.2.0 the configuration file - after include processing -
595 was dumped to stdout. Now, the configuration is only dumped to the
596 yaz log file if option -d is given.
600 Configuration may now have multiple server areas. This means that a
601 Pazpar2 instance may listen on multiple ports. Virtual hosting is not
602 yet supported - on a server basis. Configuration may also have multiple
603 services .. That is repeating service elements inside a server. Each
604 has an attribute 'id' which serves as service ID. This ID in turn may
605 be used in a Pazpar2 session, by specifying parameter service=ID for
606 command init. There can be at most one unnamed service inside a server
607 which can be referred to by not specifying an service ID for command
608 init (backwards compatible). In order to partition multiple servers and
609 services a new include directive has been added. This takes an attribute
610 'src' which specifies one or more sub-files. For example to include
611 service files, one might use:
612 <server >.. <include src=/"etc/pazpar2/conf.d/*.xml"/> .. </server>.
613 It is the intention that that completely makes the settings directive
616 Fix problem where the record command would wait forever if there were
617 no targets to wait for (activeclients == 0).
621 One result set is created per session (last search) rather than for
622 each connection which happen to be shared (bug #3009).
624 marc21 stylesheets changed for efficiency.
628 Session timeout may be specified on the command-line as option -T.
630 Pazpar2 may now be operated in a no-merged mode for records.. All records
631 will be considered unique. This mode is enabled if no mergekey is
632 generated by the normalization stylesheet (pz:xslt).
634 Pazpar2 caches original records from each target and the 'record' command
635 with offset returns the original record if 'syntax' and 'esn' are NOT
636 specified. This speeds up retrieval of original records but also means
637 that Pazpar2 uses more memory. The cached records will be freed when the
638 session terminates or a new search is executed.
640 Pazpar2 no longer uses its own ICU wrapper. Instead the ICU wrapper
641 library part of YAZ is used.
643 Added SRU client support.
645 Automatically computes pz:nativesyntax if not provided. Works for XML and
648 --- 1.0.13 2008/11/24
650 Command bytarget returns name of target (if defined).
652 --- 1.0.12 2008/11/04
654 Fixed bug #2021.. location now holds all brief elements.
656 --- 1.0.11 2008/10/15
658 Fixed check for application/x-www-form-urlencoded parameters.
660 --- 1.0.10 2008/10/14
662 Fixes for IE in pz2.js.
664 Fixed bug #2021: non-merged, brief meta data NOT included for command=show.
668 Changed the JS library pz2.js to use POST for long URL (+ params).
670 Added installation instructions for Windows. Note: NT services is
671 NOT available until we make a new release of YAZ.
673 Preserve order of repeated metadata fields (they were reversed before).
675 More MARC21 information extracted for metadata.
679 Fixed bug #1162: HTML entities are not escaped properly.
681 Native Windows port of Pazpar2. Makefile for Visual Studio provided.
685 Marc21 stylesheet updated to reflect multiple full text fields
689 Fixed bug in pz2.js WRT DOMElement attributes on IE.
691 Fixed bug 2100: Database wildcards not working
695 Added support for retrieval of records in binary.
697 Fixed bug 1794: Pazpar2 does not return valid XML.
699 Deal with ICU not returning sortkey (resulted in SEGV before).
703 JavaScript library pzw2.js throws error if WS response (from Pazpar2 or
704 other) is malformed (non-wellformed XML or missing Pazpar2 OK status).
706 Improved diagnostics when Pazpar2 HTTP decoding fails.
708 Pazpar2 requests may be POSTed as using Content-Type
709 application/x-www-form-urlencoded.
711 Pazpar2 honors LF in HTTP headers.
713 Handle targets that handle negative hit counts (should not happen, but it
718 ICU is used for tokenization and normalization of the following: mergekey,
719 sorting, relevance terms.
721 Debian package now enables ICU tokenization and normalization by default.
725 Exposed user setting values (i.e. non-pz: names) to the record systems in two
726 ways: Either as parameters to the normalization stylesheets (which would allow the
727 programmer to postprocess or use the values in any way) or after the normalization
728 step, in which case values are made part of the normalized record (and available for
729 sorting, termlists, display, or other interface-related use.
731 Implemented sorting by year.
733 Option -d dumps records to the current log file instead of stderr.
735 Fixes for compilation on cygwin.
737 Z39.50 client code uses pz:elements. pz:elements was recognized in
738 earlier Pazpar2 versions but it was not used for anything.
740 icu_chain_test is using fgets instead of getline - fixes compilation
743 Loosen the CCL query parsing so that Pazpar2 only returns error if _all_
744 query conversions fail (rather than _any_). This means targets that do
745 not support some fields are ignored in a search.
749 Improved handling of socket timeout for Z39.50 connections.
751 Misc documentation updates and spell fixes.
753 Debian package pazpar2 creates log rotate entry.
755 Debian package pazpar2-apache2 reloads Apache2.
757 jsdemo included in distribution. It illustrates the use of the js/pz2.js
762 First public release.