3 Fix SEGV for invalid PQFs and SRU/SOLR targets
4 Also refactor a bit the code that converts from PQF to SRU/SOLR queries.
6 Fix pz2.js: "null object" due to change in in bytarget result XML.
8 Fixes in tmarc.xsl: Subject-long shorten for extra commas only.
9 Added this normalization to the other subject-long fields (d6xx),
12 Fixes in marc21.xsl: Updated with most of the new tmarc.xsl.
13 Still differences around medium and holdings. marc21.xsl is not
14 longer active used by Index Data, and should be considered unsupported.
15 Use tmarc.xsl instead.
19 Fix SEGV that could occur for failed connections.
23 Fix bug for command sort that could return no results for active clients
24 (from previous search). This bug was present in 1.6.6-1.6.7.
26 Fix bug in results that could include results that should have been
27 filtered out. This bug was present in 1.6.6-1.6.7.
31 Fix bug introduced in 1.6.6 where a connection re-use could stall
34 Local filtering may now specify a local metadata field, eg.
35 pz:limitmap:somefield[t]=local:otherfield
39 For search, when limit and or filtering is in place and search
40 is identical to previous search, the result set is re-used and the
41 target is not searched.
43 Limits may work perform local filtering as well, by using "local:"
48 Updated bytarget command to contain a suggestions element with misspelled
49 words and suggestions to these. pz2.js has been updated to deliver this
50 onwards as well. Only target that currently delivers this is the solr
55 New service definition element, xslt, that allows an embedded stylesheet
56 to be defined. This can be referred to from pz:xslt as an alternative to
59 New pz:sortmap:field setting for specifying hints on how to make
60 a target natively sort on a field. This is used for command=show in
61 conjunction with sort.
63 New pz:url setting for specifying the actual URL for a target. When
64 this is used the target ID is not used as URL anymore and the target ID
65 may be almost any string (not including []).
67 command=termlist without name parameter returns all termlists/facets.
68 Previously if name parameter was omitted, only "subject" was returned.
72 Make termlist sorting stable. Terms with same frequency are now sorted by
73 their display name. This makes a pretty display and improves our
74 regression test because qsort is not a stable sort.
76 New sort parameter value 'position'. The 'position' sorts merged records
77 by their original position from the remote target. This is primarily useful
78 for debugging and may be used for targets that already perform some kind
79 of relevance ranking. Note that sort by default is decreasing; so to get
80 records in their original order sort=position:1 must be used.
84 tmarc.xsl: yet another 773$g fix. Was broken in 1.6.1 as well.
86 Facility to change working directory for pazpar2 daemon. Option -wdir
87 sets working directory to dir. This facility is useful if core dumps
88 must be saved. In this case, the current working directory must be
89 writable by the running user, such as "nobody".
93 New configuration element <icu_chain> for <server>/<service> which
94 allows a named ICU rule (chain) to be defined. The names relevance,
95 sort, mergekey and facet are used for those operations. The definition
96 <icu_chain id="sort" locale="en"> .. </icu_chain>
98 <sort> <icu_chain locale="en> ... </icu_chain> </sort>
99 And so on.. for relevance, mergekey and facet as well. The latter
100 style is deprecated. The facet terms are normalized by the facet
101 rule by default. This may be changed on a metadata field basis by
102 defining the new attribute 'facetrule' for the metadata element.
104 <icu_chain id="myrule" locale="en"> ... </icu_chain>
105 <metadata name="author" termlist="yes" facetrule="myrule"/>
107 Preserve rorder for merged metadata. Fixes issue as reported by Sven
108 Porst: http://lists.indexdata.dk/pipermail/yazlist/2011-July/003230.html
110 tmarc.xsl: set journal-subpart to 773$ only.
114 Modify the behavior for the limit parameter (first defined in 1.5.7).
115 Mapping of limit searches are now defined by the new configuration item
116 pz:limitmap. Fix a dead-lock problem with the limit parameter.
118 Extend tmarc.xsl to extract 773$g data (OpenURL).
122 ICU default maps remove backquote (`).
124 Command 'search' takes limit parameter (optional). The limit parameter
125 allows a search to be limited one or more facets and the corresponding
126 values. This is for server side filtering.
128 Configure tweak: Use -lm for log(3) if needed
132 Fix a problem with skiparticle sortkey that could be completely
133 ignored (and reduced to "").
135 Fix dependency problem in pazpar2 RPM package (did not require
136 libyaz4 as it should).
140 Fix memory leak that occurred for command=termlist&name=xtargets .
142 Pazpar2 may save HTTP requests. Enabled by option -R.
146 Experimental support for DTIC DADS target. New dads-pz2.xsl.
148 Support for query_syntax (overrides the default for SRU | Z39.50)
150 Support for extraArgs (ZOOM "extraArgs" option) for targets
152 New commands: status-server and status-session
156 Fix for threaded runs: Client now have a copy of the database URL,
157 which can used after the database has been release from the client.
158 This makes the logging in the connection idle timeout of the client nicer (no NOURL) and should be thread-safe.
160 tmarc.xsl: Add journal-title-abbrev and full text.
162 cf.xsl: new fields: isbn, issn, journaltitle, volume, issue
164 Fix for cmd=record before search.
166 Session Logging clean up.
168 Fix wrong termlist factor when maxrecs is different from 100.
172 Fix missing pz:termlist_term_factor in settings.c messed up pz:preferred.
173 Term factor is default enabled but can be diseabled by
174 pz:termlist_term_factor=0
178 Add scaling of facet count. Currently always enabled, needs fixing.
179 Allow user-defined info for target suffix. This has no meaning in
180 Pazpar2 except to distinguish targets from each other. The suffix
181 data begins with #. For example z3950.indexdata.com/gils#Mydata
183 Added exact-match recordfilter; format name=value
187 SOLR support. Pazpar2 may operate as web service client for SOLR.
191 Fix for show command and block=1 (dead lock). Bug was introduced in
196 New RPM packages: pazpar2, pazpar2-js, pazpar2-doc. These have been
197 tested on CentOS 5.5 only.
201 Fix problem with result sets being removed from a client session
202 if the connection for it was resused by another session. Bug #3489.
204 New iphone UI for Pazpar2 (www/iphone).
208 Fixes for threaded operation.
210 New stylesheets for TurboMARC: tmarc.xsl and opac_turbomarc.xsl.
212 New example services in etc/services in source. In the Debian packages
213 these are located in /etc/pazpar2/services-available
215 Threaded mode operational on Windows. Requires Windows 7 or Windows
218 Default value of setting pz:max_connections is 0 which means that there
219 is no limit on number of connections.
223 Pazpar2 may operate in threaded mode. Enabled by element threads in
224 the configuration. See pazpar2_conf for details.
226 New setting setting: pz:max_connections. Setting pz:max_connections is
227 a limit of number of sockets to a host. When this limit is reached,
228 Pazpar2 will wait up to 5 seconds for a connection to becomes available.
229 The client will be marked Client_Error when it can not be searched
230 (other clients in a session may work). If pz:max_connections is not set
231 for a target, a value of 30 will be used. Note: the pz:max_connections
232 will only work in threaded mode.
234 pz2.js: JSON support for show.
236 Debian package: Enable default service, default.xml, before starting
237 Pazpar2 only if there is no service already in /etc/pazpar2/services-enabled.
241 Debian version depends on on libyaz4. Note that Pazpar2 will still
242 compile from source with YAZ 3.
244 Split services into separate files. The example configuration file
245 pazpar2.cfg.dist now includes a default service default.xml (part of
246 etc). And default.xml includes settings/edu.xml. The default.xml file,
247 not to be confused with settings/defaults.xml, is a template for jsdemo
248 and other services. The Debian package installs /etc/pazpar2/server.xml
249 which is now the main pazpar2 configuration (used to be called pazpar2.cfg).
250 server.xml includes services from /etc/pazpar2/services-enabled/*.xml .
251 The default.xml (from etc) is installed in /etc/pazpar2/services-available
252 and a symlink to it is created from services-enabled. The default.xml
253 service is unnamed and, thus, will be used by jsdemo and test1.
255 New setting pz:negotiation_charset. Patch from Andrei V. Toutoukine. The
256 new setting pz:negotiation_charset specifies character set for Z39.50 Init.
260 Support for additional fields in cf.xsl and pazpar2.conf.dist:
261 publisher, available, due, location (=locallocation), callno
262 (=callnumber), thumburl and score.
264 Describe pz:xslt and the auto setting.
266 Move mergekey definition away from the normalization stylesheets and
267 define a mergekey common for all target types in pazpar2.cfg.
269 Code update: Use the Odr_int type for hit counts. This is part of
270 YAZ 3.0.47 and later and so configure checks for that.
274 Metadata attribute 'skiparticle' also works for ICU based
275 normalization. (was only working for the non-ICU/ASCII before).
277 Command bytarget with argument settings=1 will show settings per
278 target.. This is to be able to verify correct settings and be able to
279 test that they are correct. The database settings array size is now
280 also stored.. Problems with database settings array is that if not
281 careful it will be too small (smaller than dictionary per-service
284 Make record list sorting stable by comparing mergekey for records if
285 relevance/title or other sorting criteria all match. This is merely to
286 ensure that our regressions tests works (reproducible output).
288 Relevance calculation changes: use a different denominator (length) for
289 per-field relevance scoring.. Instead of length of all ranked fields we
290 now use length of individual fields (as if they were individual "free"
291 text fields). This will ensure that documents with a long field with no
292 match (say description) will not "hurt" a title match.
294 Diagnostic member was not set on connection error. Fixed
298 Command search takes two optional parameters, startecs and maxrecs,
299 that specifies the start offset (0, 1, ...) and maximum number of records
300 to fetch for each target.
302 XSLTs + MARC maps are cached within a session so we don't re-parse
303 them over and over again. Even for a session with a single search
304 there's much to be gained because many targets use the same
307 The metadata attribute 'mergekey' now takes one of three values 'no',
308 'required', 'optional' . And the resulting mergekey from metadata
309 is now ordered in the same way as metadata in the service definition.
310 Older Pazpar2 version use the order in which metadata appeared in a
313 The search argument 'filter' now offers a new operator ~ which does a
314 substring match. The = operator works as before: string match for
315 anything but pz:id, or target match for pz:id.
317 New setting pz:recordfilter. The value of this setting takes the
318 form name[~value]. This setting makes Pazpar2 ignore all retrieved
319 records that do not have the metadata element name with value substring
322 Pazpar2 allows YAZ log level to be set (option -v).
326 For WS responses Pazpar2 creates XML header. Exception: raw record.
328 Setting XML files are now stored in etc/settings instead of etc. This
329 reflects the layout with the Debian package layout.
331 Settings may be posted for command=settings. The POSTed settings must
332 have root element 'settings' like regular setting files. In order to be
333 recognized, the POST request must use Content-Type=text/xml.
335 A service may be posted for command=init. This service will be used
336 during the session. The service may have its own target settings,
337 ICU config, timeout, etc. In order to be recognized, the POST request
338 must use Content-Type=text/xml.
340 Timeout values may be given per-service. That's element 'timeout'
341 which takes three attribute values (a subset may be given): 'session',
342 'z3950_operation', 'z3950_session'. Option -T is no longer supported
343 - used to specify session timeout.
345 Option -t tests the Pazpar2 configuration and returns exit code
346 (0=success, non-zero=failure). In previous version of Pazpar2, -t
347 specified local settings.
349 In version 1.2.0 the configuration file - after include processing -
350 was dumped to stdout. Now, the configuration is only dumped to the
351 yaz log file if option -d is given.
355 Configuration may now have multiple server areas. This means that a
356 Pazpar2 instance may listen on multiple ports. Virtual hosting is not
357 yet supported - on a server basis. Configuration may also have multiple
358 services .. That is repeating service elements inside a server. Each
359 has an attribute 'id' which serves as service ID. This ID in turn may
360 be used in a Pazpar2 session, by specifying parameter service=ID for
361 command init. There can be at most one unnamed service inside a server
362 which can be referred to by not specifying an service ID for command
363 init (backwards compatible). In order to partition multiple servers and
364 services a new include directive has been added. This takes an attribute
365 'src' which specifies one or more sub-files. For example to include
366 service files, one might use:
367 <server >.. <include src=/"etc/pazpar2/conf.d/*.xml"/> .. </server>.
368 It is the intention that that completely makes the settings directive
371 Fix problem where the record command would wait forever if there were
372 no targets to wait for (activeclients == 0).
376 One result set is created per session (last search) rather than for
377 each connection which happen to be shared (bug #3009).
379 marc21 stylesheets changed for efficiency.
383 Session timeout may be specified on the command-line as option -T.
385 Pazpar2 may now be operated in a no-merged mode for records.. All records
386 will be considered unique. This mode is enabled if no mergekey is
387 generated by the normalization stylesheet (pz:xslt).
389 Pazpar2 caches original records from each target and the 'record' command
390 with offset returns the original record if 'syntax' and 'esn' are NOT
391 specified. This speeds up retrieval of original records but also means
392 that Pazpar2 uses more memory. The cached records will be freed when the
393 session terminates or a new search is executed.
395 Pazpar2 no longer uses its own ICU wrapper. Instead the ICU wrapper
396 library part of YAZ is used.
398 Added SRU client support.
400 Automatically computes pz:nativesyntax if not provided. Works for XML and
403 --- 1.0.13 2008/11/24
405 Command bytarget returns name of target (if defined).
407 --- 1.0.12 2008/11/04
409 Fixed bug #2021.. location now holds all brief elements.
411 --- 1.0.11 2008/10/15
413 Fixed check for application/x-www-form-urlencoded parameters.
415 --- 1.0.10 2008/10/14
417 Fixes for IE in pz2.js.
419 Fixed bug #2021: non-merged, brief meta data NOT included for command=show.
423 Changed the JS library pz2.js to use POST for long URL (+ params).
425 Added installation instructions for Windows. Note: NT services is
426 NOT available until we make a new release of YAZ.
428 Preserve order of repeated metadata fields (they were reversed before).
430 More MARC21 information extracted for metadata.
434 Fixed bug #1162: HTML entities are not escaped properly.
436 Native Windows port of Pazpar2. Makefile for Visual Studio provided.
440 Marc21 stylesheet updated to reflect multiple full text fields
444 Fixed bug in pz2.js WRT DOMElement attributes on IE.
446 Fixed bug 2100: Database wildcards not working
450 Added support for retrieval of records in binary.
452 Fixed bug 1794: Pazpar2 does not return valid XML.
454 Deal with ICU not returning sortkey (resulted in SEGV before).
458 JavaScript library pzw2.js throws error if WS response (from Pazpar2 or
459 other) is malformed (non-wellformed XML or missing Pazpar2 OK status).
461 Improved diagnostics when Pazpar2 HTTP decoding fails.
463 Pazpar2 requests may be POSTed as using Content-Type
464 application/x-www-form-urlencoded.
466 Pazpar2 honors LF in HTTP headers.
468 Handle targets that handle negative hit counts (should not happen, but it
473 ICU is used for tokenization and normalization of the following: mergekey,
474 sorting, relevance terms.
476 Debian package now enables ICU tokenization and normalization by default.
480 Exposed user setting values (i.e. non-pz: names) to the record systems in two
481 ways: Either as parameters to the normalization stylesheets (which would allow the
482 programmer to postprocess or use the values in any way) or after the normalization
483 step, in which case values are made part of the normalized record (and available for
484 sorting, termlists, display, or other interface-related use.
486 Implemented sorting by year.
488 Option -d dumps records to the current log file instead of stderr.
490 Fixes for compilation on cygwin.
492 Z39.50 client code uses pz:elements. pz:elements was recognized in
493 earlier Pazpar2 versions but it was not used for anything.
495 icu_chain_test is using fgets instead of getline - fixes compilation
498 Loosen the CCL query parsing so that Pazpar2 only returns error if _all_
499 query conversions fail (rather than _any_). This means targets that do
500 not support some fields are ignored in a search.
504 Improved handling of socket timeout for Z39.50 connections.
506 Misc documentation updates and spell fixes.
508 Debian package pazpar2 creates log rotate entry.
510 Debian package pazpar2-apache2 reloads Apache2.
512 jsdemo included in distribution. It illustrates the use of the js/pz2.js
517 First public release.