3 zoom: new config element passthru. This defines an element set that is
4 passed verbatim to the backend in pz2 mode.
6 auth_simple: allow both idPass and open auth. Also fix SEGV that could
11 z3950_client: fix string handling for init diagnostics in the case where
14 query_rewrite: no charset conversion for error queries. Avoid performing
15 charset conversion for query conversion errors. The resulting Type-1
16 query is invalid in this case.
20 For graceful stop (SIGUSR1), metaproxy will kill children processes.
22 Log when metaproxy is invoked in test mode (-t).
24 Log PID for each log message.
26 multi: combine both NSDs and multiple NSDs into one (diagnostics).
28 z3950_client: fixup multiple NSD's too. Like surrogate diagnostics,
29 a (backend=target) is appended for each diagnostic so that can be
30 determined the origin of error.
32 multi: hideerrors also in use for present response.
34 Solr example in zoom man page.
36 Fix typo in query_rewrite config example.
38 Log at least first non surrogate diagnostic. Instead of
39 "Z_Records_multipleNSD"
41 multi: Fix check for opt Records in present response. A SEGV could occur
42 if a target did not return records and no diagnostics.
46 Yet another fix for graceful stop.
50 sru_z3950: relay surrogate diagnostics.
54 zoom: Honor extraArgs, strip # in zurl.
56 multi: fix SEGV for scan. Reported by Rustam T. Usmanov.
58 zoom: url_recipe allow %{var[reg]} substitutions.
59 It's like ${var[reg]} but does URI component encoding as well.
63 RPM/Deb: require yazpp 1.3.2, because it fixes a missing close of
64 non-keepalive HTTP sessions.
66 http_file: allow zero size files (bug fix)
68 Fix zoom schema: auth_url is optional
72 zoom: fix un-init memory (could result in SEGV at session close)
74 frontend_net: handle X-Forwarded-For. This is stored in package.origin()
75 and may be used in filters as usual.
77 zoom: authentication via Torus lookup
78 Enabled by new configuration setting: auth_url . If this is defined,
79 the zoom module will contact Torus for auth/IP lookup and that will
80 define realm to be used in session.
84 virt_db: Fix database name - first character we removed by mistake.
85 Bug introduced in 1.3.40.
89 virt_db: Don't use "unix path" as database for Unix sockets.
93 z3950_client: Don't tamper with / in zurl - fixes problems
94 using Unix sockets for backends.
96 session_shared: fix integer overflow problem with hit count.
98 session_shared: avoid long usage of cached queries.
100 zoom: New database argument , torus_url to set URL of Torus
101 record to be fetched.
103 --- 1.3.38 2012/07/04
105 Fixes for 'metaproxy reload' for RPM/Debian packages.
107 zoom: Allow rpn2cql conversion to be augmented with a properties file.
108 Element rpn2cql can be specified per-target for the zoom filter.
110 log: New category "line" which is a single-line log message with
111 most important parts of a HTTP/Z39.50 request/response.
113 --- 1.3.37 2012/06/01
115 frontend_net: add facility to report live statistics about response
116 times and current thread usage. The report is an XML document
117 and is triggered for a specific URL path. This is set by <stat-req>
118 in the configuration of frontend_net. By default this report is
119 disabled (same as empty value for <stat-req>).
121 Fix reload problem in init.d script for RPM/Debian.
122 The '/etc/init.d/metaproxy reload' in some cases would stop but not
123 start metaproxy - effectively stopping the service.
125 Don't display start log message when config testing (-t).
127 zoom: avoid CCL message limit of 1K in ZOOM filter.
129 --- 1.3.36 2012/05/10
131 metaproxy init script reacts to reload which does graceful stop +
132 start and is used by logrorate script.
134 --- 1.3.35 2012/05/09
136 Fixes for Windows compilation.
138 Metaproxy daemon reacts to SIGUSR1 in which case it will leave
139 existing sessions running but terminate the monitor process that is
140 used in keepalive mode.
142 Change start log message to 'metaproxy start'. Was 'Metaproxy start'
143 before. Consistent with 'metaproxy stop' message.
145 zoom: document zoom's proxy_timeout
147 z3950_client: fixup addinfo for init response
149 virt_db: return other diagnostic if init is rejected. Diagnostic 236:
150 "Access to specified database denied" is returned, but only if init
151 response from backend does not carry diagnostics already.
153 multi: relays user info for first rejected target. The other-information
154 field for init response for first rejected target is in common init
155 response. This ensures that some backend init diagnostics are returned to
158 --- 1.3.34 2012/04/25
160 frontend_log: extend logging (when message is given in config).
162 --- 1.3.33 2012/04/25
164 zoom: Fix check for proxy. Use diagnostic 1074 for proxy failures.
166 --- 1.3.32 2012/04/24
168 sru_z3950 filter passes through incoming Z39.50 requests again. Was
169 not handled by mistake in Metaproxy versions 1.3.29, 1.3.30 and 1.3.31.
171 --- 1.3.31 2012/04/24
173 zoom: fixes for retry/proxy failover logic. More logging.
175 --- 1.3.30 2012/04/23
177 zoom: return addinfo="proxy failure" for proxy failures. This also
178 changes the scheme for dealing with proxies a bit. zoom now always
179 do proxy check, even if there's no failover. Fortunately, that's not
182 zoom: Init rejected wo diagnostics maps to Bib-1:1014.
183 Bib-1 1014, "Init/AC: Authentication System error", seems pretty
184 generic. We don't know why the init would fail; IP, user, block IP, etc.
185 sru_z3950 filter maps this to SRU diagnostic 3: "Authentication error".
187 zoom: guess if invalid SRU response was an auth failure
189 session_shared: disable session retry by default . Can be enabled by
190 restart="true" in resultset section.
192 --- 1.3.29 2012/04/18
194 zoom: may read cf-proxy's config settings.
195 This reduces manual configuration of the content proxy server host.
196 The ZOOM module will use the old style session.proxyhost if "server"
197 is given (deprecated) and use the new style proxyhost/session if
198 "config_file" is in use.
200 frontend_net: remove incoming requests that can not be handled
201 Requests in queue for filter frontend_net are removed if client
202 closes connection for the session and request is not yet handled.
204 sru_z3950: serialize requests. This is to ensure that pipelined
205 HTTP requests are handled properly.
207 --- 1.3.28 2012/04/04
209 New class, wrbuf, that wraps YAZ' WRBUF.
211 --- 1.3.27 2012/03/30
213 Metaproxy daemon is now running as user "metaproxy". This user is
214 managed by package maintainer scripts. The user information is stored
215 in /etc/metaproxy/metaproxy.user .
217 --- 1.3.26 2012/03/20
219 New filter, sort, that allows sorting of XML records via Z39.50
222 Filter record_transform: USEMarcon support. For example:
223 <retrieval syntax="usmarc">
224 <backend syntax="unimarc" name="F">
225 <usemarcon stage1="/etc/usemarcon/uni2us/uni2us.ini"/>
229 Filter multi: empty route pattern uses route as default.
231 <target route="z3950.indexdata.com">z3950.indexdata.com</route>
232 may use the shorter notation
233 <target route="z3950.indexdata.com"/>
235 Filter query_rewrite: use config path to locate XSLT.
237 Filter frontend_net: log statistics.
238 If a custom message element in frontend_net configuration it makes
239 the frontend_net filter produce a log entry whenever an operation
240 is completed. The log is always written using yaz_log with the
241 custom message given, the package ID (session ID) , the time
242 the operation has been active (end-to-end) , the threads currently
243 active/total and the size of the request input/output queues.
245 query_rewrite: charset conversion of query terms.
246 This is an optional feature. Enabled by element <charset> in
247 configuration with attribute 'from' and 'to' that specifies
248 from/to encodings. The default 'from' encoding is UTF-8.
250 Filter record_transform: honor piggyback searches.
252 Filter zoom: re-connect when database is reused.
253 Re-connect is necessary if a connection is lost or a timeout occurs.
254 If not, we'll be seeing zero hits, when we shouldn't.
256 --- 1.3.25 2012/02/29
258 zoom: Change scheme for dealing with content proxy URLs. The module
259 now has its own common XSL transform that is enabled by record_xsl
260 attribute in the configuration. This XSL gets parameter cproxyhost
261 that holds the host for the content proxy session . The genearted-url
262 is created only if urlRecipe is enabled. In previous version default
263 value of urlRecipe was ${md-electronic-url}; it is now empty and
264 disabled by default. Also generated-url no longer includes the content
265 proxy host. Content proxying URLs is supposed to the handle in the record_xsl
268 zoom: new setting timeout that specifies timeout for outgoing Z39.50/SRU
269 connections. Default value is 40 (seconds).
271 zoom: tune CCL to Z39.50 diagnostics mapping. Throw Bib-1: 108
272 "Malformed query" for syntax errors and related ones; throw
273 Bib-1: 3 "Unsupported search" for everything else. Details from
274 the CCL parser is provided as additional-info.
276 multi: new setting hideerrors that makes multi only relay diagnostics
277 to a client if all backends fail.
279 --- 1.3.24 2012/02/23
281 zoom: empty ccl_map field-mappings disables CCL map.
283 zoom: fix Explain search that could return error if session was
286 --- 1.3.23 2012/02/10
288 zoom: returns Explain records for database IR-Explain---1. The
289 target profile records are converted to Explain records via XSLT.
291 session_shared: make error recovery configurable. Whether a search
292 is restarted on errors or not, is configurable by attribute "restart"
293 in element "resultset" in configuration.
295 session_shared: only relay init for first request. session_shared, for
296 each backend class, now only relays first init request (rather than all).
297 The module only needs to save init response info once. This avoids the
298 need to check for session max.
300 --- 1.3.22 2012/02/02
302 zoom: fix memory leaks.
304 session_shared: maximum number of sessons is configurable. Default
305 value is 100. Result-sets for backends without named result sets are
306 only re-used if databases is identical for a search. This is under the
307 assumption that performance may suffer for those backends when
308 changing the database for each search.
310 multi: combines diagnostics from all backends. All diagnostics from
311 backends are combined into multiple non-surrogate diagnostics; rather
312 than just one diagnostic from first backend.
314 --- 1.3.21 2011/12/27
316 zoom: relay realm to cproxy parameter file.
318 Configuration: Allow filter collection element inside route. This
319 makes it possible to include multiple filters at once.
321 zoom: nocproxy also disables urlRecipe.
323 --- 1.3.20 2011/12/21
325 zoom: return diagnostic if CQL sortby conv fails. Diagnostic Bib-1 214
326 "Illegal sort relation" is returned. Gets translated to SRU diagnostic
327 90: "Unsupported direction value".
329 zoom: honor database argument nocproxy. This disables creating a content
330 connector session (for non-CF targets, usually) and also relays nocproxy
331 as database argument to a CF target.
333 --- 1.3.19 2011/12/16
335 zoom: Multiple proxy hosts may be given for proxy rich database
336 parameter. List is comma separated.
338 http_file: decodes URI paths properly. Does not return 404: not found
339 if no prefix are matches. Instead packages are passed on.
341 --- 1.3.18 2011/12/08
343 zoom: content-{user,password,proxy} are passed on to content proxy
344 system and takes precedence over user,password,proxy .
346 sru_z3950: SRU extra argument x-log-enable=1 enables a search response
347 with a log element including log material from children filters. At
348 this point, only the zoom filter uses this log facility. It will
349 eventually be used by other filters as well. In particular it might be
350 useful for this feature to track all filters in used during a search.
352 zoom: for x-log-enable=1, APDUs sent by ZOOM C, records as received
353 and queries as they are manipulated are logged.
355 zoom: extend addinfo with native diagnostics. Diagnostics as received
356 by ZOOM C are put into addinfo - including ZOOM, HTTP, SRU and Bib-1
357 diagnostics. This indicates both a backend diagnostic and the original
360 zoom: default urlRecipe value is ${md-electronic-url}
362 zoom: target profile setting sruVersion is recognized.
364 --- 1.3.17 2011/11/17
366 zoom: log target profile record fetched (AKA Torus).
368 --- 1.3.16 2011/11/17
370 Change target profile parameter CfSubDb to CfSubDB.
372 --- 1.3.15 2011/11/12
374 zoom: bug fix: realm was relayed to CF targets by mistake (realm only
375 makes sense to the zoom module itself).
377 --- 1.3.14 2011/11/11
379 zoom: realm may be given as part of database.
381 --- 1.3.13 2011/10/26
383 Fix Origin class assigment method; was removd by mistake in 1.3.12.
385 --- 1.3.12 2011/10/26
387 zoom: database args x-name=value are passed through to backend database.
389 SRU filter: user may specify session ID that is passed to log entries
390 that follows. It is specified by using x-session-id in the SRU arguments.
392 zoom: fix null pointer reference which would occur for bad url parameter
393 inside "torus" or if url was not given.
395 --- 1.3.11 2011/09/09
396 Fix bug in filter session_shared where sessions would not expire
397 properly (session ttl). This in turn could make the target close the
398 connection before expected and session_shared would return diagnostic
399 2: system temporarily available.
401 --- 1.3.10 2011/09/09
403 zoom: new target profile setting: sortmap_field. If sortmap_field is given
404 the field is mapped to the value for the sortmap. Values not matching
405 sortmap_field are passed verbatim.
407 zoom: new target profile setting: sortStrategy. One of: "z3950", "type7",
408 "cql", "sru11" or "embed". The "embed" chooses type-7 or CQL sortby
409 depending on whether RPN or CQL is actually sent to the target.
413 zoom: urlRecipe result is stored in <pz:metadata type="generated-url">.
415 zoom: For content proxy authentication is stored in param name
416 "auth", instead of "authentication".
420 zoom: relay schema in SRU mode. Previously recordSchema was not affected
425 zoom: new target profile element literalTransform that holds XSLT
426 (encoded as string) for target. Only if present and non-zero length it is
427 considered present. This takes precedence over transform.
429 Don't rely on trang anymore for Git checkout builds. Configuration in etc
430 will only be properly validated if trang is available. From now on only
431 relax NC files are bundled with the source dist.
433 Updates to check for Boost unit test (autoconf stuff). On MacOSX this
434 component was not detected when it should be.
438 z3950_client: mark non-surrogate diagnostics from backend.
440 zoom: a proxy may be specified for HTTP fetch of target profiles.
444 Honor piggyback. Makes some SRU targets behave that insist on
445 maximumRecords > 0, but also increases performance.
449 Fix dlpath in installed metaproxy.xml for RPM/Deb package. The
450 file was not update for libdir/metaproxy4 in version 1.3.2.
454 Change location of Metaproxy modules from libdir/metaproxy to
455 libdir/metaproxy4. Directory libdir/metaproxy is also owned by
456 libmetaproxy3 and, thus, should not be used by libmetaproxy4.
458 New facility to dump APDUs from zoom filter. Enabled by
459 <log apdu="true"/> in the zoom filter configuation.
461 zoom filter bug fix: SRU diagnostics received from backend was
462 not properly converted to Bib-1 diagnostics.
464 Package metaproxy depends on libmetaproxy4, same version.
468 zoom: fix a bug WRT passing authentication parameters to backends.
472 Fixes for windows Makefile.
474 --- 1.2.10 2011/07/25
476 Change major version of metaproxy library from 3 to 4. This in turn
477 makes new package names for Debian and RPM.
481 zoom: enable marc-8 encoding by default for usmarc and opac (in pz2
484 zoom: configurable element sets for record transforms.
486 zoom: the target profile based transform is specified by element_transform
487 in element torus of of the ZOOM filter configuration ; default value
488 is "pz2". Target profile based retrieval is specified by element_raw
489 in configuration. It will perform same steps, but omit the XSL transform.
490 Default value is "raw".
492 zoom: element set is omitted if not set in profile. But only if pz2
493 transform is in effect. Previously the default element set was to "F".
495 zoom: For non-CF targets database arguments are parsed. User may supply
496 db,user=myuser&password=mypass for non-CF targets. These values override
497 what's given the authentication from the target profile.
499 session_shared: fix reference to freed memory. Would occur if a search
500 response had non-surrogate diagnostics.
504 Route may be given for each port in frontend_net.
508 Documentation for zoom filter. Local target profiles may be specified.
510 Fixes for zoom filter.
514 First version with zoom filter.
518 Filter z3950_client may be configured to always send a close APDU
519 to backend target when connection/session is closed. This will only
520 happen if close APDU is not already sent. Enabled by configuration
521 for z3950_client (force_close).
523 Filter virt_db and multi: relay close APDU.
525 Filter virt_db and multi: relay init size parameters
526 preferredMessageSize and maximumRecordSize.
528 Filter sru_z3950: increase init sz parameters from 1MB to 10 MB.
532 RPM package for Metaproxy works for both SLES11 and Centos 5.5.
534 metaproxy-config: exec_prefix, libdir, includedir. Use these variables
535 as determined by configure.
537 RPM: Release includes 'indexdata' name. Use indexdata in release name so
538 this package can be distinguished from other vendors.
542 Fix filter multi: relay idAuthentication parameters.
546 Add metaproxy-config for returning compiler flags, libraries.
550 Fix --with-boost=PREFIX .
554 session_shared: fix use of maintenance thread.
556 Introduce 'start' method for filters and routers. It gets called after
557 configure and after fork, but before processing. This changes API -
558 from libmetaproxy2 to libmetaproxy3.
562 session_shared: ensure backend session is closed if init rejected
565 z3950_client: fix NULL ptr reference that could occur if two clients
566 connected concurrently and if max-sockets was in effect.
570 Fix bug 3497 - Metaproxy unstable while stopping.
572 Fix bug 3495 - SEGV in SRU filter.
576 Filter multi throws diagnostic 'Present Out of Range' when
577 appropriate - rather than returning 0 records.
579 Filter virt_db: fix bad result reuse. Could happen if a result
580 was overriden and a backend did not supported named result sets.
584 Filter log: category access logs diagnostics even if searchStatus
585 or presentStatus is 'success'.
589 The SRU to Z39.50 filter does repeated Z39.50 present requests
590 to fetch all records. Needed when a Z39.50 server returns partial
591 results (due to message size limits).
595 Metaproxy now kills all child processes that are spawned by
596 modules (in case of fork, exec).
600 Debian package now installs /etc/metaproxy/metaproxy.xml which
601 includes all XML files /etc/metaproxy/filters-enabled .
602 The include facility uses YAZ' yaz_xml_include_simple (YAZ 4.0.1).
606 Package based on YAZ version 4 and YAZ++ 1.2.0. It now includes a
607 development package to facilitate 3rd party Metaproxy modules.
609 --- 1.0.23 2010/01/04
611 Filter virt_db: Close backend filter session when Init rejected is received
612 from it (in which case it's not really closed yet). This will ensure we don't
613 run out of resources such as FDs in this case.
615 sru_z3950 filter: Accept any database (including those not listed).
617 --- 1.0.22 2009/12/03
619 Prepare for 64-bit YAZ. yazproxy will still work with YAZ++ 1.1.4 or
620 later which means YAZ 3.0.47 or later.
622 --- 1.0.20 2009/11/19
624 For filter virt_db, the database is a glob pattern and the virtual
625 database sections are consulted in the order given.
627 --- 1.0.19 2009/07/08
629 For filter multi, the target (for routing) may be given as a glob-pattern.
631 More documentation for filters virt_db and multi.
633 Update schema to allow for no database in sru_z3950 filter.
635 --- 1.0.18 2009/06/30
637 Added limit filter to put sleep (penalty) for heavy use.
639 Filter frontend_net may be configured to limit the number of new connections
640 from same IP to some value (connect-max).
642 Fixed the load_balance filter.
644 --- 1.0.17 2009/06/16
646 Fix path for metaproxy executable for Debian init.d script.
648 --- 1.0.16 2009/06/10
650 SRU to Z39.50 filter honors x-username and x-password. If only
651 x-username is given that translates to open authentication (Z39.50v2). If
652 both x-username and x-password is present that translates to idPass
653 authentication (Z39.50v3).
655 New facility to change current working directory (-w).
657 Update to upstream skeleton of init.d script. Should fix bug #2619.
659 Filter session_shared now caches records (used to only cache connections
662 Filter session_shared now invalidates a Z39.50 session if a server
663 returns -1 hits. This facility was also part of yazproxy and gets around
664 a bug in Voyager-based Z39.50 servers.
666 --- 1.0.15 2008/11/18
668 Fixed reference counting bug in Z39.50 client module (could cause SEGV).
670 --- 1.0.14 2008/06/09
672 Fixed the value of SRU record schema in searchRetrieve response. It was set
673 to a fixed value of "dc" before.
675 Serialize identical SRU requests. This is useful if HTTP client software
676 fires identical requests against metaproxy.
678 Allow Z39.50 target to be specified for SRU requests by using SRU extra
679 parameter x-target. Allows SRU to contact arbitrary Z39.50 targets and
680 avoid static configuration. This facility is handled by the sru_z3950 module
681 and the z3950_client module.
683 Allow maximum number of sockets to be used for a Z39.50 client by the
684 use of SRU extra parameter x-max-sockets. This facility is handled by the
685 sru_z3950 module and the z3950_client module.
687 --- 1.0.13.1 2008/02/29
689 Simplified check for Boost libraries.
691 --- 1.0.13 2008/02/29
693 First Debian package.
695 The sru_z9350 filter now recognizes SRU parameter z-target as being
696 a ZURL of the form host/database .
698 Mergetype may be specified for filter 'multi'.
700 --- 1.0.12 2007/09/19
702 Fixed problem in Makefile WRT library specifictions. This could produce
703 linker errors (when they should not happen).
705 Fixed bug #1341: MetaProxy returning different results. Problem was that
706 fix-up of record database did not normalize the database name.
708 Documentation updates for Windows compilation.
710 --- 1.0.11 2007/06/14
712 Fixed problem with logfile being truncated.
714 --- 1.0.10 2007/05/23
716 Fixed bug #1136: Access rights not enforced.
718 Extension for log module. The log category 'access' logs more details.
719 New category 'user-access' added which is simlar to 'access' but adds
720 the Z39.50 user as for all entries in a session.
724 Added support for configurable default/force target/vhost for module Z39.50
729 Implemented filter, session_shared, which optimizes usage of Z39.50
730 sessions and result sets.
732 New filter, record_transform, which converts MARC/XML records.
734 New filter, sru_z3950, which converts SRU to Z39.50 packages.
736 New filter, load_balance, in development.
738 New filter, zeerex_xplain.
740 XML framework for config files started.
742 Master RelaxNG Compact schema's are found in xml/schema/*.rnc,
743 RelaxNG XML schemas and XML Schema's are generated from RNC syntax.
744 XML config files are checked against RNG and XML Schema's.
746 XSLT transformation framework added in xml/xslt/*.xsl, consisting mainly of
747 LOC Marc21 stylesheets found at http://www.loc.gov/marcxml/xslt.
748 Added stylesheet explain2cqlpqftxt.xsl which transforms a standard SRU ZeeReX
749 explain record to YAZ CQL-to-PQF text config file.
753 Added 'timeout' setting for filter frontend_net which specifies the number
754 of seconds a session is idle before it is closed. Default timeout is
759 Database name in records returned set for virt_db/multi filters.
761 Filter session_shared now functional.
763 Log filter can be configured to write to a given file using 'filename'
764 element. Also information category can be specified with 'category'. If
765 filename is omitted, the log filter writes to stdout. See etc/config5.xml
770 Fixed compilation on Windows.
774 Added documentation for the hidenavailable option for the multi filter.
778 Fixed bug #573: Hide errors for individual servers. If the element
779 'hideunavailable' is given in the configuration, individual servers
780 being unavailable are hidden (unless all are unavailable).
782 Fixed bug #574: Database names are recognised case-sensitively.
784 Fixed bug #567: Fix up database name in Name-Plus-Record.
786 Fixed bug #568: Update win/makefile for VS 2005.
788 Fixed bug #569: Some options throws unknown exception.
792 More information about virtual hosts. The 'etc' directory is part of the
797 Build instructions for Windows. NMake makefile part of distribution tar.
803 Filters functional: auth_simple, backend_test, frontend_net, http_file,
804 log, multi, query_rewrite, template, virt_db, z3950_client.
806 Filters in development: session_shared.
808 Pseudo-filters: template, dl