[Yazlist] Performance tweaks for async. search?

Johannes Leveling johannes.leveling at FernUni-Hagen.de
Thu Nov 29 15:28:06 CET 2007


Am Donnerstag, 29. November 2007 14:05 schrieb Mike Taylor:
> Johannes Leveling writes:
>  > Hello,
>  > I am looking for even better performance of a Z39.50 database search
>  > (coupling Zebra and YAZ/ZOOM).
>  > My application searches multiple target systems via ZOOM,
>  > retrieves the top-N ranked documents asynchronously,
>  > converts the results, and writes them to a file.
>  >
>  > Is there a way to process result records as soon as they come
>  > in (instead of waiting until the slowest target system has returned the
>  > requested number of results)?
>  > I would like to convert partial results as soon as they are
>  > available (preferably with some event-driven approach,
>  > e.g. triggered by ZOOM_EVENT_RECV_DATA).
>
> Hi, Johannes.  This is pretty much exactly what IRSpy does.  You can
> download the software from
> 	http://search.cpan.org/~mirk/ZOOM-IRSpy/
Hi Mike!
Thanks for the quick response. I'm not quite sure IRSpy does
what I want (at least I didn't see it in the sources at first glance).

When the Zebra server logs this:
17:29:35-28/11 zebrasrv(16) [request] Search Default OK 12553 1 1+21 RPN ... 
17:29:42-28/11 zebrasrv(16) [request] Present Partial 2 -  1 22+79
17:29:44-28/11 zebrasrv(16) [request] Present Partial 2 -  1 37+64
17:29:47-28/11 zebrasrv(16) [request] Present Partial 2 -  1 50+51
...
17:29:56-28/11 zebrasrv(16) [request] Present OK -   1 91+10

My application receives the following ZOOM events (from connections to 2 
databases):
conn:2 last_ev:1 ZOOM_EVENT_CONNECT
conn:2 last_ev:6 ZOOM_EVENT_SEND_APDU
conn:2 last_ev:2 ZOOM_EVENT_SEND_DATA
conn:3 last_ev:1 ZOOM_EVENT_CONNECT
conn:3 last_ev:6 ZOOM_EVENT_SEND_APDU
conn:3 last_ev:2 ZOOM_EVENT_SEND_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:7 ZOOM_EVENT_RECV_APDU
// conn: 3: 0 hits
conn:3 last_ev:6 ZOOM_EVENT_SEND_APDU
conn:3 last_ev:2 ZOOM_EVENT_SEND_DATA
conn:2 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:2 last_ev:7 ZOOM_EVENT_RECV_APDU
// conn: 2: 0 hits
conn:2 last_ev:6 ZOOM_EVENT_SEND_APDU
conn:2 last_ev:2 ZOOM_EVENT_SEND_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:3 last_ev:7 ZOOM_EVENT_RECV_APDU
// conn: 3: 5281 hits
conn:2 last_ev:3 ZOOM_EVENT_RECV_DATA
conn:2 last_ev:7 ZOOM_EVENT_RECV_APDU
// conn: 2: 1564 hits

So, I want to start processing records at 
17:29:35-28/11 (from the Zebra log), but records become available after
17:29:56-28/11 (21 seconds later).

Should I take another look at IRSpy?

Johannes
> The code you'll want to consult is in check() subroutine of
> lib/ZOOM/IRSpy.pm
>
>  _/|_	 ___________________________________________________________________
> /o ) \/  Mike Taylor    <mike at indexdata.com>   
> http://www.miketaylor.org.uk )_v__/\  "If Microsoft Word had been around
> 400 years ago, Martin Luther would never have got around to nailing his
> thesis to the church
> 	 door.  He'd have spent the evening messing around with the font
> 	 size and found out the next day that the Reformation had broken
> 	 out without him" -- Andrew Rilstone.
>
>
> _______________________________________________
> Yazlist mailing list
> Yazlist at lists.indexdata.dk
> http://lists.indexdata.dk/cgi-bin/mailman/listinfo/yazlist

-- 
Dr. Johannes Leveling        
Praktische Informatik VII, Intelligente Informations- und 
Kommunikationssysteme
FernUniversität in Hagen

Email : Johannes.Leveling at FernUni-Hagen.De  
Tel.  : +49 2331 987-4525



More information about the Yazlist mailing list