[Zebralist] Updating a record "clears" the index?

Paul Ertz pertz at silo.lib.ia.us
Tue Sep 30 14:32:52 CEST 2008


Hi all,

We haven't been able to resolve the problem that we are having with our setup of Zebra. We 
have safe updating (shadow registers) enabled. When we run the below script to update a 
record, and then send a commit to the index, we are able to search for and see the updated 
record. However, every other record in the index appears to "disappear". As in, no other 
record will show up in any search. I suspect that we may have something configured 
incorrectly, but we are at a loss as to what to look for and where.

Any help or suggestions would be greatly appreciated.

Thanks in advance,
Paul



Paul Ertz wrote:
> Hello,
> 
> We are trying to write functionality to update records contained within our index but have 
> run into a snag in the process. When we run our script (see below) the MARC record is 
> loaded into the index and we are able to search for and find the record. Unfortunately, 
> that is the only record that we can search for. We are wondering if perhaps we do not have 
> something configured quite right or are missing a step in the update script. We did notice 
> a couple of warnings in the error log but are not sure whether these might be the root of 
> the problem or what they mean exactly.
> 
> 
> 08:06:21-26/09 zebrasrv-2.0(1) [session] Session - OK 1 tcp:::ffff:127.0.0.1 30258
> 08:06:21-26/09 zebrasrv-2.0(1) [request] Auth idPass admin -
> 08:06:21-26/09 zebrasrv-2.0(1) [request] Init OK - ID:81/81 Name:ZOOM-C/YAZ 
> Version:1.154/3.0.32
> 08:06:22-26/09 zebrasrv-2.0(1) [log] function: 1
> 08:06:22-26/09 zebrasrv-2.0(1) [log] Waitaction: 2
> 08:06:22-26/09 zebrasrv-2.0(1) [log] Received DB Update
> 08:06:22-26/09 zebrasrv-2.0(1) [log] action
> 08:06:22-26/09 zebrasrv-2.0(1) [log] specialUpdate
> 08:06:22-26/09 zebrasrv-2.0(1) [log] database: testdatabase
> 08:06:22-26/09 zebrasrv-2.0(1) [log] enabling shadow spec=/var/lib/zebra/shadow:-1b
> 08:06:22-26/09 zebrasrv-2.0(1) [log] cache_fname = /var/lib/zebra/shadow/cache
> 08:06:22-26/09 zebrasrv-2.0(1) [log] record 0 type XML
> 08:06:22-26/09 zebrasrv-2.0(1) [log] 1143 bytes:
> 01143nam a2200313Ia 
> 4500001003300000003000500033005001700038008004100055010001300096020001800109020001500127035001600142040001900158100002800177245008500205260004200290 
> ...
> 08:06:22-26/09 zebrasrv-2.0(1) [log] dom-config-stanford.xml dom filter: loading config 
> file /var/lib/zebra/profile/dom-config-stanford.xml
> 08:06:22-26/09 zebrasrv-2.0(1) [warn] Unknown register type:
> 08:06:22-26/09 zebrasrv-2.0(1) [log] add dom.dom-config-stanford.xml <no file> 0
> 08:06:22-26/09 zebrasrv-2.0(1) [log] Iterations: isam/dict 124/114
> 08:06:22-26/09 zebrasrv-2.0(1) [log] Dict: inserts/updates/deletions: 113/1/0
> 08:06:22-26/09 zebrasrv-2.0(1) [log] Records: 1 i/u/d 1/0/0
> 08:06:22-26/09 zebrasrv-2.0(1) [request] Extended Service: Update (done)
> 08:06:22-26/09 zebrasrv-2.0(1) [log] function: 1
> 08:06:22-26/09 zebrasrv-2.0(1) [log] Waitaction: 2
> 08:06:22-26/09 zebrasrv-2.0(1) [log] adm request database testdatabase
> 08:06:22-26/09 zebrasrv-2.0(1) [log] adm-commit
> 08:06:22-26/09 zebrasrv-2.0(1) [log] enabling shadow spec=/var/lib/zebra/shadow:-1b
> 08:06:22-26/09 zebrasrv-2.0(1) [log] cache_fname = /var/lib/zebra/shadow/cache
> 08:06:22-26/09 zebrasrv-2.0(1) [request] Extended Service: Admin (done)
> 08:06:22-26/09 zebrasrv-2.0(1) [session] Connection closed by client
> 
> 
> 
> Also, when we run find @attr 1=_ALLRECORDS @attr 2=103 "" while we are logged into the 
> yaz-client, we receive just one hit.
> 
> 
> 
> Below is the test script that I was using to try to update a record in the index:
> 
> 
> ###################################################################
> ## admarcrecord.pl
> 
> #!/usr/bin/perl
> 
> use strict;
> use Encode;
> use ZOOM;
> use MARC::Record;
> use MARC::Field;
> use MARC::File::XML (BinaryEncoding => 'utf8', RecordFormat => 'USMARC');
> use utf8;
> 
> my $host         = '_host_';
> my $port         = '_port_number_';
> my $db           = 'testdatabase';
> my $syntax       = 'USMARC';
> my $connection;
> 
>    eval {
>      $connection = new ZOOM::Connection($host,$port,
>        preferredRecordSyntax => "$syntax",
>        databaseName => "$db",
>        user => "_dbuser_",
>        password => "_dbpassword_");
>    };
> 
> 
>    if ($@) {
>      die "Error ", $@->code(), ": ", $@->message(), "\n";
>    }
> 
>    my $data = content_of('/ZOOM/testmarcrecord.marc');
> 
>    send_package($connection,
>                 "update",
>                 action => "specialUpdate",
>                 record => $data);
> 
>    send_package($connection, "commit");
> 
>    $connection->destroy();
> 
> sub send_package {
>      my($conn, $op, %options) = @_;
> 
>      my $p = $conn->package();
>      foreach my $key (keys %options) {
>          $p->option($key, $options{$key});
>      }
>      $p->send($op);
>      $p->destroy();
> }
> 
> sub content_of {
>      my($filename) = @_;
> 
>      use IO::File;
>      my $f = new IO::File("<$filename")
>          or die "can't open file '$filename': $!";
>      my $text = join("", <$f>);
>      $f->close();
> 
>      return $text;
> }
> 
> 
> 
> 
> 
> 
> And the zebra config file:
> 
> 
> ###################################################################
> ## zebra config file
> 
> encoding: utf-8
> 
> profilePath: /var/lib/zebra/profile:/usr/share/idzebra-2.0/tab
> modulePath:  /usr/lib/idzebra-2.0/modules
> 
> # Files that describe the attribute sets supported
> attset: bib1.att
> attset: explain.att
> 
> # Enables indexing using ICU
> index: icu.idx
> 
> # Enable Term Frequecy over Inverse Document Frequency Rank
> rank: rank-1
> 
> recordType: dom.dom-config-stanford.xml
> 
> isam: b
> 
> database:  testdatabase
> storeKeys: 1
> storeData: 1
> 
> perm.anonymous: r
> perm.admin:  rw
> passwd:  passwordfile
> 
> ##register: /var/lib/zebra/register:40G
> register: /var/lib/zebra/register:-1b
> ##shadow: /var/lib/zebra/shadow:40G
> shadow: /var/lib/zebra/shadow:-1b
> 
> # Lock File Area
> lockDir: /var/lib/zebra/lock
> 
> # Temp File area for result sets
> setTmpDir:  /var/lib/zebra/tmp
> 
> # Temp File area for index program
> keyTmpDir:  /var/lib/zebra/tmp
> 
> # Approximate memory usage during indexing
> memMax: 1024
> 
> 
> We haven't been able to figure out why the problem is happening. Any help or suggestions 
> would be greatly appreciated.
> 
> Thanks,
> Paul
> 
> 
> 
> 
> 
> _______________________________________________
> Zebralist mailing list
> Zebralist at lists.indexdata.dk
> http://lists.indexdata.dk/cgi-bin/mailman/listinfo/zebralist



More information about the Zebralist mailing list