Debugging solr rss DataImportHandler -

-

i have existing collection, want add rss importer. i've copied gleam example-dih/solr/rss code.

the details below, bottom line seems run, says "fetched: 0" (and no documents). there no exceptions in tomcat log.

questions:

  • is there way turn debugging on rss importers?
  • can see solr's actual request , response?
  • what cause request succeed, no rows fetched?
  • is there tutorial adding rss dih existing collection?

thanks!

my solrconfig.xml file contains requesthandler:

<requesthandler name="/dataimport"     class="org.apache.solr.handler.dataimport.dataimporthandler">     <lst name="defaults">             <str name="config">rss-data-config.xml</str>     </lst> </requesthandler>

and rss-data-config.xml:

<dataconfig>     <datasource type="urldatasource" />     <document>         <entity name="slashdot"                 pk="link"                 url="http://rss.slashdot.org/slashdot/slashdot"                 processor="xpathentityprocessor"                 foreach="/rss/channel | /rss/item"                 transformer="dateformattransformer">              <field column="source_name" xpath="/rss/channel/title" commonfield="true" />              <field column="title" xpath="/rss/item/title" />             <field column="link" xpath="/rss/item/link" />             <field column="body" xpath="/rss/item/description" />             <field column="date" xpath="/rss/item/date" datetimeformat="yyyy-mm-dd't'hh:mm:ss" />         </entity>     </document> </dataconfig>

and schema.xml:

<fields>    <field name="title" type="text_general" required="true" indexed="true" stored="true"/>    <field name="link" type="string" required="true" indexed="true" stored="true"/>    <field name="source_name" type="text_general" required="true" indexed="true" stored="true"/>    <field name="body" type="text_general" required="false" indexed="false" stored="true"/>    <field name="date" type="date" required="true" indexed="true" stored="true" />    <field name="text" type="text_general" indexed="true" stored="false" multivalued="true"/>    <field name="_version_" type="long" indexed="true" stored="true"/> <fields>

when run dataimport admin web page, seems go well. shows "requests: 1" , there no exceptions in tomcat log:

mar 12, 2013 9:02:58 pm org.apache.solr.handler.dataimport.dataimporter maybereloadconfiguration info: loading dih configuration: rss-data-config.xml mar 12, 2013 9:02:58 pm org.apache.solr.handler.dataimport.dataimporter loaddataconfig info: data configuration loaded mar 12, 2013 9:02:58 pm org.apache.solr.handler.dataimport.dataimporter dofullimport info: starting full import mar 12, 2013 9:02:58 pm org.apache.solr.handler.dataimport.simplepropertieswriter readindexerproperties info: read dataimport.properties mar 12, 2013 9:02:59 pm org.apache.solr.handler.dataimport.docbuilder execute info: time taken = 0:0:0.693 mar 12, 2013 9:02:59 pm org.apache.solr.update.processor.logupdateprocessor finish info: [articles] webapp=/solr path=/dataimport params={optimize=false&clean=false&indent=true&commit=false&verbose=true&entity=slashdot&command=full-import&debug=true&wt=json} {} 0 706

your problem here due rss-data-config.xml , defined xpaths.

if open url http://rss.slashdot.org/slashdot/slashdot in internet explorer , hit f12 developer tools show structure of html.

you can see node <item> child of <channel> , not <rss>. config should follows:

<?xml version="1.0" encoding="utf-8" ?> <dataconfig> <datasource type="urldatasource" /> <document> <entity name="slashdot"             pk="link"             url="http://rss.slashdot.org/slashdot/slashdot"             processor="xpathentityprocessor"             foreach="/rss/channel | /rss/channel/item"             transformer="dateformattransformer">          <field column="source_name" xpath="/rss/channel/title" commonfield="true" />          <field column="title" xpath="/rss/channel/item/title" />         <field column="link" xpath="/rss/channel/item/link" />         <field column="body" xpath="/rss/channel/item/description" />         <field column="date" xpath="/rss/channel/item/date" datetimeformat="yyyy-mm-dd't'hh:mm:ss" />     </entity> </document> </dataconfig>

Comments

Post a Comment

Popular posts from this blog

c++ - llvm function pass ReplaceInstWithInst malloc -

-
#include "llvm/pass.h" #include "llvm/ir/module.h" #include "llvm/ir/function.h" #include "llvm/support/raw_ostream.h" #include "llvm/ir/legacypassmanager.h" #include "llvm/ir/instrtypes.h" #include "llvm/transforms/ipo/passmanagerbuilder.h" #include "llvm/ir/irbuilder.h" #include "llvm/transforms/utils/basicblockutils.h" using namespace llvm; namespace { struct replacepass : public functionpass { static char id; replacepass() : functionpass(id) {} virtual bool runonfunction(function &f) { allocainst* insttoreplace = ??? basicblock::iterator ii(insttoreplace); replaceinstwithinst(insttoreplace->getparent()->getinstlist(), ii, new allocainst(type::int32ty, 0, insttoreplace)); return true; } }; } char replacepass::id = 0; static void registerreplacepass(const passmanagerbuilder &, legacy::passmanagerbase &pm) { pm.add(new re...
Read more

Cross-Compiling Linux Kernel for Raspberry Pi - ${CCPREFIX}gcc -v does not work -

-
i'm trying follow this guide . i'm running both ubuntu 12.04.5 lts (gnu/linux 3.13.0-74-generic x86_64) on "real" hardware , 14.04.1 via virtualbox on mac. problem don't past step 1: hoffmann@angl99:~$ export ccprefix=/home/hoffmann/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin/arm-linux-gnueabihf- hoffmann@angl99:~$ ${ccprefix}gcc -v i'm getting following error: -bash: /home/hoffmann/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin/arm-linux-gnueabihf-gcc: no such file or directory however, file i'm told missing there: hoffmann@angl99:~$ less /home/hoffmann/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin/arm-linux-gnueabihf-gcc "/home/hoffmann/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin/arm-linux-gnueabihf-gcc" may binary file. see anyway? this result of basic error/misconception. suggest solution? thanks! sebastian ok - i've worked out (with of person po...
Read more

java.lang.NoClassDefFoundError When Creating New Android Project -

-
i've tried installing (and re-installing) android studio several times now, , each time, studio isn't able create new android project. when attempt initial android studio window (ie. basic 'create', 'open', 'import' etc options), i'm asked of questions in wizard, brief progress dialog, followed nothing @ all. if open incomplete project process creates choosing 'open' list, can try choosing create new android project menu. this has same issue, error message in output console time... java.lang.noclassdeffounderror: com/sun/xml/internal/bind/v2/model/annotation/annotationreader (wrong name: com/sun/xml/internal/bind/v0/model/qnnotation/annotationreader) note: when installed android studio, installed incompatible jdk (1.8, not 1.7). have had go default project settings change jdk uses 1.8. does know how can avoid this? there place should downloading jxb classes? missing in java paths? this appears have been caused cor...
Read more