python - Document classification in spark mllib -

-

i want classify documents if belong sports, entertainment, politics. have created bag of words output somthing :

(1, 'saurashtra') (1, 'saumyajit') (1, 'satyendra')

i want implement naive bayes algorithm classification using spark mllib. question how convert output can naive bayes use input classifcation rdd or if there trick can convert directly html files can used mllib naive bayes.

for text classification, need:

  • a word dictionary
  • convert document vector using dictionary

  • label document vectors:

    doc_vec1 -> label1

    doc_vec2 -> label2

    ...

this sample pretty straghtforward.


Comments

Post a Comment

Popular posts from this blog

c - How to retrieve a variable from the Apache configuration inside the module? -

-
i'm building custom apache module. how retrieve variable apache configuration inside module? see variable "some_var_config_from_apache_or_htaacess" inside apache module: /* source: http://people.apache.org/~humbedooh/mods/examples/mod_example_1.c */ /* include required headers httpd */ #include "httpd.h" #include "http_core.h" #include "http_protocol.h" #include "http_request.h" static void register_hooks(apr_pool_t *pool); static int example_handler(request_rec *r); /* define our module entity , assign function registering hooks */ module ap_module_declare_data example_module = { standard20_module_stuff, null, // per-directory configuration handler null, // merge handler per-directory configurations null, // per-server configuration handler null, // merge handler per-server configurations null, // directives may have httpd register_hoo
Read more

c# - Constructor arguments cannot be passed for interface mocks -

-
when debug code , read line mocklessonplannerafactory creation error: constructor arguments cannot passed interface mocks. var mockschoolclasscodeservice = new mock<ischoolclasscodeservice>(); var mockdateservice = new mock<idateservice>(); var mocklessonplannerafactory = new mock<ilessonplannerafactory>(mockdateservice.object); var mocklessonplannerbfactory = new mock<ilessonplannerbfactory>(mockdateservice.object); var service = new timetableservice(mockunitofwork.object, mocklessonplannerafactory.object, mocklessonplannerbfactory.object, mockschoolclasscodeservice.object); my timetableservice accepts instances of interface type only. mocklessonplannerafactory , bfactory... want in constructor idateservice passed. what wrong code? the clue in error message "constructor arguments cannot passed interface mocks." the mock created interface have default constructor because interfaces don't have constructor. remember mockin
Read more

python - malformed header from script index.py Bad header -

-
i want run python code in apache2(ubuntu 14.04) server. have followed these steps , getting error: step 1: configuration 1: have created directory , file under /var/www/cgi-bin configuration 2 : have edited /etc/apache2/sites-available/000-default.conf alias /cgi-bin /var/www/cgi-bin <directory "/var/www/cgi-bin"> options indexes followsymlinks execcgi addhandler cgi-script .cgi .py allow </directory> <directory /var/www/cgi-bin> options </directory> step 2: and python script is: index.py #!/usr/bin/env python import cgi; import cgitb;cgitb.enable() print "content-type: text/plain\n" print "<b>hello python</b>" step 3: when ran through chrome browser using: url : http://localhost/cgi-bin/index.py step 4: i getting error in error-log malformed header script 'index.py': bad header: hello python you should end header \r\n, must print out yet \r\n signal body coming. (i
Read more