What are "good" expected response times when processing large tables in MySQL remotely? -

-

as have never worked large data remotely have absolutely no idea "slow" , "fast" when processing large tables in remote server.

say have table: 

    'create table `kardex` (       `kardexid` int(10) unsigned not null auto_increment,       `codproduct` char(14) not null,       `date` date not null,       `reference` char(3) not null,       `effect` char(1) not null,       `priority` char(2) not null,       `quantity` double(14,6) default null,       `numdocument` char(8) default null,       `serie` char(10) default null,       `subsidiary` char(2) not null,       `subsidiary2` char(2) default null,       `quant_entry` double(14,6) default null,       `cost_entry` decimal(12,2) default null,       `quant_egress` double(14,6) default null,       `cost_egress` decimal(12,2) default null,       `quant_balance` double(14,6) default null,       `cost_balance` decimal(12,2) default null,       `cost_average` decimal(12,4) default null,       `costentry` decimal(12,4) default null,       `production` tinyint(1) default null,       `numdocument2` char(15) default null,       `consume` tinyint(1) default null,       `lot` char(6) default null,       `nameprovider` varchar(40) default null,       `nation` varchar(15) default null,       `transaction` char(3) default null,       `productserie` char(16) default null,       `user` char(15) default null,       `regdate` datetime default null,       `pcname` varchar(20) default null,       unique key `kardexid` (`kardexid`),       key `codproduct` (`codproduct`,`date`,`priority`),       key `sucprod` (`subsidiary`,`codproduct`,`date`,`priority`),       key `codrefdatsub` (`codproduct`,`reference`,`date`,`subsidiary`),       key `subdatref` (`subsidiary`,`date`,`reference`),       key `subrefdatdoc` (`subsidiary`,`reference`,`date`,`numdocument`),       key `productserie` (`productserie`),       key `sepdatpri` (`serieproduct`,`date`,`priority`),       key `num_document` (`numdocument`,`serie`,`reference`),       key `prodsub` (`codproduct`,`subsidiary`),       key `sepsub` (`productserie`,`subsidiary`)     ) engine=innodb auto_increment=88109 default charset=utf8'

registering every single movement of 8,000 products contains 5,000,000 rows. calculate average cost of every product @ given range of dates id need query iterate , last calculated average before starting date selected range. there on, have calculate average cost based off registries documents know affect average , update row, there can several hundreds of thousands of updates , few thousand selects (because of products).

this, locally, can done in few minutes. should expect "slow" or "fast" in remote server?

it depends. in short: if server comparable in performance computer, difference going in how long takes transmit result set. if result set large, slower. if result set small, doesn't matter.


whether (and what) "slow" or "fast" going largely depend on 3 factors: mysql server itself, connection between client , server, , query that's run. in general, if query "slow" locally, probably going "slow" when run on remote server.

the data stored on server, , processed there. how fast (or slowly) processing query depends on data, server, , query. (faster servers faster; more data slower.) work happens on server; local mysql client tool talk , view results of queries.

when query local (same computer or same network) server, time transmit data usually negligible, , small result set, doesn't matter. when query remote server, round-trip latency , bandwidth limitations apply. if result set small, difference negligible. if result set huge, difference can significant.

for example: if have simple query large result set (such select * 5 million row table), time process query might relatively small compared time takes transmit 5 million rows. (reading 5 million rows disk much faster sending data across internet.)

alternatively, if run highly analytical query returns one-column one-row integer result, server spend ages calculating result , no time sending once it's finished.


Comments

Post a Comment

Popular posts from this blog

c++ - llvm function pass ReplaceInstWithInst malloc -

-
#include "llvm/pass.h" #include "llvm/ir/module.h" #include "llvm/ir/function.h" #include "llvm/support/raw_ostream.h" #include "llvm/ir/legacypassmanager.h" #include "llvm/ir/instrtypes.h" #include "llvm/transforms/ipo/passmanagerbuilder.h" #include "llvm/ir/irbuilder.h" #include "llvm/transforms/utils/basicblockutils.h" using namespace llvm; namespace { struct replacepass : public functionpass { static char id; replacepass() : functionpass(id) {} virtual bool runonfunction(function &f) { allocainst* insttoreplace = ??? basicblock::iterator ii(insttoreplace); replaceinstwithinst(insttoreplace->getparent()->getinstlist(), ii, new allocainst(type::int32ty, 0, insttoreplace)); return true; } }; } char replacepass::id = 0; static void registerreplacepass(const passmanagerbuilder &, legacy::passmanagerbase &pm) { pm.add(new re
Read more

java.lang.NoClassDefFoundError When Creating New Android Project -

-
i've tried installing (and re-installing) android studio several times now, , each time, studio isn't able create new android project. when attempt initial android studio window (ie. basic 'create', 'open', 'import' etc options), i'm asked of questions in wizard, brief progress dialog, followed nothing @ all. if open incomplete project process creates choosing 'open' list, can try choosing create new android project menu. this has same issue, error message in output console time... java.lang.noclassdeffounderror: com/sun/xml/internal/bind/v2/model/annotation/annotationreader (wrong name: com/sun/xml/internal/bind/v0/model/qnnotation/annotationreader) note: when installed android studio, installed incompatible jdk (1.8, not 1.7). have had go default project settings change jdk uses 1.8. does know how can avoid this? there place should downloading jxb classes? missing in java paths? this appears have been caused cor
Read more

Decoding a Python 2 `tempfile` with python-future -

-
i'm attempting write python 2/3 compatible routine fetch csv file, decode latin_1 unicode , feed csv.dictreader in robust, scalable manner. for python 2/3 support, i'm using python-future including imporing open builtins , , importing unicode_literals consistent behaviour i'm hoping handle exceptionally large files spilling disk, using tempfile.spooledtemporaryfile i'm using io.textiowrapper handle decoding latin_1 encoding before feeding dictreader this works fine under python 3. the problem textiowrapper expects wrap stream conforms bufferediobase . unfortunately under python 2, although have imported python 3-style open , vanilla python 2 tempfile.spooledtemporaryfile still of course returns python 2 cstringio.stringo , instead of python 3 io.bytesio required textiowrapper . i can think of these possible approaches: wrap python 2 cstringio.stringo python 3-style io.bytesio . i'm not sure how approach - need write such wrapper or 1 ex
Read more