python - Run many simulations simultaneously -


i want run many random deposition simulations in order analyze statistics.

the individual simulations aren't dependent on each other, each produces 1-d numpy array , want result 2-d array composed of these 1-d arrays.

i know nothing parallel computing have have 2 ideas how achieve this:

  1. simple for-loop take days run (i'm not in hurry after all)
  2. run same file many, each shorter for-loop, let os take care of distributing cpu power among runs , join files after finished

however, think there must better way this. current code following:

l = 60 n = 100 t = 100  hrubost_relax = np.zeros((n,t))  n in range(0,n):     level = np.zeros((t,l))     heights = np.zeros(level.shape[1])      t in range(1,level.shape[0]):         pos = np.random.randint(level.shape[1])          left = heights[pos-1] if pos-1>0 else heights[-1]         right = heights[pos+1] if pos+1<l else heights[0]          if left<heights[pos]:             if right<heights[pos]:                 direction = np.random.randint(2)*2-1                 heights[(pos+direction)%l] += 1             else:                 heights[pos-1] += 1         elif right<heights[pos]:             heights[(pos+1)%l] += 1         else:             heights[pos] += 1          hrubost_relax[n,t] = heights.std()/l 

i want parallelize outer for-loop

edit show solution multiprocessing.pool

from multiprocessing import pool  l = 60 n = 1000 t = 100000  hrubost_relax = np.zeros((n,t))  def deposition(n):     level = np.zeros((t,l))     heights = np.zeros(level.shape[1])     w = np.zeros(t)      t in range(1,level.shape[0]):         pos = np.random.randint(level.shape[1])          left = heights[pos-1] if pos-1>0 else heights[-1]         right = heights[pos+1] if pos+1<l else heights[0]          if left<heights[pos]:             if right<heights[pos]:                 direction = np.random.randint(2)*2-1                 heights[(pos+direction)%l] += 1             else:                 heights[pos-1] += 1         elif right<heights[pos]:             heights[(pos+1)%l] += 1         else:             heights[pos] += 1          w[t] = heights.std()/l      return w  p = multiprocessing.pool(4) i,x in enumerate(p.map(deposition, range(n))):     hrubost_relax[i] = x 

threading cannot give speedup on problem if using cpython implementation due use of global interpreter lock, allows single thread executing @ time (i assuming computations cpu-bound). if have multiple processors in computer, however, multiprocessing module let run number of processes in parallel. there still no real value trying run more processes have processors for, however, should consider using multiprocessing.pool


Comments

Popular posts from this blog

c++ - llvm function pass ReplaceInstWithInst malloc -

Cross-Compiling Linux Kernel for Raspberry Pi - ${CCPREFIX}gcc -v does not work -

java.lang.NoClassDefFoundError When Creating New Android Project -