Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: parallelism v.python

by oiskuu (Pilgrim)
on Feb 28, 2014 at 21:54 UTC ( #1076622=note: print w/ replies, xml ) Need Help??


in reply to parallelism v.python

See python docs. I believe this is a problem with their hashlib implementation. (In python2.6 ?)


Comment on Re: parallelism v.python
Re^2: parallelism v.python
by BrowserUk (Pope) on Feb 28, 2014 at 22:09 UTC

    Perhaps this is (part of) the cause?

    CPython implementation detail: In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Holy batpoo! Yeah, that might explain a bit...
      BTW, been thinking about this a bit...

      If it was just a matter of parallelism V non-parallelism, that would be 1 issue.

      But the 2nd big issue: Look at the total cpu time used:

      lang #thrds #cores_used %efficency clocktime cputime(Usr+Sy +s) python 9 1.82 20.2% 67.35s 122.53s perl 9 8.70 96.7% 3.36s 29.23
      From the above, stated as percentages or multipliers, perl is 478% more efficient in making use of multicore resources.

      In real time, python takes 64 seconds longer over the base time that was needed, of 3.36s. Python is 1900% SLOWER.

      Someone mentioned python might not be optimal in parallelism due to threading problems.

      So look at the actual amount of CPUtime used for each to do the work. 122.53/29.23. For heavy number crunching, perl (with max precision possible in x86-64 HW, takes less than 1/4th the time, i.e. perl is 4.2x the speed of python.
      ----

      (Notes collected for a response to the "said" note writer...)

        What processor was this run on? (Can I safely assume that you were running in a non-VM environment?)


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        Yup. https://wiki.python.org/moin/Concurrency/ applies. In short: Don't use threading on CPython unless you have to.

        Also for multiline prints, you can either wrap in brackets, use triple quotes, or use \ line continuance.

        ...and some say that Python has only one way of doing things... ;-)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1076622]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (12)
As of 2014-10-30 16:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (208 votes), past polls