Thursday, September 8, 2016

Many core functions now parallelized

I've now got parallelized code running for the background and identity tests, as well as the linear and blob rangebreak tests.  The ribbon rangebreak test is going to take longer, because it contains some necessary failure detection code that needs to be wrapped differently than the other tests.

This code is not yet on the main branch on GitHub, it's on the branch named "apply".

As it stands, each of the functions by default uses all of the cores available in the system.  You can decrease that by just supplying a "cores = x" argument, where x is however many cores you want it to use.  If you're happy using all of the cores on your system, you can just call the functions exactly as before.

Obviously the speed differences here are going to depend on how many cores you have on your system.  I've got a 24-core machine I'm working on right now, and going from 1 core to 24 on my test data results in massive speed increases - identity and background tests for 20 reps drop from ~10 minutes to ~1 minute.  Pretty slick!

Anyway, give it a shot if you can and let me know if you run into any issues with it.  Thanks to Nick Huron for reminding me!

No comments:

Post a Comment