Brief description:

  • Generation of reference data (with reference job)
  • Generation of other data by other time stepping methods / time step sizes
  • Computing the difference
  • Plotting the results

Supplementary material: corresponding tutorial folder

Benchmark job generation script

We skip some boilerplate code symbolized by this code (see tutorial folder for full code)

Job template

We can configure a default compile, runtime and parallelization parameters of this job. After this, we can make a deepcopy of it in order to reuse it later as a fresh job template.

jg = JobGeneration()

[...]

jgbase = copy.deepcopy(jg)
jg = None   # Deactivate it just to be sure

Reference job

To generate the reference data, we

  • tag this job as a reference job
  • create the job directory for it
  • untag this job as a reference job (to reuse the ‘jg’ also for other jobs)
# Create fresh job generation
jgref = copy.deepcopy(jgbase)

# Set this to true to say that this is one of the reference jobs
jgref.reference_job = True

# Create job with reference data
jgref.gen_jobscript_directory('job_benchref_'+jg.getUniqueID())

[...]

Benchmark jobs

Then, we can create all jobs

for tsm in ts_methods:
    [...]	# all benchamrks created here

We can start generating a new job generation script for each job

    jg = copy.deepcopy(jgbase)

To let the other jobs know to which data to compare to, we can set the referene job id in the other jobs:

    # We now reuse the unique job ID of the reference solution to tell the other jobs about their reference solution!
    jg.reference_job_unique_id = jgref.job_unique_id

Generate the job script

    [...]
    jg.gen_jobscript_directory('job_benchref_'+jg.getUniqueID())

Run jobs

We can run all benchmarks by just calling

$ mule.benchmark.jobs_run_directly_nonstop

Postprocessing

To compute the error norms we can simply call

$ mule.postprocessing.pickle.alljobs.sphere2d_data_norms_physical_space

Plotting

To plot the results, we can call

$ ./postprocessing_plot.py

The plots are generated as .pdf files in the current folder.

WARNING

The generated plots are only useful if they show time step size vs. error.

The wallcock time results shouldn’t be used for performance discussions since many things could have gone wrong.

This is discussed in the following HPC benchmark tutorial.