JM

High Performance Computing in Julia

Course Info

  1. 1. Introduction

    1. 1.1. Setup

  2. 2. Foundations

    1. 2.1. Overview
    2. 2.2. von Neumann Architecture
    3. 2.3. Data Types
    4. 2.4. Operating Systems
    5. 2.5. Compilers and Machine Code
    6. 2.6. Hardware
    7. 2.7. Summary

  3. 3. Julia

    1. 3.1. Overview

  4. 4. Measuring Performance

    1. 4.1. Overview
    2. 4.2. Timing
    3. 4.3. Benchmarking
    4. 4.4. Profiling
    5. 4.5. Memory Allocations
    6. 4.6. Computational Complexity

  5. 5. Optimisation

    1. 5.1. Overview
    2. 5.2. Caching and Memory Locality
    3. 5.3. Stack vs Heap Memory
    4. 5.4. Reducing Allocations
    5. 5.5. Memoisation
    6. 5.6. Vectorisation vs Loops
    7. 5.7. Views
    8. 5.8. Speed of Operations
    9. 5.9. Type Stability
    10. 5.10. Performance Annotations
    11. 5.11. Conclusion

  6. 6. Parallel Programming

    1. 6.1. Overview
    2. 6.2. Dependency Graphs
    3. 6.3. Theoretical Expectations
    4. 6.4. Maps and Reductions
    5. 6.5. Thread Safety and Race Conditions
    6. 6.6. Producer and Consumer
    7. 6.7. When to Parallelise?

  7. 7. Multithreading

    1. 7.1. Overview
    2. 7.2. Monte Carlo Pi Estimation
    3. 7.3. Naive Parallel Implementation
    4. 7.4. False Sharing
    5. 7.5. Chunking
    6. 7.6. Task-based Parallelism
    7. 7.7. Packages
    8. 7.8. Summary
    9. 7.9. Exercises

  8. 8. GPU Programming

    1. 8.1. Introduction to GPU Programming
    2. 8.2. Modern GPU Hardware
    3. 8.3. GPU Vendor Overview
    4. 8.4. High Level Introduction to CUDA
    5. 8.5. CUDA Kernels
    6. 8.6. CUDA Libraries
    7. 8.7. Benchmarking & Profiling
    8. 8.8. Tips
    9. 8.9. Case Study - Monte-Carlo Simulations
    10. 8.10. Exercises

Table of Contents

Tips

In order to fully utilise the additional resources provided by using a GPU, we must keep in mind some causes of poor performance.

Copying Data

Remember that a GPU has its own local memory, and any data processed by the GPU, must first be transferred onto the device memory. To make the terms clear, we refer to the main memory that the CPU has access to as the host memory. Often, you will see the CPU (and the rest of the machine) referred to as the host. The GPU only has access to its own memory, called device memory. It can take a long time for the host to copy memory to the GPU, and vice versa. This is why it is strongly recommended that all possible processing is done on the GPU until it is needed back on the CPU.

If you have a function that performs much better on the GPU, you must consider whether it is worth transferring the data to the GPU, processing it there, and then transferring it back to the CPU. Sometimes this will be beneficial, as the speed-up is so significant that the copy times are usually worth the cost. However, one can consider streamlining the processing of the data, so that it all occurs on the GPU. Even if one part of the process is actually faster on the CPU, it may be worth implementing a GPU version in order to keep the memory on the device and avoid copying.

Launching Kernels

An individual kernel launch can have quite a high latency, especially when compared with launching a CPU thread. While the CPU is constructing the kernel and sending it to the GPU for scheduling, the GPU may be quite idle. Launching many smaller kernels can incur a huge overhead cost, and cause the GPU to be idle for a large amount of time. If profiling with the NVIDIA tools, this is the performance bug to look out for.

To remedy this, one should attempt to fuse as many kernel calls together, and group the work into larger items. For example, if one is using array programming, make full use of fused broadcasting:

julia 
y .= sin.(x)
y .*= x
y .+= x .^ 3
y .= exp.(y)
# use a single statement instead
y .= exp.(x.*sin.(x) .+ x .^ 3)

While it may look better in the source code to split up the operations into multiple lines, it is often better to put these calls on a single line. The first section calls 4 4 kernels, whereas the single line only calls one kernel. If the call is extremely long, write a function for a single element and broadcast it across the entire array:

julia 
_f(x) = exp(x*sin(x) + x * x * x)
y .= _f.(x)

We will give an example of this optimisation in the case study section, by loading our kernel with work.

Back Next