JM

High Performance Computing in Julia

Course Info

  1. 1. Introduction

    1. 1.1. Setup

  2. 2. Foundations

    1. 2.1. Overview
    2. 2.2. von Neumann Architecture
    3. 2.3. Data Types
    4. 2.4. Operating Systems
    5. 2.5. Compilers and Machine Code
    6. 2.6. Hardware
    7. 2.7. Summary

  3. 3. Julia

    1. 3.1. Overview

  4. 4. Measuring Performance

    1. 4.1. Overview
    2. 4.2. Timing
    3. 4.3. Benchmarking
    4. 4.4. Profiling
    5. 4.5. Memory Allocations
    6. 4.6. Computational Complexity

  5. 5. Optimisation

    1. 5.1. Overview
    2. 5.2. Caching and Memory Locality
    3. 5.3. Stack vs Heap Memory
    4. 5.4. Reducing Allocations
    5. 5.5. Memoisation
    6. 5.6. Vectorisation vs Loops
    7. 5.7. Views
    8. 5.8. Speed of Operations
    9. 5.9. Type Stability
    10. 5.10. Performance Annotations
    11. 5.11. Conclusion

  6. 6. Parallel Programming

    1. 6.1. Overview
    2. 6.2. Dependency Graphs
    3. 6.3. Theoretical Expectations
    4. 6.4. Maps and Reductions
    5. 6.5. Thread Safety and Race Conditions
    6. 6.6. Producer and Consumer
    7. 6.7. When to Parallelise?

  7. 7. Multithreading

    1. 7.1. Overview
    2. 7.2. Monte Carlo Pi Estimation
    3. 7.3. Naive Parallel Implementation
    4. 7.4. False Sharing
    5. 7.5. Chunking
    6. 7.6. Task-based Parallelism
    7. 7.7. Packages
    8. 7.8. Summary
    9. 7.9. Exercises

  8. 8. GPU Programming

    1. 8.1. Introduction to GPU Programming
    2. 8.2. Modern GPU Hardware
    3. 8.3. GPU Vendor Overview
    4. 8.4. High Level Introduction to CUDA
    5. 8.5. CUDA Kernels
    6. 8.6. CUDA Libraries
    7. 8.7. Benchmarking & Profiling
    8. 8.8. Tips
    9. 8.9. Case Study - Monte-Carlo Simulations
    10. 8.10. Exercises

Table of Contents

Views

We have seen that allocating memory on the heap can negatively impact performance, but what if we already have allocated memory, and simply want a convenient way to look at a part of an array, without increasing code complexity. This is where the view comes in handy. If you have used Python or MATLAB, you will be familiar with slicing syntax:

julia 
a = rand(3,3)
b = a[1:2, 1:2]

This syntax, allows us to take a section of an array and manipulate it. The only issue is that we have just created a copy of that part of the array! To prove this, let’s mutate part of b and then look at the values of a:

julia 
b .= 0
a

Notice that a is unchanged. This is a good behaviour by default, as if any other behaviour were implemented, this would lead to countless bugs. It is important to remember that simply assigning a variable to an array, will assign by reference and hence not make a copy, since the array variable is simply a pointer to the start of the array:

julia 
c = a; # c and a point to the same memory now

However, if we want to mutate part of an array, while not copying the memory, we can just use a view. There is a handy macro that does this for us. Take the first example using the @views macro:

julia 
a = rand(3,3)  # Reset a
b = @views a[1:2, 1:2]

Now, b points to the same four elements in memory as the top-left square of a. To show this, let’s manipulate b and check a:

julia 
b .= 0
a

Notice that we have propagated this information back to a. This can be very useful when one would like to avoid making copies of arrays, but needs to access only part of the array.

Note: One should still be very careful, since the memory layout has not changed, but the indexing scheme has changed. This means that even though one may loop through the view using the correct indexing scheme, it is likely that the memory is not contiguous and hence can have severe performance penalties for using a view. In situations where memory locality is more impactful than allocating memory, a copy may be worth making. Again, this trade-off comes down to benchmarking, necessary for making an informed decision.

Back Next