Welcome to the Julia for High-performance Computing Tutorial

Goal

The goal of the tutorial is to introduce an audience familiar with high-performance computing (HPC) concepts to the Julia programming language using a hands-on approach.

It’s a good time to get Julia v1.9 installed from their website

Covered:

  • Workflow aspects: simulation CPU/CPU, parallel I/O, data analysis (Notebooks)
  • Multiple dispatch for array back ends (CPU, CUDA, AMDGPU)
  • Multithreaded CPU using Julia @threads
  • GPU programming using CUDA.jl and AMDGPU.jl
  • Parallel I/O with ADIOS2.jl (outputs can be visualized in ParaView)
  • Julia on OLCF’s JupyterHub and Pluto.jl Notebooks (read and plot)

Not Covered (future topics?):

  • More general aspects of the language (software engineering, AI)
  • AI infrastructure: Flux.jl, SciML
  • Performance portability layers: e.g. KernelAbstractions.jl
  • Ahead-of-time compilation: PackageCompiler.jl
  • Tools ecosystem for profiling, debugging (well some)

Why Julia?

“Can a machine translate a sufficiently rich mathematical language into a suffienciently economical program at a sufficiently low cost to make the whole affair feasible?” – John Backus on Fortran (1980)

  1. Compiled via LLVM for performance. Julia is Compiled
  2. Provides a unified tested ecosystem (packaging, testing, tools, interactivity, etc.). See JuliaCon presentation on GeneralRegistry.
  3. Community of HPC people from day-1 (the community is you, low entry barrier)
  4. Sustained by JuliaHub, MIT’s JuliaLab, NumFocus and several open-source contributors
  5. Great math (Fortran) and data science (Python) simple syntax
  6. Rapid prototyping: stay in one language+ecosystem for your custom kernels + data analysis
  7. Interoperate with Fortran, C, C++ (via C), Python, R code in a lightweight manner
  8. Renewed interest for high-performance + high-productivity languages
  9. Great first language to expose newcomers to both HPC + math concepts
  10. Contribute to HPC stack: CUDA.jl, MPI.jl, AMDGPU.jl, ADIOS2.jl, HDF5.jl…for us, by us

Julia’s value proposition

Why not Julia?

  1. Already existing infrastructure/expertise (Fortran is perfect!)
  2. Too much of an investment to learn a new language
  3. Performance/productivity solutions already in place
  4. Having N-language/ecosystems is not a problem
  5. Python + X is well supported for your needs
  6. Garbage collection, Just-in-time (JIT) compilation are issues
  7. Developing low-level system libraries
  8. Domain outside “technical computing”
  9. Overall, you do not see any value added

Resources

  1. Julia’s value proposition for Better Scientific Software. High-level intro and many resources for Scientific Software.

  2. Bridging HPC Communities through the Julia Programming Language

  3. JuliaParallel.org. Checkout the monthly HPC call and previous events: SC22 BoF, ECP BoF, Tutorial days.

  4. JuliaCon

  5. Exascale Computing Project Julia for HPC Tutorial.

  6. OLCF September User’s Call presentations

  7. Julia for ORNL Science Workshop

  8. Julia Basics Notebooks. Run on myBinder or locally.

  9. From zero to Julia!. Series of introductory lessons.

  10. Julia for Sustainable HPC Software by Valentin Churavy.

  11. Godoy, William F., Pedro Valero-Lara, T. Elise Dettling, Christian Trefftz, Ian Jorquera, Thomas Sheehy, Ross G. Miller, Marc Gonzalez-Tallada, Jeffrey S. Vetter, and Valentin Churavy. “Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes.” IPDPS HIPS workshop (2023), arXiv:2303.06195

Acknowledgements

This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration.

ECP PROTEAS-TUNE, ASCR Bluestone, IDEAS projects, and the Sustainable Research Pathways Program

The many people in the Julia community that made this possible. Thanks to Suzanne Parete-Koon from the Oak Ridge Leadership Center (OLCF).