Content from Progressive workflows
Last updated on 2026-01-06 | Edit this page
The sections below aim to support researchers to experiment with accessing an HPC system with peer support. Researchers will test a simple workflow that is useful in the context of their work. For this, they will develop a simple workflow based on existing code, libraries and datasets.
Simple workflow
This is a group exercise to develop and test a simple workflow within the context of your research and deploy in HPC.
Challenge: Simple workflow onboarding
- Select one of the codes already available within your project. This can be a Python script to clear a dataset, to process it or to analyse it. If easier, you can simplify an existing code, which can be shared with your group.
- Under one of your individual accounts create a local Git repository, as well as a repository on Github.com. Then, add the other members of the group to share the repository. Push the selected script and a sample dataset to the remote repository. Next, clone the repository and ensure you can run it in your local setup. If using Python, you will need to create a local Python environment to install (pip install or conda install) the libraries required.
- Login to BEDE via SSH with your credentials.
- Pull the Git repository and replicate the setup from your PC onto the front end.
- As a team, create a SLURM job to run the script, including creating any environment that is required. Run the job and see the output.
Advanced workflow
This is a group exercise to test an advanced workflow which could be relevant to the context of your research and deploy in HPC.
Challenge: Advanced workflow onboarding
Select a tutorial, which uses a programming language and AI libraries, from the lists below. Create a new Git repository with the sample code and deploy as above in BEDE.
Examples of such exercises/tutorials are:
Natural Language Processing (NLP)
Understanding and Creating Word Embeddings
Visualisation
Content from Test and discuss
Last updated on 2026-01-06 | Edit this page
This section is dedicated to working on researchers’ workflows to prepare for deployment onto HPC systems. Group reflection is also included to identify issues and explore potential collaboration by sharing knowledge, including solutions and code.
Challenge: Test your workflows
In 20 minutes, work on your own workflows to:
- Specify libraries and tools which need to be “transferred or prepared” for the HPC solution.
- Experiment with transferring a simple workflow adopting the approach followed in the previous section.
- Document the tasks which are performed at every stage. This will form the basis of the documentation you will need to provide.
- Identify questions, issues that need resolving and pain points in the workflow.
Challenge: Workflow collaboration
In 20 minutes, work in your groups to:
- Identify and discuss the next steps to onboard your individual workflows to HPC.
- Describe your workflow to your peers and investigate potential for sharing with them.
- Examine how feasible it is to exchange solutions/code with your peers.
Content from Further resources
Last updated on 2026-01-06 | Edit this page
Python:
Python, Coding for Reproducible Research Training Programme, University of Exeter
Introduction to Python For Humanists, Hot Source! project, University of Exeter
Python exercises:
https://www.practicepython.org/exercises/
https://www.w3resource.com/bash-script-exercises/index.php
https://github.com/krother/bash_tutorial
Git and Github:
Version control with Git and Github
HPC Training:
Digital Humanities & Research Software Engineering Summer School 2025
High performance computing for Digital Humanities - KCL E-research
Coding for Reproducible Research (CfRR) Training: High Performance Computing