Intermediate Research Software Development with Python

Netherlands eScience Center

december 11 - 13, 2023

9:30 - 17:00 CET

Instructors: Sven van der Burg, Sander van Rijn, Candace Moore, Flavio Hafner

Helpers:

Some adblockers block the registration window. If you do not see the registration box below, please check your adblocker settings.

General Information

The eScience Center offers a range of free workshops and training courses, open to all researchers affiliated with Dutch research organizations. We organize workshops covering digital skills needed to put reproducible research into practice. These include online collaboration, reproducible code and good programming practices. We also offer more advanced workshops such as GPU Programming, Parallel Programming and Deep Learning.

This course aims to teach a core set of established, intermediate-level software development skills and best practices for working as part of a team in a research environment using Python as an example programming language. The core set of skills we teach is a selective set of tried-and-tested collaborative development skills that forms a firm foundation for continuing on your learning journey..

A typical learner for this course may be someone who is working in a research environment, needing to write some code, has gained basic software development skills either by self-learning or attending, e.g., a novice Software Carpentry Python course. They have been applying those skills in their domain of work by writing code for some time, e.g. half a year or more. However, their software development-related projects are now becoming larger and are involving more researchers and other stakeholders (e.g. users), for example:

Software is becoming more complex and more collaborative development effort is needed to keep the software running
Software is going further that just the small group developing and/or using the code - there are more users and an increasing need to add new features
‘Technical debt’ is increasing with demands to add new functionality while ensuring previous development efforts remain functional and maintainable

They now need intermediate software engineering skills to help them design more robust software code that goes beyond a few thrown-together proof-of-concept scripts, taking into consideration the lifecycle of software, writing software for stakeholders, working in a team and applying a process to understanding, designing, building, releasing, and maintaining software.

Who:

This course is for you if:

You have been writing software for a while, which may be used by people other than yourself, but it is currently undocumented or unstructured
You want to learn:
- more intermediate software engineering techniques and tools
- how to collaborate with others to develop software
- how to prepare software for others to use
You are currently comfortable with:
- basic Python programming (though this may not be the main language you use) and applying it to your work on a regular basis
- basic version control using Git
- command line interface (shell)

This course is not for you if:

You have not yet started writing software (in which case have a look at the Software Carpentry course or some other Python course for novices first)
You have learned the basics of writing software but have not applied that knowledge yet (or are unsure how to apply it) to your work. In this case, we suggest you revisit the course after you have been programming for at least 6 months
You are already reasonably familiar with the learning objectives of the course and those of individual episodes
The software you write is fully documented and well architected

Prerequisites:

To attend this course you should meet the following criteria. You can also test your prerequisite knowledge by taking this short quiz.

Git
- You are familiar with the concept of version control
- You have experience configuring Git for the first time and creating a local repository
- You have experience using Git to create and clone a repository and add/commit changes to it and to push to/pull from a remote repository
- Ideally, you have experience comparing various versions of tracked files or ignoring specific files
Python
- You have a basic knowledge of programming in Python (using variables, lists, conditional statements, functions and importing external libraries)
- You have previously written Python scripts or iPython/Jupyter notebooks to accomplish tasks in your domain of work
Shell
- You have experience using a command line interface, such as Bash, to navigate a UNIX-style file system and run commands with arguments
- Ideally, you have experience redirecting inputs and outputs from a command

Instead of this workshop you could also consider: If you find the topics that are covered in this workshop slightly too advanced, you could consider joining the ‘Good Practices in Research Software Development’ workshop. That workshop has a similar focus, but is targeted to a slightly less advanced audience. For example, there is an optional introduction to git. Checkout our upcoming workshops for more information.

Where: Science Park 402, 1098 XH Amsterdam. Get directions with OpenStreetMap or Google Maps.

When: december 11 - 13, 2023, 9:30 - 17:00 CET.

Requirements: Participants must bring a laptop with a Mac, Linux, or Windows operating system (not a tablet, Chromebook, etc.) that they have administrative privileges on. They should have a few specific software packages installed (listed below).

Accessibility: We are committed to making this workshop accessible to everybody. The workshop organizers have checked that:

The room is wheelchair / scooter accessible.
Accessible restrooms are available.

Materials will be provided in advance of the workshop and large-print handouts are available if needed by notifying the organizers in advance. If we can help making learning easier for you (e.g. sign-language interpreters, lactation facilities) please get in touch (using contact details below) and we will attempt to provide them.

Contact: Please email or training@esciencecenter.nl for more information.

Code of Conduct

Participants are expected to follow these guidelines:

Use welcoming and inclusive language
Be respectful of different viewpoints and experiences
Gracefully accept constructive criticism
Focus on what is best for the community
Show courtesy and respect towards other community members

Syllabus

Setting up Environment For Collaborative Code Development

Introduction to Our Software Project
Virtual Environments For Software Development
Integrated Software Development Environments
Collaborative Software Development Using Git and gitHub
Python Code Style Conventions
Verifying Code Style Using Linters

Ensuring Correctness of Software at Scale

Automatically Testing Software
Scaling Up Unit Testing
Continuous Integration for Automated Testing
Diagnosing Issues and Improving Robustness

Software Development as a Process

Software Requirements
Software Architecture and Design
Programming Paradigms
Functional Programming Intuition
Object Oriented Programming Intuition

Collaborative Software Development for Reuse

Developing Software in a Team: Code Review
Preparing Software for Reuse and Release
Packaging Code for Release and Distribution

Managing and Improving Software over Its Lifetime

Managing a Collaborative Software Project
Software Improvement Through Feedback

Schedule

Day 1

time	what
09:30	Welcome and icebreaker
09:45	Section 1: Setting up Environment For Collaborative Code Development
10:30	Break
10:40	Section 1: Setting up Environment For Collaborative Code Development
11:30	Break
11:40	Section 1: Setting up Environment For Collaborative Code Development
12:30	Lunch Break
13:30	Section 2: Ensuring Correctness of Software at Scale
14:30	Break
14:40	Section 2: Ensuring Correctness of Software at Scale
15:30	Break
15:40	Section 2: Ensuring Correctness of Software at Scale
16:15	Wrap-up
16:30	END

Day 2

time	what
09:30	Welcome and recap
09:45	Section 3: Software Development as a Process
10:30	Break
10:40	Section 3: Software Development as a Process
11:30	Break
11:40	Section 3: Software Development as a Process
12:30	Lunch Break
13:30	Section 4: Collaborative Software Development for Reuse
14:30	Break
14:40	Section 4: Collaborative Software Development for Reuse
15:30	Break
15:40	Section 4: Collaborative Software Development for Reuse
16:15	Wrap-up
16:30	END

Day 3

time	what
09:30	Welcome and recap
09:45	Section 5: Managing and Improving Software over Its Lifetime
10:30	Break
10:40	Section 5: Managing and Improving Software over Its Lifetime
11:30	Break
11:40	Section 5: Managing and Improving Software over Its Lifetime
12:30	Lunch Break
13:30	Apply to your own project
14:30	Break
14:40	Apply to your own project
15:30	Break
15:40	Apply to your own project
16:15	Post-workshop Survey
16:30	Drinks

Setup

To participate in this workshop, you will need access to software as described below. In addition, you will need an up-to-date web browser.

We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.

Software setup

It is important that you setup everything on your laptop before the start of the course. This includes setting up a Command Line Tool, Git, a Python environment, and Integrated Development Environment.. Please follow these setup instructions. Send us an email if you encounter any problems.