Research Software Engineering with Python: Building software that makes research possible

Writing and running software is now as much a part of science as telescopes and test tubes, but most researchers are never taught how to do either well. As a result, it takes them longer to accomplish simple tasks than it should, and it is harder for them to share their work with others than it needs to be. This book introduces the concepts, tools, and skills that researchers need to get more done in less time and with less pain. Based on the practical experiences of its authors, who collectively have spent several decades teaching software skills to scientists, it covers everything graduate-level researchers need to automate their workflows, collaborate with colleagues, ensure that their results are trustworthy, and publish what they have built so that others can build on it. The book assumes only a basic knowledge of Python as a starting point, and shows readers how it, the Unix shell, Git, Make, and related tools can give them more time to focus on the research they actually want to do. Research Software Engineering with Python can be used as the main text in a one-semester course or for self-guided study. A running example shows how to organize a small research project step by step; over a hundred exercises give readers a chance to practice these skills themselves, while a glossary defining over two hundred terms will help readers find their way through the terminology. All of the material can be re-used under a Creative Commons license, and all royalties from sales of the book will be donated to The Carpentries, an organization that teaches foundational coding and data science skills to researchers worldwide.

Welcome0.1The Big Picture0.2 Intended Audience0.3 What You Will Learn0.4 Using this Book0.5 Contributing and Re-Use0.6 AcknowledgmentsGetting Started1.1 Project Structure1.2 Downloading the Data1.3 Installing the Software1.4 Summary1.5 Exercises1.6 Key PointsThe Basics of the Unix Shell2.1 Exploring Files and Directories2.2 Moving Around2.3 Creating New Files and Directories2.4 Moving Files and Directories2.5 Copying Files and Directories2.6 Deleting Files and Directories2.7 Wildcards2.8 Reading the Manual2.9 Summary2.10 Exercises2.11 Key PointsBuilding Tools with the Unix Shell3.1 Combining Commands3.2 How Pipes Work3.3 Repeating Commands on Many Files3.4 Variable Names3.5 Redoing Things3.6 Creating New Filenames Automatically3.7 Summary3.8 Exercises3.9 Key PointsGoing Further with the Unix Shell4.1 Creating New Commands4.2 Making Scripts More Versatile4.3 Turning Interactive Work into a Script4.4 Finding Things in Files4.5 Finding Files4.6 Configuring the Shell4.7 Summary4.8 Exercises .4.9 Key PointsBuilding Command-Line Tools with Python5.1 Programs and Modules5.2 Handling Command-Line Options5.3 Documentation5.4 Counting Words5.5 Pipelining5.6 Positional and Optional Arguments…