Shell scripting essentials for large-scale data processing
This lab introduces shell scripting for automating repetitive tasks in Rosetta workflows. Instead of manually processing thousands of files, you’ll learn how to use bash scripts to streamline analysis of PDBs, score files, and silent files. The session covers core concepts like scripting structure, permissions, and execution, along with powerful command-line tools such as awk, sed, and grep for searching, filtering, and transforming data. You’ll also see how piping enables chaining commands into efficient workflows and how real-world examples can automate selecting top-scoring structures from large datasets. By the end, you’ll understand when to use shell scripting versus Python or C++ and how to customize your shell environment for productivity. This is a practical introduction designed to save time and reduce manual errors in large-scale computational projects.
Presenter: Lilly (Mac) Bolz
Affiliation: UCLA
Chapters
00:00 Introduction and motivation for automation
01:30 Rosetta outputs: PDBs, score files, silent files
02:47 Shell scripting basics and use cases
04:06 Choosing the right tool: bash vs Python vs C++
05:07 Writing a shell script and key commands
09:05 Data processing tools: awk, sed, grep
14:00 Practical use cases and messy directory cleanup
20:02 End-to-end automation example (score file → structures)
22:13 Shell environments, shortcuts, and best practices
26:02 Lab tasks and workflow overview
In questa pagina del sito puoi guardare il video online Shell scripting essentials for large-scale data processing della durata di ore minuti seconda in buona qualità , che l'utente ha caricato RosettaCommons 25 marzo 2026, condividi il link con amici e conoscenti, su youtube questo video è già stato visto 196 volte e gli è piaciuto 4 spettatori. Buona visione!
