Advanced Bash-Scripting Guide

Advanced Bash-Scripting Guide

An in-depth exploration of the art of shell scripting

Mendel Cooper

10

10 Mar 2014

Revision History

Revision 6.5

'TUNGSTENBERRY' release

Revision 6.6

'YTTERBIUMBERRY' release

Revision 10

'PUBLICDOMAIN' release

05 Apr 2012

Revised by: mc

27 Nov 2012

Revised by: mc

10 Mar 2014

Revised by: mc

This tutorial assumes no previous knowledge of scripting or programming, yet progresses rapidly toward an

intermediate/advanced level of instruction . . . all the while sneaking in little nuggets of UNIX? wisdom and

lore. It serves as a textbook, a manual for self-study, and as a reference and source of knowledge on shell

scripting techniques. The exercises and heavily-commented examples invite active reader participation, under

the premise that the only way to really learn scripting is to write scripts.

This book is suitable for classroom use as a general introduction to programming concepts.

This document is herewith granted to the Public Domain. No copyright!

Dedication

For Anita, the source of all the magic

Advanced Bash-Scripting Guide

Table of Contents

Chapter 1. Shell Programming!.........................................................................................................................1

Chapter 2. Starting Off With a Sha-Bang........................................................................................................3

2.1. Invoking the script............................................................................................................................6

2.2. Preliminary Exercises.......................................................................................................................6

Part 2. Basics.......................................................................................................................................................7

Chapter 3. Special Characters...........................................................................................................................8

Chapter 4. Introduction to Variables and Parameters..................................................................................30

4.1. Variable Substitution......................................................................................................................30

4.2. Variable Assignment.......................................................................................................................33

4.3. Bash Variables Are Untyped..........................................................................................................34

4.4. Special Variable Types...................................................................................................................35

Chapter 5. Quoting...........................................................................................................................................41

5.1. Quoting Variables...........................................................................................................................41

5.2. Escaping..........................................................................................................................................43

Chapter 6. Exit and Exit Status.......................................................................................................................51

Chapter 7. Tests................................................................................................................................................54

7.1. Test Constructs...............................................................................................................................54

7.2. File test operators............................................................................................................................62

7.3. Other Comparison Operators..........................................................................................................65

7.4. Nested if/then Condition Tests.......................................................................................................70

7.5. Testing Your Knowledge of Tests..................................................................................................71

Chapter 8. Operations and Related Topics....................................................................................................72

8.1. Operators.........................................................................................................................................72

8.2. Numerical Constants.......................................................................................................................78

8.3. The Double-Parentheses Construct.................................................................................................80

8.4. Operator Precedence.......................................................................................................................81

Part 3. Beyond the Basics.................................................................................................................................84

Chapter 9. Another Look at Variables...........................................................................................................85

9.1. Internal Variables............................................................................................................................85

9.2. Typing variables: declare or typeset.............................................................................................104

9.2.1. Another use for declare.......................................................................................................107

9.3. $RANDOM: generate random integer..........................................................................................107

Chapter 10. Manipulating Variables.............................................................................................................119

10.1. Manipulating Strings...................................................................................................................119

10.1.1. Manipulating strings using awk........................................................................................127

10.1.2. Further Reference..............................................................................................................127

10.2. Parameter Substitution................................................................................................................128

i

Advanced Bash-Scripting Guide

Table of Contents

Chapter 11. Loops and Branches..................................................................................................................138

11.1. Loops..........................................................................................................................................138

11.2. Nested Loops..............................................................................................................................152

11.3. Loop Control...............................................................................................................................153

11.4. Testing and Branching................................................................................................................156

Chapter 12. Command Substitution.............................................................................................................165

Chapter 13. Arithmetic Expansion................................................................................................................171

Chapter 14. Recess Time................................................................................................................................172

Part 4. Commands..........................................................................................................................................173

Chapter 15. Internal Commands and Builtins.............................................................................................181

15.1. Job Control Commands..............................................................................................................210

Chapter 16. External Filters, Programs and Commands...........................................................................215

16.1. Basic Commands........................................................................................................................215

16.2. Complex Commands...................................................................................................................221

16.3. Time / Date Commands..............................................................................................................231

16.4. Text Processing Commands........................................................................................................235

16.5. File and Archiving Commands...................................................................................................258

16.6. Communications Commands......................................................................................................276

16.7. Terminal Control Commands.....................................................................................................291

16.8. Math Commands.........................................................................................................................292

16.9. Miscellaneous Commands..........................................................................................................303

Chapter 17. System and Administrative Commands..................................................................................318

17.1. Analyzing a System Script..........................................................................................................349

Part 5. Advanced Topics.................................................................................................................................351

Chapter 18. Regular Expressions..................................................................................................................353

18.1. A Brief Introduction to Regular Expressions..............................................................................353

18.2. Globbing.....................................................................................................................................357

Chapter 19. Here Documents.........................................................................................................................359

19.1. Here Strings................................................................................................................................369

Chapter 20. I/O Redirection...........................................................................................................................373

20.1. Using exec...................................................................................................................................376

20.2. Redirecting Code Blocks............................................................................................................379

20.3. Applications................................................................................................................................384

Chapter 21. Subshells.....................................................................................................................................386

ii

Advanced Bash-Scripting Guide

Table of Contents

Chapter 22. Restricted Shells.........................................................................................................................391

Chapter 23. Process Substitution...................................................................................................................393

Chapter 24. Functions....................................................................................................................................398

24.1. Complex Functions and Function Complexities.........................................................................402

24.2. Local Variables...........................................................................................................................413

24.2.1. Local variables and recursion............................................................................................414

24.3. Recursion Without Local Variables............................................................................................417

Chapter 25. Aliases.........................................................................................................................................420

Chapter 26. List Constructs...........................................................................................................................423

Chapter 27. Arrays.........................................................................................................................................427

Chapter 28. Indirect References....................................................................................................................456

Chapter 29. /dev and /proc.............................................................................................................................460

29.1. /dev..............................................................................................................................................460

29.2. /proc............................................................................................................................................463

Chapter 30. Network Programming.............................................................................................................469

Chapter 31. Of Zeros and Nulls.....................................................................................................................472

Chapter 32. Debugging...................................................................................................................................476

Chapter 33. Options........................................................................................................................................487

Chapter 34. Gotchas.......................................................................................................................................490

Chapter 35. Scripting With Style..................................................................................................................499

35.1. Unofficial Shell Scripting Stylesheet..........................................................................................499

Chapter 36. Miscellany...................................................................................................................................502

36.1. Interactive and non-interactive shells and scripts.......................................................................502

36.2. Shell Wrappers............................................................................................................................503

36.3. Tests and Comparisons: Alternatives..........................................................................................509

36.4. Recursion: a script calling itself..................................................................................................509

36.5. "Colorizing" Scripts....................................................................................................................512

36.6. Optimizations..............................................................................................................................525

36.7. Assorted Tips..............................................................................................................................528

36.7.1. Ideas for more powerful scripts.........................................................................................528

36.7.2. Widgets..............................................................................................................................539

36.8. Security Issues............................................................................................................................541

36.8.1. Infected Shell Scripts.........................................................................................................541

36.8.2. Hiding Shell Script Source................................................................................................541

iii

Advanced Bash-Scripting Guide

Table of Contents

Chapter 36. Miscellany

36.8.3. Writing Secure Shell Scripts.............................................................................................541

36.9. Portability Issues.........................................................................................................................541

36.9.1. A Test Suite.......................................................................................................................542

36.10. Shell Scripting Under Windows...............................................................................................543

Chapter 37. Bash, versions 2, 3, and 4..........................................................................................................544

37.1. Bash, version 2............................................................................................................................544

37.2. Bash, version 3............................................................................................................................548

37.2.1. Bash, version 3.1...............................................................................................................551

37.2.2. Bash, version 3.2...............................................................................................................552

37.3. Bash, version 4............................................................................................................................552

37.3.1. Bash, version 4.1...............................................................................................................559

37.3.2. Bash, version 4.2...............................................................................................................560

Chapter 38. Endnotes.....................................................................................................................................564

38.1. Author's Note..............................................................................................................................564

38.2. About the Author........................................................................................................................564

38.3. Where to Go For Help.................................................................................................................565

38.4. Tools Used to Produce This Book..............................................................................................565

38.4.1. Hardware...........................................................................................................................565

38.4.2. Software and Printware.....................................................................................................565

38.5. Credits.........................................................................................................................................566

38.6. Disclaimer...................................................................................................................................567

Bibliography....................................................................................................................................................569

Appendix A. Contributed Scripts..................................................................................................................577

Appendix B. Reference Cards........................................................................................................................787

Appendix C. A Sed and Awk Micro-Primer................................................................................................792

C.1. Sed................................................................................................................................................792

C.2. Awk..............................................................................................................................................795

Appendix D. Parsing and Managing Pathnames.........................................................................................798

Appendix E. Exit Codes With Special Meanings.........................................................................................802

Appendix F. A Detailed Introduction to I/O and I/O Redirection.............................................................803

Appendix G. Command-Line Options..........................................................................................................805

G.1. Standard Command-Line Options...............................................................................................805

G.2. Bash Command-Line Options......................................................................................................806

Appendix H. Important Files.........................................................................................................................808

iv

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download