$re = '/(?<command>^\s*[0-9].[^0-9][A-z ,’]+)/mis';
$str = ' Part I
Table of Contents
How to Use This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix
Basic Unix Environment
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 What’s Special About Unix? 3
1.2 Power Grows on You 4
1.3 The Core of Unix 6
1.4 Communication with Unix 6
1.5 Programs Are Designed to Work Together 8
1.6 There Are Many Shells 9
1.7 Which Shell Am I Running?
1.8 Anyone Can Program the Shell
1.9 Internal and External Commands
1.10 The Kernel and Daemons
1.11 Filenames
1.12 Filename Extensions
1.13 Wildcards
1.14 The Tree Structure of the Filesystem
1.15 Your Home Directory
1.16 Making Pathnames
1.17 File Access Permissions
1.18 The Superuser (Root)
1.19 When Is a File Not a File?
1.20 Scripting
1 1 1 1 1 3 1 4 1 6 1 7 1 8 2 0 2 2 2 3 2 5 2 6 2 7 2 7
iii
Part II
1.21 Unix Networking and Communications 2 8
1.22 The X Window System 3 0
2. Getting Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.1 The man Command 32
2.2 whatis: One-Line Command Summaries 33
2.3 whereis: Finding Where a Command Is Located 33
2.4 Searching Online Manual Pages 34
2.5 How Unix Systems Remember Their Names 36
2.6 Which Version Am I Using? 36
2.7 What tty Am I On? 37
2.8 Who’s On? 38
2.9 The info Command 38
Customizing Your Environment
3. Setting Up Your Unix Shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.1 What Happens When You Log In 43
3.2 The Mac OS X Terminal Application 44
3.3 Shell Setup Files—Which, Where, and Why 44
3.4 Login Shells, Interactive Shells 48
3.5 What Goes in Shell Setup Files? 49
3.6 Tip for Changing Account Setup: Keep a Shell Ready 50
3.7 Use Absolute Pathnames in Shell Setup Files 51
3.8 Setup Files Aren’t Read When You Want? 51
3.9 Gotchas in set prompt Test 53
3.10 Automatic Setups for Different Terminals 54
3.11 Terminal Setup: Testing TERM 55
3.12 Terminal Setup: Testing Remote Hostname and X Display 56
3.13 Terminal Setup: Testing Port 57
3.14 Terminal Setup: Testing Environment Variables 57
3.15 Terminal Setup: Searching Terminal Table 58
3.16 Terminal Setup: Testing Window Size 58
3.17 Terminal Setup: Setting and Testing Window Name 59
3.18 A .cshrc.$HOST File for Per Host Setup 60
3.19 Making a “Login” Shell 61
3.20 RC Files 62
3.21 Make Your Own Manpages Without Learning troff 65
3.22 Writing a Simple Manpage with the –man Macros 67
Table of Contents
iv
4. Interacting with Your Environment
. . . . . . . . . . . . . . . . . . . . . 70 70
4.1 Basics of Setting the Prompt
4.2 Static Prompts
4.3 Dynamic Prompts
4.4 Simulating Dynamic Prompts
4.5 C-Shell Prompt Causes Problems in vi, rsh, etc. 73
4.6 Faster Prompt Setting with Built-ins 74
4.7 Multiline Shell Prompts 76
4.8 Session Info in Window Title or Status Line 77
4.9 A “Menu Prompt” for Naive Users 79
4.10 Highlighting and Color in Shell Prompts 79
4.11 Right-Side Prompts 81
4.12 Show Subshell Level with $SHLVL 82
4.13 What Good Is a Blank Shell Prompt? 83
4.14 dirs in Your Prompt: Better Than $cwd 84
4.15 External Commands Send Signals to Set Variables 86
4.16 Preprompt, Pre-execution, and Periodic Commands 87
4.17 Running Commands When You Log Out 89
4.18 Running Commands at Bourne/Korn Shell Logout 90
4.19 Stop Accidental Bourne-Shell Logouts 90
5. Getting the Most out of Terminals, xterm,
and X Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.1 There’s a Lot to Know About Terminals
5.2 The Idea of a Terminal Database
5.3 Setting the Terminal Type When You Log In
5.4 Querying Your Terminal Type: qterm
5.5 Querying Your xterm Size: resize
5.6 Checklist: Terminal Hangs When I Log In
5.7 Find Out Terminal Settings with stty
5.8 Setting Your Erase, Kill, and Interrupt Characters
5.9 Working with xterm and Friends
5.10 Login xterms and rxvts
5.11 Working with Scrollbars
5.12 How Many Lines to Save?
5.13 Simple Copy and Paste in xterm
5.14 Defining What Makes Up a Word for Selection Purposes
5.15 Setting the Titlebar and Icon Text
5.16 The Simple Way to Pick a Font
Table of Contents
92 93 95 97 99
1 00 1 04 1 04 1 06 1 07 1 08 1 09 1 09 1 10 1 11 1 12
v
70 71 72
Part III
5.17 The xterm Menus 113
5.18 Changing Fonts Dynamically 115
5.19 Working with xclipboard 117
5.20 Problems with Large Selections 119
5.21 Tips for Copy and Paste Between Windows 120
5.22 Running a Single Command with xterm –e 122
5.23 Don’t Quote Arguments to xterm –e 123
6. Your X Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
6.1 Defining Keys and Button Presses with xmodmap 124
6.2 Using xev to Learn Keysym Mappings 127
6.3 X Resource Syntax 128
6.4 X Event Translations 130
6.5 Setting X Resources: Overview 133
6.6 Setting Resources with the –xrm Option 135
6.7 How –name Affects Resources 135
6.8 Setting Resources with xrdb 136
6.9 Listing the Current Resources for a Client: appres 139
6.10 Starting Remote X Clients 140
Working with Files and Directories
7. Directory Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.1 What? Me, Organized?
7.2 Many Homes
7.3 Access to Directories
7.4 A bin Directory for Your Programs and Scripts 149
7.5 Private (Personal) Directories 150
7.6 Naming Files 150
7.7 Make More Directories! 151
7.8 Making Directories Made Easier 152
8. Directories and Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
8.1 Everything but the find Command
8.2 The Three Unix File Times
8.3 Finding Oldest or Newest Files with ls –t and ls –u 155
8.4 List All Subdirectories with ls –R 157
8.5 The ls –d Option 157
8.6 Color ls 158
Table of Contents
147 148 148
1 54 1 54
vi
8.7 Some GNU ls Features
8.8 A csh Alias to List Recently Changed Files
8.9 Showing Hidden Files with ls –A and –a
8.10 Useful ls Aliases
8.11 Can’t Access a File? Look for Spaces in the Name
8.12 Showing Nonprintable Characters in Filenames
8.13 Counting Files by Types
8.14 Listing Files by Age and Size
8.15 newer: Print the Name of the Newest File
8.16 oldlinks: Find Unconnected Symbolic Links
8.17 Picking a Unique Filename Automatically
9. Finding Files with find . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9.1 How to Use find
9.2 Delving Through a Deep Directory Tree
9.3 Don’t Forget –print
9.4 Looking for Files with Particular Names
9.5 Searching for Old Files
9.6 Be an Expert on find Search Operators
9.7 The Times That find Finds
9.8 Exact File-Time Comparisons
9.9 Running Commands on What You Find
9.10 Using –exec to Create Custom Tests
9.11 Custom –exec Tests Applied
9.12 Finding Many Things with One Command
9.13 Searching for Files by Type
9.14 Searching for Files by Size
9.15 Searching for Files by Permission
9.16 Searching by Owner and Group
9.17 Duplicating a Directory Tree
9.18 Using “Fast find” Databases
9.19 Wildcards with “Fast find” Database
9.20 Finding Files (Much) Faster with a find Database
9.21 grepping a Directory Tree
9.22 lookfor: Which File Has That Word?
9.23 Using Shell Arrays to Browse Directories
9.24 Finding the (Hard) Links to a File
9.25 Finding Files with –prune
9.26 Quick finds in the Current Directory
Table of Contents
1 61 1 62 1 63 1 63 165 166 167 168 1 69 1 69 1 70
171
1 71 1 73 175 175 175 176 178 179 179 181 182 182 184 185 185 186 187 187 189 190 192 193 194 197 198 199
vii
9.27 Skipping Parts of a Tree in find
9.28 Keeping find from Searching Networked Filesystem
1 9 9 2 0 0
10. Linking, Renaming, and Copying Files . . . . . . . . . . . . . . . . 201
10.1 What’s So Complicated About Copying Files 201
10.2 What’s Really in a Directory? 201
10.3 Files with Two or More Names 203
10.4 More About Links 205
10.5 Creating and Removing Links 208
10.6 Stale Symbolic Links 209
10.7 Linking Directories 210
10.8 Showing the Actual Filenames for Symbolic Links 212
10.9 Renaming, Copying, or Comparing a Set of Files 212
10.10 Renaming a List of Files Interactively 213
10.11 One More Way to Do It 213
10.12 Copying Directory Trees with cp –r 214
10.13 Copying Directory Trees with tar and Pipes 216
11. Comparing Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
11.1 Checking Differences with diff 218
11.2 Comparing Three Different Versions with diff3 220
11.3 Context diffs 221
11.4 Side-by-Side diffs: sdiff 224
11.5 Choosing Sides with sdiff 225
11.6 Problems with diff and Tabstops 225
11.7 cmp and diff 226
11.8 Comparing Two Files with comm 227
11.9 More Friendly comm Output 229
11.10 make Isn’t Just for Programmers! 230
11.11 Even More Uses for make
12. Showing What’s in a File
232
236
12.1 Cracking the Nut
12.2 What Good Is a cat?
12.3 “less” is More
12.4 Show Nonprinting Characters with cat –v or od –c 237
12.5 What’s in That Whitespace? 239
12.6 Finding File Types 240
12.7 Squash Extra Blank Lines 241
12.8 How to Look at the End of a File: tail 242
viii Table of Contents
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 234 234
12.9 Finer Control on tail
12.10 How to Look at Files as They Grow
12.11 GNU tail File Following
12.12 Printing the Top of a File
12.13 Numbering Lines
243 243 245 246 246
247
247 248 249 250 251 251 252 254 255 256 257 258 258 2 59 2 59 2 60
262
2 62 2 62 2 63 265 265 266 266 267 268 269 269 270 271 271
ix
13. Searching Through Files
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13.1 Different Versions of grep
13.2 Searching for Text with grep
13.3 Finding Text That Doesn’t Match
13.4 Extended Searching for Text with egrep
13.5 grepping for a List of Patterns
13.6 Approximate grep: agrep
13.7 Search RCS Files with rcsgrep
13.8 GNU Context greps
13.9 A Multiline Context grep Using sed
13.10 Compound Searches
13.11 Narrowing a Search Quickly
13.12 Faking Case-Insensitive Searches
13.13 Finding a Character in a Column
13.14 Fast Searches and Spelling Checks with “look”
13.15 Finding Words Inside Binary Files
13.16 A Highlighting grep
14. Removing Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.1 The Cycle of Creation and Destruction
14.2 How Unix Keeps Track of Files: Inodes
14.3 rm and Its Dangers
14.4 Tricks for Making rm Safer
14.5 Answer “Yes” or “No” Forever with yes
14.6 Remove Some, Leave Some
14.7 A Faster Way to Remove Files Interactively
14.8 Safer File Deletion in Some Directories
14.9 Safe Delete: Pros and Cons
14.10 Deletion with Prejudice: rm –f
14.11 Deleting Files with Odd Names
14.12 Using Wildcards to Delete Files with Strange Names
14.13 Handling a Filename Starting with a Dash (–)
14.14 Using unlink to Remove a File with a Strange Name
Table of Contents
Part IV
14.15 Removing a Strange File by its i-number
14.16 Problems Deleting Directories
14.17 Deleting Stale Files
14.18 Removing Every File but One
14.19 Using find to Clear Out Unneeded Files
15. Optimizing Disk Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15.1 Disk Space Is Cheap
15.2 Instead of Removing a File, Empty It
15.3 Save Space with “Bit Bucket” Log Files and Mailboxes
15.4 Save Space with a Link
15.5 Limiting File Sizes
15.6 Compressing Files to Save Space
15.7 Save Space: tar and compress a Directory Tree
15.8 How Much Disk Space?
15.9 Compressing a Directory Tree: Fine-Tuning
15.10 Save Space in Executable Files with strip
15.11 Disk Quotas
Basic Editing
16. Spell Checking, Word Counting, and Textual Analysis . .
16.1 The Unix spell Command
16.2 Check Spelling Interactively with ispell
16.3 How Do I Spell That Word?
16.4 Inside spell
16.5 Adding Words to ispell’s Dictionary
16.6 Counting Lines, Words, and Characters: wc
16.7 Find a a Doubled Word
16.8 Looking for Closure
16.9 Just the Words, Please
17. vi Tips and Tricks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.1 The vi Editor: Why So Much Material?
17.2 What We Cover
17.3 Editing Multiple Files with vi
17.4 Edits Between Files
17.5 Local Settings for vi
17.6 Using Buffers to Move or Copy Text
Table of Contents
2 72 2 72 2 74 275 276
277
277 277 279 279 280 281 284 286 288 2 89 290
295
295 296 298 299 301 303 305 305 306
308
308 309 309 311 312 313
x
17.7 Get Back What You Deleted with Numbered Buffers 313
17.8 Using Search Patterns and Global Commands 314
17.9 Confirming Substitutions in vi 315
17.10 Keep Your Original File, Write to a New File 316
17.11 Saving Part of a File 316
17.12 Appending to an Existing File 317
17.13 Moving Blocks of Text by Patterns 317
17.14 Useful Global Commands (with Pattern Matches) 318
17.15 Counting Occurrences; Stopping Search Wraps 320
17.16 Capitalizing Every Word on a Line 320
17.17 Per-File Setups in Separate Files 321
17.18 Filtering Text Through a Unix Command 322
17.19 vi File Recovery Versus Networked Filesystems 324
17.20 Be Careful with vi –r Recovered Buffers 325
17.21 Shell Escapes: Running One Unix
Command While Using Another 326
17.22 vi Compound Searches 327
17.23 vi Word Abbreviation 328
17.24 Using vi Abbreviations as Commands
(Cut and Paste Between vi’s) 330
17.25 Fixing Typos with vi Abbreviations 330
17.26 vi Line Commands Versus Character Commands 331
17.27 Out of Temporary Space? Use Another Directory 332
17.28 Neatening Lines
17.29 Finding Your Place with Undo
17.30 Setting Up vi with the .exrc File
333 334 334
. . . . . . . . . . . . . . . . . . . . 336 336
18.2 Save Time and Typing with the vi map Commands 336
18.3 What You Lose When You Use map! 339
18.4 vi @-Functions 340
18.5 Keymaps for Pasting into a Window Running vi 343
18.6 Protecting Keys from Interpretation by ex 343
18.7 Maps for Repeated Edits 345
18.8 More Examples of Mapping Keys in vi 347
18.9 Repeating a vi Keymap 348
18.10 Typing in Uppercase Without CAPS LOCK 348
18.11 Text-Input Mode Cursor Motion with No Arrow Keys 349 Table of Contents xi
18. Creating Custom Commands in vi
18.1 Why Type More Than You Have To?
18.12 Don’t Lose Important Functions with vi Maps: Use noremap
18.13 vi Macro for Splitting Long Lines
18.14 File-Backup Macros
19. GNU Emacs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.1 Emacs: The Other Editor
19.2 Emacs Features: A Laundry List
19.3 Customizations and How to Avoid Them
19.4 Backup and Auto-Save Files
19.5 Putting Emacs in Overwrite Mode
19.6 Command Completion
19.7 Mike’s Favorite Timesavers
19.8 Rational Searches
19.9 Unset PWD Before Using Emacs
19.10 Inserting Binary Characters into Files
19.11 Using Word-Abbreviation Mode
19.12 Directories for Emacs Hacks
19.13 An Absurd Amusement
20. Batch Editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.1 Why Line Editors Aren’t Dinosaurs
20.2 Writing Editing Scripts
20.3 Line Addressing
20.4 Useful ex Commands
20.5 Running Editing Scripts Within vi
20.6 Change Many Files by Editing Just One
20.7 ed/ex Batch Edits: A Typical Example
20.8 Batch Editing Gotcha: Editors Fail on Big Files
20.9 patch: Generalized Updating of Files That Differ
20.10 Quick Reference: awk
20.11 Versions of awk
21. You Can’t Quite Call This Editing . . . . . . . . . . . . . . . . . . . . . .
21.1 And Why Not?
21.2 Neatening Text with fmt
21.3 Alternatives to fmt
21.4 Clean Up Program Comment Blocks
21.5 Remove Mail/News Headers with behead
21.6 Low-Level File Butchery with dd
xii Table of Contents
3 50 3 50 3 51
353
353 354 358 358 360 360 361 362 363 363 364 366 366
367
367 368 369 370 373 373 375 376 377 378 388
390
390 391 392 394 395 396
Part V
21.7 offset: Indent Text 396
21.8 Centering Lines in a File 397
21.9 Splitting Files at Fixed Points: split 398
21.10 Splitting Files by Context: csplit 401
21.11 Hacking on Characters with tr 404
21.12 Encoding “Binary” Files into ASCII 406
21.13 Text Conversion with dd 410
21.14 Cutting Columns or Fields 410
21.15 Making Text in Columns with pr 411
21.16 Make Columns Automatically with column 413
21.17 Straightening Jagged Columns 415
21.18 Pasting Things in Columns 416
21.19 Joining Lines with join 417
21.20 What Is (or Isn’t) Unique? 418
21.21 Rotating Text 419
22. Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
22.1 Putting Things in Order 421
22.2 Sort Fields: How sort Sorts 422
22.3 Changing the sort Field Delimiter 424
22.4 Confusion with Whitespace Field Delimiters 424
22.5 Alphabetic and Numeric Sorting 426
22.6 Miscellaneous sort Hints 427
22.7 lensort: Sort Lines by Length 429
22.8 Sorting a List of People by Last Name 430
Processes and the Kernel
23. Job Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
23.1 Job Control in a Nutshell 435
23.2 Job Control Basics 437
23.3 Using jobs Effectively 438
23.4 Some Gotchas with Job Control 440
23.5 The “Current Job” Isn’t Always What You Expect 442
23.6 Job Control and autowrite: Real Timesavers! 442
23.7 System Overloaded? Try Stopping Some Jobs 443
23.8 Notification When Jobs Change State 444
23.9 Stop Background Output with stty tostop 444
23.10 nohup 445 Table of Contents xiii
23.11 Disowning Processes
23.12 Linux Virtual Consoles
23.13 Stopping Remote Login Sessions
24. Starting, Stopping, and Killing Processes . . . . . . . . . . . . .
24.1 What’s in This Chapter
24.2 fork and exec
24.3 Managing Processes: Overall Concepts
24.4 Subshells
24.5 The ps Command
24.6 The Controlling Terminal
24.7 Tracking Down Processes
24.8 Why ps Prints Some Commands in Parentheses
24.9 The /proc Filesystem
24.10 What Are Signals?
24.11 Killing Foreground Jobs
24.12 Destroying Processes with kill
24.13 Printer Queue Watcher: A Restartable Daemon Shell Script
24.14 Killing All Your Processes
24.15 Killing Processes by Name?
24.16 Kill Processes Interactively
24.17 Processes Out of Control? Just STOP Them
24.18 Cleaning Up an Unkillable Process
24.19 Why You Can’t Kill a Zombie
24.20 The Process Chain to Your Window
24.21 Terminal Windows Without Shells
24.22 Close a Window by Killing Its Process(es)
25. Delayed Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25.1 Building Software Robots the Easy Way
25.2 Periodic Program Execution: The cron Facility
25.3 Adding crontab Entries
25.4 Including Standard Input Within a cron Entry
25.5 The at Command
25.6 Making Your at Jobs Quiet
25.7 Checking and Removing Jobs
25.8 Avoiding Other at and cron Jobs
25.9 Waiting a Little While: sleep
xiv Table of Contents
4 4 6 4 4 7 4 4 9
451
4 51 4 52 4 53 455 456 4 59 4 60 4 62 4 63 468 469 470 471 473 474 476 478 479 480 480 482 484
488
488 4 89 494 495 495 496 496 497 498
Part VI
26.1 Timing Is Everything
26.2 Timing Programs
26.3 What Commands Are Running and How Long Do They Take?
26.4 Checking System Load: uptime
26.5 Know When to Be “nice” to Other Users...and When Not To
26.6 A nice Gotcha
26.7 Changing a Running Job’s Niceness
Scripting
27. Shell Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.1 What the Shell Does
27.2 How the Shell Executes Other Commands
27.3 What’s a Shell, Anyway?
27.4 Command Evaluation and Accidentally Overwriting Files
27.5 Output Command-Line Arguments One by One
27.6 Controlling Shell Command Searches
27.7 Wildcards Inside Aliases
27.8 eval: When You Need Another Chance
27.9 Which One Will bash Use?
27.10 Which One Will the C Shell Use?
27.11 Is It “2 >&1 file” or “> file 2 >&1 ”? Why?
27.12 Bourne Shell Quoting
27.13 Differences Between Bourne and C Shell Quoting
27.14 Quoting Special Characters in Filenames
27.15 Verbose and Echo Settings Show Quoting
27.16 Here Documents
27.17 “Special” Characters and Operators
27.18 How Many Backslashes?
28. Saving Time on the Command Line . . . . . . . . . . . . . . . . . . .
28.1 What’s Special About the Unix Command Line
28.2 Reprinting Your Command Line with CTRL-r
28.3 Use Wildcards to Create Files?
26. System Performance and Profiling
. . . . . . . . . . . . . . . . . . .
500
50 0 50 3 50 4 506 506 51 0 51 0
513
51 3 51 4 515 517 518 518 52 0 52 1 52 3 52 4 526 526 53 1 53 3 53 3 53 4 535 54 0
542
54 2 54 3 54 4 545 547 54 9
xv
28.4 Build Strings with { }
28.5 String Editing (Colon) Operators
28.6 Automatic Completion
Table of Contents
28.7 Don’t Match Useless Files in Filename Completion 553
28.8 Repeating Commands 554
28.9 Repeating and Varying Commands 554
28.10 Repeating a Command with Copy-and-Paste 557
28.11 Repeating a Time-Varying Command 558
28.12 Multiline Commands, Secondary Prompts 559
28.13 Here Document Example #1 : Unformatted Form Letters 560
28.14 Command Substitution 561
28.15 Handling Lots of Text with Temporary Files 563
28.16 Separating Commands with Semicolons 563
28.17 Dealing with Too Many Arguments 565
28.18 Expect 567
29. Custom Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
29.1 Creating Custom Commands 571
29.2 Introduction to Shell Aliases 571
29.3 C-Shell Aliases with Command-Line Arguments 572
29.4 Setting and Unsetting Bourne-Type Aliases 574
29.5 Korn-Shell Aliases 575
29.6 zsh Aliases 576
29.7 Sourceable Scripts 576
29.8 Avoiding C-Shell Alias Loops 578
29.9 How to Put if-then-else in a C-Shell Alias 579
29.10 Fix Quoting in csh Aliases with makealias and quote 580
29.11 Shell Function Basics 581
29.12 Shell Function Specifics 585
29.13 Propagating Shell Functions 586
29.14 Simulated Bourne Shell Functions and Aliases
59 1
30. The Use of History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593
30.1 The Lessons of History 593
30.2 History in a Nutshell 594
30.3 My Favorite Is !$ 595
30.4 My Favorite Is !:n* 595
30.5 My Favorite Is ^^ 596
30.6 Using !$ for Safety with Wildcards 597
30.7 History by Number 597
30.8 History Substitutions 599
30.9 Repeating a Cycle of Commands 604
xvi Table of Contents
30.10 Running a Series of Commands on a File 604
30.11 Check Your History First with :p 605
30.12 Picking Up Where You Left Off 606
30.13 Pass History to Another Shell 608
30.14 Shell Command-Line Editing 609
30.15 Changing History Characters with histchars 615
30.16 Instead of Changing History Characters 616
31. Moving Around in a Hurry . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617
31.1 Getting Around the Filesystem 617
31.2 Using Relative and Absolute Pathnames 618
31.3 What Good Is a Current Directory? 620
31.4 How Does Unix Find Your Current Directory? 621
31.5 Saving Time When You Change Directories: cdpath 622
31.6 Loop Control: break and continue 623
31.7 The Shells’ pushd and popd Commands 624
31.8 Nice Aliases for pushd 626
31.9 Quick cds with Aliases 627
31.10 cd by Directory Initials 627
31.11 Finding (Anyone’s) Home Directory, Quickly 629
31.12 Marking Your Place with a Shell Variable 630
31.13 Automatic Setup When You Enter/Exit a Directory 630
32. Regular Expressions (Pattern Matching) . . . . . . . . . . . . . . 633
32.1 That’s an Expression
633
32.2 Don’t Confuse Regular Expressions with Wildcards 634
32.3 Understanding Expressions 635
32.4 Using Metacharacters in Regular Expressions 637
32.5 Regular Expressions: The Anchor Characters ^ and $ 638
32.6 Regular Expressions: Matching a Character
with a Character Set 639
32.7 Regular Expressions: Match Any Character with . (Dot) 640
32.8 Regular Expressions: Specifying
a Range of Characters with [...] 640
32.9 Regular Expressions: Exceptions in a Character Set 641
32.10 Regular Expressions: Repeating Character Sets with * 641
32.11 Regular Expressions: Matching a Specific
Number of Sets with \\ { and \\ } 642
32.12 Regular Expressions: Matching Words with \\ < and \\ > 643
Table of Contents xvii
32.13 Regular Expressions: Remembering Patterns with \\ (, \\ ), and \\1
32.14 Regular Expressions: Potential Problems
32.15 Extended Regular Expressions
32.16 Getting Regular Expressions Right
32.17 Just What Does a Regular Expression Match?
32.18 Limiting the Extent of a Match
32.19 I Never Meta Character I Didn’t Like
32.20 Valid Metacharacters for Different Unix Programs
32.21 Pattern Matching Quick Reference with Examples
33. Wildcards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33.1 File-Naming Wildcards
33.2 Filename Wildcards in a Nutshell
33.3 Who Handles Wildcards?
33.4 What if a Wildcard Doesn’t Match?
33.5 Maybe You Shouldn’t Use Wildcards in Pathnames
33.6 Getting a List of Matching Files with grep –l
33.7 Getting a List of Nonmatching Files
33.8 nom: List Files That Don’t Match a Wildcard
34. The sed Stream Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34.1 sed Sermon^H^H^H^H^H^HSummary
34.2 Two Things You Must Know About sed
34.3 Invoking sed
34.4 Testing and Using a sed Script: checksed, runsed
34.5 sed Addressing Basics
34.6 Order of Commands in a Script
34.7 One Thing at a Time
34.8 Delimiting a Regular Expression
34.9 Newlines in a sed Replacement
34.10 Referencing the Search String in a Replacement
34.11 Referencing Portions of a Search String
34.12 Search and Replacement: One Match Among Many
34.13 Transformations on Text
34.14 Hold Space: The Set-Aside Buffer
34.15 Transforming Part of a Line
34.16 Making Edits Across Line Boundaries
34.17 The Deliberate Scrivener
34.18 Searching for Patterns Split Across Lines
Table of Contents
64 4 64 4 645 646 648 64 9 650 651 652
657
657 658 660 662 663 664 664 666
668
668 669 669 670 672 674 675 675 676 677 678 679 680 680 683 685 688 69 0
xviii
34.19 Multiline Delete
34.20 Making Edits Everywhere Except...
34.21 The sed Test Command
34.22 Uses of the sed Quit Command
34.23 Dangers of the sed Quit Command
34.24 sed Newlines, Quoting, and Backslashes in a Shell Script
35. Shell Programming for the Uninitiated . . . . . . . . . . . . . . . .
35.1 Writing a Simple Shell Program
35.2 Everyone Should Learn Some Shell Programming
35.3 What Environment Variables Are Good For
35.4 Parent-Child Relationships
35.5 Predefined Environment Variables
35.6 The PATH Environment Variable
35.7 PATH and path
35.8 The DISPLAY Environment Variable
35.9 Shell Variables
35.10 Test String Values with Bourne-Shell case
35.11 Pattern Matching in case Statements
35.12 Exit Status of Unix Processes
35.13 Test Exit Status with the if Statement
35.14 Testing Your Success
35.15 Loops That Test Exit Status
35.16 Set Exit Status of a Shell (Script)
35.17 Trapping Exits Caused by Interrupts
35.18 read: Reading from the Keyboard
35.19 Shell Script “Wrappers” for awk, sed, etc.
35.20 Handling Command-Line Arguments in Shell Scripts
35.21 Handling Command-Line Arguments with a for Loop
35.22 Handling Arguments with while and shift
35.23 Loop Control: break and continue
35.24 Standard Command-Line Parsing
35.25 The Bourne Shell set Command
35.26 test: Testing Files and Strings
35.27 Picking a Name for a New Command
35.28 Finding a Program Name and Giving Your Program Multiple Names
35.29 Reading Files with the . and source Commands
35.30 Using Shell Functions in Shell Scripts
Table of Contents
69 2 69 3 695 696 696 697
698
698 70 0 70 2 705 705 708 709 710 711 713 714 715 716 718 719 720 721 723 724 725 727 728 73 0 73 0 73 2 735 736
736 737 738
xix
36. Shell Programming for the Initiated
36.1 Beyond the Basics
36.2 The Story of : # #!
. . . . . . . . . . . . . . . . . .
741
74 1 74 2 74 3 744 745 746 747
748 749 749 750 750 751 753 753 756 759 760 761 761 762 764 764 768 770 770 772
775
775 777 777 778 779 781 782 783 784
36.3 Don’t Need a Shell for Your Script? Don’t Use One
36.4 Making #! Search the PATH
36.5 The exec Command
36.6 The Unappreciated Bourne Shell “:” Operator
36.7 Parameter Substitution
36.8 Save Disk Space and Programming: Multiple Names for a Program
36.9 Finding the Last Command-Line Argument
36.10 How to Unset All Command-Line Parameters
36.11 Standard Input to a for Loop
36.12 Making a for Loop with Multiple Variables
36.13 Using basename and dirname
36.14 A while Loop with Several Loop Control Commands
36.15 Overview: Open Files and File Descriptors
36.16 n>&m: Swap Standard Output and Standard Error
36.17 A Shell Can Read a Script from Its Standard Input, but...
36.18 Shell Scripts On-the-Fly from Standard Input
36.19 Quoted hereis Document Terminators: sh Versus csh
36.20 Turn Off echo for “Secret” Answers
36.21 Quick Reference: expr
36.22 Testing Characters in a String with expr
36.23 Grabbing Parts of a String
36.24 Nested Command Substitution
36.25 Testing Two Strings with One case Statement
36.26 Outputting Text to an X Window
36.27 Shell Lockfile
37. Shell Script Debugging and Gotchas . . . . . . . . . . . . . . . . . .
37.1 Tips for Debugging Shell Scripts
37.2 Bourne Shell Debugger Shows a Shell Variable
37.3 Stop Syntax Errors in Numeric Tests
37.4 Stop Syntax Errors in String Tests
37.5 Quoting and Command-Line Parameters
37.6 How Unix Keeps Time
37.7 Copy What You Do with script
37.8 Cleaning script Files
37.9 Making an Arbitrary-Size File for Testing
xx Table of Contents
Part VII
Extending and Managing Your Environment
38. Backing Up Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
38.1 What Is This “Backup” Thing?
38.2 tar in a Nutshell
38.3 Make Your Own Backups
38.4 More Ways to Back Up
38.5 How to Make Backups to a Local Device
38.6 Restoring Files from Tape with tar
38.7 Using tar to a Remote Tape Drive
38.8 Using GNU tar with a Remote Tape Drive
38.9 On-Demand Incremental Backups of a Project
38.10 Using Wildcards with tar
38.11 Avoid Absolute Paths with tar
38.12 Getting tar’s Arguments in the Right Order
38.13 The cpio Tape Archiver
38.14 Industrial Strength Backups
39. Creating and Reading Archives . . . . . . . . . . . . . . . . . . . . . . .
39.1 Packing Up and Moving
39.2 Using tar to Create and Unpack Archives
39.3 GNU tar Sampler
39.4 Managing and Sharing Files with RCS and CVS
39.5 RCS Basics
39.6 List RCS Revision Numbers with rcsrevs
39.7 CVS Basics
39.8 More CVS
40. Software Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40.1 /usr/bin and Other Software Directories
40.2 The Challenges of Software Installation on Unix
40.3 Which make?
40.4 Simplifying the make Process
40.5 Using Debian’s dselect
40.6 Installing Software with Debian’s Apt-Get
40.7 Interruptable gets with wget
40.8 The curl Application and One-Step GNU-Darwin Auto-Installer for OS X
40.9 Installation with FreeBSD Ports
787
787 788 788 790 790 795 797 798 798 800 803 804 805 806
808
808 809 813 814 815 817 818 819
822
82 2 82 4 824 824 825 832 834
836 837
xxi
Table of Contents
40.10 Installing with FreeBSD Packages
40.11 Finding and Installing RPM Packaged Software
41. Perl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
41.1 High-Octane Shell Scripting
41.2 Checking your Perl Installation
41.3 Compiling Perl from Scratch
41.4 Perl Boot Camp, Part 1 : Typical Script Anatomy
41.5 Perl Boot Camp, Part 2 : Variables and Data Types
41.6 Perl Boot Camp, Part 3 : Branching and Looping
41.7 Perl Boot Camp, Part 4 : Pattern Matching
41.8 Perl Boot Camp, Part 5: Perl Knows Unix
41.9 Perl Boot Camp, Part 6: Modules
41.10 Perl Boot Camp, Part 7: perldoc
41.11 CPAN
41.12 Make Custom grep Commands (etc.) with Perl
41.13 Perl and the Internet
42. Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
42.1 What Is Python?
42.2 Installation and Distutils
42.3 Python Basics
42.4 Python and the Web
42.5 urllib
42.6 urllib2
42.7 htmllib and HTMLParser
42.8 cgi
42.9 mod_python
42.10 What About Perl?
PartVIII Communication and Connectivity
43. Redirecting Input and Output . . . . . . . . . . . . . . . . . . . . . . . . .
43.1 Using Standard Input and Output
43.2 One Argument with a cat Isn’t Enough
43.3 Send (Only) Standard Error Down a Pipe
43.4 Problems Piping to a Pager
43.5 Redirection in C Shell: Capture Errors, Too?
43.6 Safe I/O Redirection with noclobber
xxii Table of Contents
83 7 83 8
839
83 9 83 9 84 1 84 2 845 852 854 856 859 861 862 865 866
869
869 869 871 876 876 877 878 878 879 880
885
885 888 888 889 891 892
43.7 The () Subshell Operators
43.8 Send Output Two or More Places
43.9 How to tee Several Commands into One Place
43.10 Redirecting Output to More Than One Place
43.11 Named Pipes: FIFOs
43.12 What Can You Do with an Empty File?
44. Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
44.1 Quick Introduction to Hardware
44.2 Reading Kernel Boot Output
44.3 Basic Kernel Configuration
44.4 Disk Partitioning
44.5 Filesystem Types and /etc/fstab
44.6 Mounting and Unmounting Removable Filesystems
44.7 Loopback Mounts
44.8 Network Devices—ifconfig
44.9 Mounting Network Filesystems—NFS, SMBFS
44.10 Win Is a Modem Not a Modem?
44.11 Setting Up a Dialup PPP Session
44.12 USB Configuration
44.13 Dealing with Sound Cards and Other Annoying Hardware
44.14 Decapitating Your Machine—Serial Consoles
45. Printing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45.1 Introduction to Printing
45.2 Introduction to Printing on Unix
45.3 Printer Control with lpc
45.4 Using Different Printers
45.5 Using Symbolic Links for Spooling
45.6 Formatting Plain Text: pr
45.7 Formatting Plain Text: enscript
45.8 Printing Over a Network
45.9 Printing Over Samba
45.10 Introduction to Typesetting
45.11 A Bit of Unix Typesetting History
45.12 Typesetting Manpages: nroff
45.13 Formatting Markup Languages— troff, LATEX, HTML, and So On
45.14 Printing Languages—PostScript, PCL, DVI, PDF
Table of Contents
89 3 89 4 895 895 897 898
900
900 900 902 903 904 906 907 908 908 909 910 911 911 912
914
914 915 917 918 919 920 922 923 923 925 926 927
928 929
xxiii
Part IX
xxiv
45.15 Converting Text Files into a Printing Language
45.16 Converting Typeset Files into a Printing Language
45.17 Converting Source Files Automagically Within the Spooler
45.18 The Common Unix Printing System (CUPS)
45.19 The Portable Bitmap Package
46. Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
46.1 TCP/IP—IP Addresses and Ports
46.2 /etc/services Is Your Friend
46.3 Status and Troubleshooting
46.4 Where, Oh Where Did That Packet Go?
46.5 The Director of Operations: inetd
46.6 Secure Shell (SSH)
46.7 Configuring an Anonymous FTP Server
46.8 Mail—SMTP, POP, and IMAP
46.9 Domain Name Service (DNS)
46.10 Dynamic Host Configuration Protocol (DHCP)
46.11 Gateways and NAT
46.12 Firewalls
46.13 Gatewaying from a Personal LAN over a Modem
47. Connecting to MS Windows . . . . . . . . . . . . . . . . . . . . . . . . . .
47.1 Building Bridges
47.2 Installing and Configuring Samba
47.3 Securing Samba
47.4 SWAT and GUI SMB Browsers
47.5 Printing with Samba
47.6 Connecting to SMB Shares from Unix
47.7 Sharing Desktops with VNC
47.8 Of Emulators and APIs
47.9 Citrix: Making Windows Multiuser
Security
48. Security Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
48.1 Understanding Points of Vulnerability
48.2 CERT Security Checklists
48.3 Keeping Up with Security Alerts
48.4 What We Mean by Buffer Overflow
Table of Contents
930 931 932 933 933
936
936 938 939 941 942 943 944 944 945 947 948 949 9 50
951
951 951 955 956 958 9 59 9 59 9 62 9 63
969
9 69 9 70 9 71 9 72
48.5 What We Mean by DoS
48.6 Beware of Sluggish Performance
48.7 Intruder Detection
48.8 Importance of MOTD
48.9 The Linux proc Filesystem
48.10 Disabling inetd
48.11 Disallow rlogin and rsh
48.12 TCP Wrappers
49. Root, Group, and User Management
. . . . . . . . . . . . . . . . . .
9 73 9 74 977 978 9 79 9 79 9 80 9 80
982
9 82 9 82 9 84 985 985 986 988 989 990 990 991 992 993
994
994 994 998 999
1 000 1 001 1 002 1 003 1 003 1 004 1 006 1 007 1 009 1 009 1 010
xxv
49.1 Unix User/Group Infrastructure
49.2 When Does a User Become a User
49.3 Forgetting the root Password
49.4 Setting an Exact umask
49.5 Group Permissions in a Directory with the setgid Bit
49.6 Groups and Group Ownership
49.7 Add Users to a Group to Deny Permissions
49.8 Care and Feeding of SUID and SGID Scripts
49.9 Substitute Identity with su
49.10 Never Log In as root
49.11 Providing Superpowers with sudo
49.12 Enabling Root in Darwin
49.13 Disable logins
50. File Security, Ownership, and Sharing
. . . . . . . . . . . . . . . .
50.1 Introduction to File Ownership and Security
50.2 Tutorial on File and Directory Permissions
50.3 Who Will Own a New File?
50.4 Protecting Files with the Sticky Bit
50.5 Using chmod to Change File Permission
50.6 The Handy chmod = Operator
50.7 Protect Important Files: Make Them Unwritable
50.8 cx, cw, c–w: Quick File Permission Changes
50.9 A Loophole: Modifying Files Without Write Access
50.10 A Directory That People Can Access but Can’t List
50.11 Juggling Permissions
50.12 File Verification with md5sum
50.13 Shell Scripts Must Be Readable and (Usually) Executable
50.14 Why Can’t You Change File Ownership?
50.15 How to Change File Ownership Without chown
Table of Contents
51.1 51.2 51.3 51.4 51.5 51.6 51.7
Glossary . Index . . . .
Enabling Remote Access on Mac OS X 1011 Protecting Access Through SSH 1011 Free SSH with OpenSSH 1012 SSH Problems and Solutions 1013 General and Authentication Problems 1013 Key and Agent Problems 1019 Server and Client Problems 1021
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1030 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1037
51. SSH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1011
xxvi Table of Contents
';
$subst = "${command}";
$result = preg_replace($re, $subst, $str);
echo "The result of the substitution is ".$result;
Please keep in mind that these code samples are automatically generated and are not guaranteed to work. If you find any syntax errors, feel free to submit a bug report. For a full regex reference for PHP, please visit: http://php.net/manual/en/ref.pcre.php