AWK Essential Training

Course details

So you've got a large data set. New information is always coming in, and it's messy and inconsistent. Text files, Excel spreadsheets, web forms, email messages, and so on. You'd like to find a way to format it into something clean and consistent you can use in a database, and you'd like to automate that process as much as possible. Enter AWK, a command-line tool for Mac, Unix, and Linux that allows you to manipulate data in powerful ways.

In this course, award-winning author and teacher David D. Levine shows you how to use AWK to read and write data in a variety of formats, produce reports, and automate repetitive tasks. He reviews the nuts and bolts of the language, such as field separators, pattern matching, variables, operators, expressions, and control structures; functions available for manipulating data; and integration with other programs like Excel.

Learning objectives
Determine what AWK is.
Recognize how to write an AWK program.
Determine how to use AWK command-line flags.
Identify how to specify field and record separators with variables.
Distinguish how to change a CVS file to a tab-separated one.
Break down how to work with operators and arrays.
Discover how to format output.
Interpret how to string data with functions.

Concepts

Introduction

Welcome
What you should know before watching this course
Using the exercise files

What Is AWK

What is awk

Awk Command Line Basics

Writing an AWK program
Working with records, fields, patterns, and actions
Using AWK command line flags

Understanding Records and Fields

Exploring basic input field separators
Specifying field and record separators with variables
Challenge - Change a CSV file to a tab-separated one
Solution - Change a CSV file to a tab-separated one

Understanding Variables and Operators

Using built-in variables
Creating user-defined variables
Working with operators and arrays

A Quick Introduction to Regular Expressions

Regular expression basics
Working with character classes and quantifiers

Using Control Structures

Building control structures
Creating an HTML table
Challenge - Print only those lines consisting of a complete HTML entity
Solution - Print only those lines consisting of a complete HTML entity

Formatting the Output

Formatting output with printf()
Formatting output with width and precision specifiers

Functions and Arrays

Manipulating strings
Using associative arrays
Introducing awk's math functions

Combining AWK with Other Tools

Using pipes
Parsing Excel CSV files - line endings and quoting
Parsing Excel CSV files - commas and newlines
Scripting with AWK
Challenge - Perform a join
Solution - Perform a join

Conclusion

Conclusion