Command Line Data Analysis
58mAdvanced2023-04-27
Authors
Miki Tebeka
CEO at 353Solutions
Course details
This course is integrated with GitHub Codespaces, an instant cloud developer environment that offers all the functionality of your favorite IDE without the need for any local machine setup. With GitHub Codespaces, you can get hands-on practice from any machine, at any time—all while using a tool that you’ll likely encounter in the workplace. Check out the “Using GitHub Codespaces with this course” video to learn how to get started.
In this course, Miki Tebeka presents the information and instructions you need to perform data analysis using the many tools that the command line offers for quick data analysis and visualization. Learn how to use the command line and command line tools for analyzing and displaying data. Go over Unix shell basics, and then use tools like grep, jq, sed, awk, curl, gnuplot, and others to analyze and display data from various sources, such as CSV files, APIs, databases, and more.
In this course, Miki Tebeka presents the information and instructions you need to perform data analysis using the many tools that the command line offers for quick data analysis and visualization. Learn how to use the command line and command line tools for analyzing and displaying data. Go over Unix shell basics, and then use tools like grep, jq, sed, awk, curl, gnuplot, and others to analyze and display data from various sources, such as CSV files, APIs, databases, and more.
Skills covered
UnixHelp Desk SkillsWeb Development ToolsIT Help DeskData AnalysisWeb DevelopmentData ScienceBusiness Analysis and StrategyBusiness Software and ToolsOpen SourceDeep Dive (X:Y)
Concepts
0. Introduction
- 01 - Command-line data analysis
- 02 - What you should know
- 03 - Setting up
1. The Command Line
- 04 - The shell
- 05 - Running programs
- 06 - Shell expansion
- 07 - Pipes and redirects
- 08 - Files and directories
- 09 - Installing commands
2. Data Ingestion
- 10 - Cat and less
- 11 - curl
- 12 - SCP
- 13 - gsutil and AWS CLI
- 14 - Databases
- 15 - Challenge - ETL
- 16 - Solution - ETL
3. Selecting Data
- 17 - head, tail, and shuf
- 18 - grep and regular expressions
- 19 - cut
- 20 - jq
- 21 - Databases
- 22 - Challenge - Station status
- 23 - Solution - Station status
4. Data Transformations
- 24 - tr
- 25 - sed
- 26 - AWK
- 27 - Challenge - Normalized items
- 28 - Solution - Normalized items
5. Calculations
- 29 - sort and uniq
- 30 - bc
- 31 - AWK
- 32 - Challenge - Quantity sold
- 33 - Solution - Quantity sold
6. Visualization
- 34 - printf
- 35 - gnuplot - Introduction
- 36 - gnuplot - Box plots
- 37 - gnuplot - Bar charts
- 38 - Challenge - Weather
- 39 - Solution - Weather
Conclusion
- 40 - Next steps