Scalable Data Storage and Processing for AI Workloads

Scalable Data Storage and Processing for AI Workloads

1h 30mIntermediate2025-03-17

Authors

Janani Ravi

Janani Ravi

Certified Google Cloud Architect and Data Engineer

Course details

Solutions for data storage and processing are essential, but how do you manage them effectively at scale? In this course, instructor Janani Ravi covers the fundamentals of designing and implementing data storage systems that can efficiently handle the large-scale demands of AI-powered applications. Explore techniques for managing, processing, and optimizing data flow in distributed environments to ensure high-performance AI model execution. An ideal fit for tech professionals working with AI, data infrastructure, and machine learning operations, this course equips you with the skills you need to not only manage but also optimize your AI workloads.

Skills covered

Cloud StorageArtificial Intelligence FoundationsData EngineeringArtificial Intelligence (AI)Cloud ComputingData ScienceOne-Off

Concepts

Introduction

  • Scalable solutions for storage and processing
  • Prerequisites

Types of Data and Storage Requirements

  • Types of data - Structured, semistructured, and unstructured
  • Understanding structured data
  • Understanding semistructured data
  • Understanding unstructured data

Data Storage in the AI Pipeline

  • Storage requirements in the AI pipeline
  • Data storage in the AI workflow
  • AI storage considerations
  • AI storage best practices
  • Cloud storage on Google Cloud
  • Object storage with Amazon S3
  • Blob storage on Azure

Vector Databases and RAGs

  • Retrieval-augmented generation
  • Vector databases and embeddings
  • Semantic search with Pinecone

AI Workloads and Processing

  • Types of AI workloads
  • Best practices to optimize AI workloads
  • AI workloads on the cloud

Conclusion

  • Summary and next steps
40,000 Toman