Build an Image Captioning Tool for Visually Impaired Users with Gemini

Build an Image Captioning Tool for Visually Impaired Users with Gemini

1h 8mIntermediate2024-05-29

Authors

Fikayo Adepoju

Fikayo Adepoju

Technical Writer | Software Developer

Course details

When you’re designing for the web, it’s imperative that your designs are accessible to as many people as possible, and this includes visually impaired users. While there are many tools available to help you in this design task, artificial intelligence is making this easier than ever. In this course, software developer Fikayo Adepoju shows you how to use Google’s Gemini API to build an image caption generator that helps visually impaired users to better experience the web. Get hands-on experience with the Gemini API as Fikayo guides you through the building of the application from start to finish. Learn how to build the interface, connect the backend to Gemini, and then connect the frontend to the API. Whether you’re a developer, data scientist, or just have an interest in AI, join Fikayo to see how the Gemini API can make life easier for both you and your users.

Skills covered

GeminiAccessibilityNeural Networks and Deep LearningAI Productivity ToolsDiversity, Equity, and Inclusion (DEI)Generative AIUser ExperienceGoogleArtificial Intelligence (AI)Business Software and ToolsOne-Off

Concepts

Introduction

  • Image captioning with AI
  • What you should know
  • Who this course is for

Setting Up Access to Gemini API

  • Understanding Gemini models
  • Gemini pricing
  • Signing up for an Google AI Studio account
  • Getting your API key

Building the Interface

  • Cloning the seed project
  • Project code walkthrough
  • Adding the image upload functionality
  • Adding the prompt functionality
  • Writing the caption display

Building the Backend - Connecting to Gemini

  • Building out the Express.js API
  • Configuring the Generative AI SDK
  • Adding routes
  • Setting up file upload functionality
  • Writing the prompt request and response

Bringing It All Together

  • Connecting the frontend to the API
  • Adding a progress indicator
  • Using the Web Speech API to read captions

Conclusion

  • Next steps
40,000 Toman