Build an Image Captioning Tool for Visually Impaired Users with Gemini
1h 8mIntermediate2024-05-29
Authors

Fikayo Adepoju
Technical Writer | Software Developer
Course details
When you’re designing for the web, it’s imperative that your designs are accessible to as many people as possible, and this includes visually impaired users. While there are many tools available to help you in this design task, artificial intelligence is making this easier than ever. In this course, software developer Fikayo Adepoju shows you how to use Google’s Gemini API to build an image caption generator that helps visually impaired users to better experience the web. Get hands-on experience with the Gemini API as Fikayo guides you through the building of the application from start to finish. Learn how to build the interface, connect the backend to Gemini, and then connect the frontend to the API. Whether you’re a developer, data scientist, or just have an interest in AI, join Fikayo to see how the Gemini API can make life easier for both you and your users.
Skills covered
GeminiAccessibilityNeural Networks and Deep LearningAI Productivity ToolsDiversity, Equity, and Inclusion (DEI)Generative AIUser ExperienceGoogleArtificial Intelligence (AI)Business Software and ToolsOne-Off
Concepts
Introduction
- Image captioning with AI
- What you should know
- Who this course is for
Setting Up Access to Gemini API
- Understanding Gemini models
- Gemini pricing
- Signing up for an Google AI Studio account
- Getting your API key
Building the Interface
- Cloning the seed project
- Project code walkthrough
- Adding the image upload functionality
- Adding the prompt functionality
- Writing the caption display
Building the Backend - Connecting to Gemini
- Building out the Express.js API
- Configuring the Generative AI SDK
- Adding routes
- Setting up file upload functionality
- Writing the prompt request and response
Bringing It All Together
- Connecting the frontend to the API
- Adding a progress indicator
- Using the Web Speech API to read captions
Conclusion
- Next steps