Developing with Nondeterministic APIs
27mGeneral2024-02-14
Authors
Morten Rand-Hendriksen
Senior Staff Instructor, Speaker, Web Designer, and Software Developer
Course details
AI APIs produce nondeterministic output meaning the same prompt may produce different results every time it’s run. This poses unique challenges to developers because traditional software development is based on the premise that systems produce consistent and therefore deterministic outputs. In this course you’ll learn strategies for how to mitigate these unique challenges to meld traditional software development with modern AI tools.
Learning objectives
Understand the difference between deterministic and nondeterministic APIs
Identify unique challenges with using nondeterministic APIs
Learn basic mitigation practices for handling nondeterministic API responses
Learning objectives
Understand the difference between deterministic and nondeterministic APIs
Identify unique challenges with using nondeterministic APIs
Learn basic mitigation practices for handling nondeterministic API responses
Skills covered
APIsSoftware DevelopmentOne-Off
Concepts
0. Introduction
- 01 - Developing with nondeterministic APIs
1. Prompting for Consistency
- 02 - What is nondeterminism in AI APIs
- 03 - Prompting for consistency
- 04 - Giving the AI options to choose from
- 05 - Providing reference materials
2. Developing for Consistency
- 06 - Getting consistency in probablilistic systems
- 07 - Bringing the temperature down
- 08 - Bringing top-p down
- 09 - Setting a seed value
- 10 - Activating JSON output
- 11 - Using an advanced response template
- 12 - Using function calling
3. Strategies for Consistency at Scale
- 13 - Finding consistency at scale
- 14 - Creating a fine-tuned model
- 15 - Using a grounded fact source
- 16 - Using semantic cache for optimal performance
Conclusion
- 17 - Using AI only when necessary