Second phase of generative AI

Welcome to Daily Zaps — Here’s what we got for you today: 🙌

Second phase of generative AI

The second phase of generative AI, termed "Act 2," is characterized by a shift from technology-centric demonstrations to customer-centric solutions that address human problems comprehensively. Companies are now focusing on making generative AI products more useful and engaging. A shared playbook is emerging to close the gap between user expectations and model capabilities.

Reasoning Techniques: Innovations like chain-of-thought, tree-of-thought, and reflexion are enhancing models' ability for complex reasoning tasks, bridging the gap between user expectations and model capabilities.

Transfer Learning: Techniques like RLHF and fine-tuning are becoming more accessible. Recent fine-tuning support for GPT-3.5 and Llama-2 allows companies to adapt foundation models to specific domains, leveraging user feedback.

Retrieval-Augmented Generation (RAG): RAG incorporates contextual information, reducing errors and improving truthfulness. Vector databases, such as those from Pinecone, serve as infrastructure support for RAG.

Developer Tools: New tools and frameworks empower companies to build advanced AI applications. They aid in evaluation, improvement, and monitoring of AI models in production.

AI Infrastructure: Companies are providing abundant, on-demand, and scalable GPUs, offering a developer-friendly platform-as-a-service (PaaS) experience, reducing reliance on public clouds.

Is GPU capacity getting overbuilt?

The demand for GPUs and AI model training has surged, driven by Nvidia's Q2 earnings and the success of AI models like ChatGPT. However, questions remain about what all these GPUs are being used for and how they will generate value. For every $1 spent on a GPU, roughly $1 must be spent on energy costs to run it in a data center, making the total cost substantial.

While tech giants like Google, Microsoft, and Meta are investing in data center build-out, it's unclear how much of this is driven by actual end-customer demand versus anticipation. Startups have an opportunity to fill the gap by leveraging AI technology to create genuine end-customer value. This is similar to how YouTube was a success because of low cost Internet bandwidth due to over investment in fiber in the early 2000s.

The Generative AI Market Map and Infrastructure Stack

The map is organized by use case rather than by model modality. This reflects two important thrusts in the market: Generative AI’s evolution from technology hammer to actual use cases and value, and the increasingly multimodal nature of generative AI applications.

The LLM developer stack reflects the compute and tooling vendors that companies are turning to as they build generative AI applications in production.

Microsoft Windows 11 Adds AI Powered Copilot

Microsoft is introducing Windows Copilot, an AI assistant, to Windows 11. This feature will be integrated into the operating system, accessible from the taskbar across all apps and programs. Windows Copilot offers various functions, such as summarizing content, rewriting it, or providing explanations. It can be used for tasks beyond basic search, allowing users to customize settings and take actions on their PC.

Developers can extend plug-ins written for Bing or OpenAI’s ChatGPT to Windows Copilot, ensuring compatibility and future improvements. Microsoft plans to begin public testing of Windows Copilot in June, with a wider rollout to Windows 11 users this fall.