Scale's AI Readiness Report

Summary: The Zeitgeist report

Hey Everyone,

Scale AI has released its AI readiness report. Scale surveyed over 1,800 ML practitioners and leaders directly involved in building or applying AI solutions. See last year’s edition here.

Some of these Enterprise grade reports do have takeaways that are somewhat useful and interesting. This is also a visual post, for those of us visual learners among the readers.

They call this the “Zeitgeist AI Readiness Report 2024”, in short, they want to raise more funds. Last time I checked, Scale AI is seeking a $13 Billion valuation round. Scale AI has raised a total of $602.6 million over seven funding rounds so far. It’s not clear to me how big their next round will be, but it could be considerable.

Which AI models do enterprises work with?

Keep in mind that this is somewhat backwards looking:

• GPT-4

• Gemini

• DallE-2

• Llama2

Top Use Cases of AI

Copywriting was the 1st prominent use case of GenAI, and continues to attract many users. Speaking of which, this is from 2023 but still a good overview (not from the Scale AI report, but had some fascinating tools.

According to Unwind AI:

🌟 Key Highlights:

1. Model Usage: Closed-source model usage has significantly increased, with 86% of organizations utilizing them compared to 37% in 2023. Open-source model adoption also saw a rise, from 41% to 66%.

2. Model Preferences: OpenAI’s GPT-4 leads in popularity (58%), followed by GPT-3.5 (44%) and Google Gemini (39%). Open-source models like Falcon, Mixtral, and DBRX also gain traction due to their efficiency and flexibility.

3. AI Adoption Stages: 22% of organizations have one model in production, 27% have multiple models deployed, while 49% are still in the early stages of evaluating use cases or developing their first model.

4. Model Customization: While 65% of organizations currently use models out-of-the-box, many are exploring customization through fine-tuning (43%) and retrieval-augmented generation (RAG) (38%) for improved performance.

5. Domain-Specific Data & Human Expertise: The next leap in AI capabilities will require domain-specific, human-generated datasets to capture nuances and edge cases.

6. Investment Trends: In 3 years, organizations plan to increase investments in both commercial closed and open-source models.

7. Barriers to Adoption: Infrastructure limitations (61%), budget constraints (54%), and data privacy concerns (52%) remain key challenges hindering wider AI adoption.

8. New Model Architectures: MoE models like Mixtral and DBRX are gaining prominence for their high performance with fewer parameters and less compute demands.

💼 Business Opportunities:

1. Managed Data Labeling Services: With growing emphasis on high-quality, domain-specific data, there’s a rising need for specialized data labeling services to cater to the unique requirements of different industries and AI applications.

2. Computational Resources: The industry is shifting towards GPUs and TPUs for AI workloads, demanding new programming models, tooling, and optimization techniques.

3. AI Development Tools: The demand for AI development tools like GitHub continues to grow. New businesses can focus on niche areas within software development, catering to specific programming languages or domains.

4. Customizing AI Models: Businesses providing tools for fine-tuning, RAG, and prompt engineering will be in high demand as organizations seek to customize and optimize AI models for specific use cases.

Open vs. Closed Source