What is Baichuan Intelligence?

China's lack of well-funded LLM startups is a bit shocking.

Hello Everyone,

I’ve been trying to track the Generative A.I. startups in China, and there’s been not as much as I was hoping to talk about.

Given state censorship, Chinese A.I. regulation and the bounce back from the Tech crowndown of 2020 to 2023 hasn’t been the best, outside of the Beijing Academy of A.I. and China’s BigTech, it doesn’t look great so far in 2023.

On July 13th, 2023, China’s top cyberspace regulator unveiled a set of provisional rules to govern generative AI services, including API providers, that serve China-based users.

The Meituan executive led A.I. startup was acquired by Meituan after just a few months, not a great sign. Not good! So long Lightyear AI!

Which leaves us with Baichuan Intelligence and Kai-Fung Lu’s startup and just a few others we know about. We can only hope China is playing a good game of developing startups in stealth. Although that seems somewhat unlikely from what we know about the current state of the economy. So Baichuan remains one of the better stories in a sea of A.I. uncertainty in China.

Elon Musk says he told senior leaders in China that the creation of an AI-led "digital superintelligence" could usurp the Chinese Communist Party and take control of the country.  He likely made a deal with them to fund x.AI in my humble opinion. Now that’s a salesman, but a dangerous one too. Beijing certainly has the budget to power BAAI and perhaps an outsider like x.AI. So what of Baichuan anyways?

With Baichuan, Tsinghua graduates and search company Sogou executives Wang Xiaochuan and Ru Liyun founded “Baichuan Intelligence”. The startup launched Baichuan-7B, an open-source, pre-trained model available in Chinese and English. Wang has worked on natural language processing services as early as 2019, when Sogou integrated an AI-powered word prediction function into its widely used Pinyin input software.

It took China nearly a year later to have its own Stability A.I. We indeed have to assume outside of BAAI they are a few months behind the west, and significantly behind when it comes to access to funding.

A lot of the developments seem to be led by former Tech all-starts of China. Baichuan is being touted as one of China’s most promising LLM developers, thanks to its founder’s storied past as a computer science prodigy from Tsinghua University and founding the search engine provider Sogou, which was later acquired by Tencent.

Where are the Chinese LLM Startups? CNBC covered this topic recently

Baichuan-13B

Like other homegrown LLMs of China, Baichuan, a 13 billion-parameter model based on the Transformer architecture (which also undergirds GPT), is trained on Chinese and English data. (Parameters refer to variables that the model uses to generate and analyze text.) The model is open source and optimized for commercial application, according to its GitHub page.

• https://huggingface.co/baichuan-inc/Baichuan-13B-Base

Baichuan had a $50 million round on April 11th, 2023.

Baidu claiming its Ernie bot beats ChatGPT in some domains isn’t impressing anyone.

If it’s hard to find legit China based Generative A.I. startups, it’s a problem. So who is left?

Zhipu AI

Product: ChatBLM, a large language model 
Founder: Tang Jie 
Funding: Qiming Venture Partners, Legend Capital, Tsinghua Holdings 
Location: Sohu Internet Plaza 

Zhipu AI also draws from Tsinghua University’s AI talent pool. Founded by Tang Jie, associate dean of the Beijing Academy of Artificial Intelligence, Zhipu AI developed a pre-trained large language model called ChatBLM. The company has also launched a partnership with internet security company Qihoo to collaborate on 360GLM — a large language model that can be integrated into Qihoo’s search engine.

DeepLang AI

Product: Chinese-language-trained large language models such as WantWords and Jiuge 
Founders: Qi Fanchao and Li Xiaoxiang
Funding: Tencent, HongShan Seedfund, MiraclePlus, 1VC
Location: Dongsheng Plaza 

Founded by two natural language processing researchers at Tsinghua University, DeepLang AI set out to create large language models better equipped to deal with Chinese-language content. DeepLang’s products include WantWords, a dictionary that suggests a list of words based on users’ descriptions; and Jiuge, a classical Chinese poetry generator. The company has received funding from notable investors such as Tencent and MiraclePlus, and is backed by Tsinghua NLP Lab and the government-affiliated Beijing Academy of Artificial Intelligence.

1. Baichuan Intelligence

2. Zhipu AI

3. DeepLang AI

Are the main ones that come to mind, I’m sure there are others still in stealth. Baidu, Alibaba, Tencent, ByteDance and Meituan clearly are hitting LLM and generative A.I. pretty hard, to the extent that they can with China’s A.I. regulations.

PR that trickled into English media said:

Founder Wang Xiaochuan expressed his willingness to collaborate with other enterprises and developers to foster the growth and prosperity of the domestic open-source community

The Alibaba founder lost about half of his fortune for speaking out against Government regulators. So it’s a weird time in Chinese tech and their startups scene looks anemic in 2023, and still haven’t recovered from either the Tech crackdown or Covid-19. This gives the U.S. a bit of head start when it comes to Generative A.I. innovation like we have been seeing with the lucrative rounds in the past few months.

Baichuan-13 is trained on 1.4 trillion tokens. In comparison, Meta’s LLaMa uses 1 trillion tokens in its 13 billion-parameter model. Wang previously said in an interview that his startup was on track to release a large-scale model comparable to OpenAI’s GPT-3.5 by the end of this year. If that’s true, then that will be quite impressive.

Sogou was a half decent search startup company. Baichuan Intelligence emphasized the pressing need for China to establish its own open-source large-model ecosystem. All this being said China likely has a lot more data in some areas than the West does. Maybe however proprietary data that isn’t as lucrative.

https://github.com/baichuan-inc

TL;DR:

• Baichuan Intelligence, founded by Wang Xiaochuan, launched Baichuan-13B, a cutting-edge large language model (LLM) aimed at rivaling OpenAI.
  

• Baichuan-13B, based on the Transformer architecture, is a 13 billion-parameter model trained on Chinese and English data.
  

• Wang’s startup has gained significant momentum, securing $50 million in financing and releasing the pre-training model Baichuan-7B.
  

• Baichuan-13B offers an open-source platform optimized for commercial applications and can operate on consumer-grade hardware.
  

• Other Chinese firms, such as Baidu, Zhipu.ai, and IDEA, have also invested heavily in LLM development.
  

• China’s stringent AI regulations and the need for licenses may impact the competition between China and the US in the large language model industry.

China’s A.I. regulations and rules require generative AI providers to adhere to core socialist values, which prohibit everything from pornography and terrorism to racism and content that threatens China’s national security. How will China deal with AGI or ASI, it’s hard to feasibly understand how they are even evaluating the risk to the CCP.

A.I. could certainly undermine voting, democracy, free media, misinformation and a whole gamut of other things. Yet the U.S. does not seem to be terribly bothered.

China’s unique power structure and CCP style ethics means it wants the power of A.I., but without the risk that some of these startups might bring into the world. AI development in China has been a top-down effort. The document is calling for the creation of a public data training platform and the sharing of computing power. Concrete rules have already been proposed in Beijing for a state-backed, centralized platform that allocates public cloud resources based on customer needs. It feels as if Beijing is struggling how to bring next-gen A.I. to China.

We will see.

This month, Wang’s nascent startup, Baichuan Intelligence, has taken a significant step towards realizing this ambition with the introduction of its cutting-edge large language model (LLM), Baichuan-13B.

Thanks for reading!