The Coming Rise of the Citizen AI Engineer
The enterprise role of the 2020s will be the Citizen AI Engineer.
This post is going to introduce a role of the future. The Citizen AI Engineer.
This role is fundamentally nontechnical but that heavily uses AI to solve business problems.
100s if not 1000s of new products will be built to service this end user. Let’s review some background, history, and get to know this coming role better. Then we’ll discuss the competitive landscape for serving this user.
Background: AI Engineering
The Rise of the AI Engineer on Latent Space is an insightful post about the generational shift in applied AI. In short, developers who are not applied researchers (or ML engineers) can now build incredibly powerful AI-powered applications. This development is set to change the game for AI adoption.
You should read the post, but here's my TL;DR: changes in technology are allowing a new role to emerge at companies of all sizes. The two key technologies are:
Foundation models - Developers can build with AI without needing to understand AI.
Foundational AI tools - Tools like LangChain, LlamaIndex, and others make it easy to integrate foundation models and providers into various applications. These exist across 2 dominant languages, Python and JavaScript.
The tools mentioned above are shifting the balance from ML engineering and ML researchers (a specialized skillset that will remain important but requires more training and experience) to the vast number of developers who know Python and JavaScript. Swyx put the consequence concisely:
A wide range of AI tasks that used to take 5 years and a research team to accomplish in 2013, now just require API docs and a spare afternoon in 2023.
Lastly, while it's not technology, the shear amount of news, excitement, attention, and money in AI is creating a Cambrian explosion of new projects, tools, and products in the AI domain. Our economy is built around what gets attention. AI has sucked the air out of the room.
In short, Swyx posits a new role, the "AI Engineer," is likely to emerge as a dominant role in software engineering over the next decade.
I agree AI Engineers will rise. However, I argue there’s even more to come that will have larger impact: The Citizen AI Engineer.
The Citizen AI Engineer
Just as we've seen the rise of Citizen Data Scientists over the past decade, I posit that the next decade will see the rise of the Citizen AI Engineer. This role will overshadow AI Engineers because of shear numbers. It is a business user, someone non-technical, who gains significant capabilities with LLMs on their own data.
What is a Citizen AI Engineer?
A Citizen AI Engineer is a role in which a business-focused, but fundamentally non-developer persona, will be able to leverage AI tools to build and ship all kinds of new products within enterprises - without a heavy reliance on developers (day to day).
Citizen AI engineers will use tools like ChatGPT and tools custom-built for them by AI engineers, to drive large-scale business changes in everything from operations to forecasting and supply chains.
Why Now?
There are several trends that are going to enable this user to succeed.
General Purpose Foundation models - The models aren't perfect, but as we've seen with the rise of ChatGPT, they can handle a number of tasks.
Foundational tools - Tools like LangChain, LlamaIndex, and others yet to be created, make it easy to integrate foundation models and providers into different applications. While Citizen AI Engineers won't use these tools directly, they're going to enable the proliferation of tools built by AI Engineers that give Citizen AI engineers superpowers.
Coding isn't just for Developers - Tools like Cursor, Github Copilot, and ChatGPT Advanced Data Analysis can do the work of entire developer teams. They aren't perfect, but they're extremely powerful. This is just the beginning and these tools have personally, written devOps pipelines, helped teach me typescript, and administer databases.
The combination of all these tools (and the growth they're going to experience in the next 5 years) represents a state change in enterprise applications.
What Does the Old World Look Like?
You're a data analyst at Home Depot. Every week, you're running a financial forecast against your data lake, bringing together several pieces of data to solve a business problem.
This report is one that is sent up to upper leadership.
You know some basic SQL, but have no knowledge of Python, R, or any other data science tools. The report and forecast you build involve bringing together several kinds of data in a way that is particular to your business. Every week, you and your manager get questions about the forecast and how certain factors affect that forecast. You get follow-ups about:
"What's happening in this region?"
"Why is there so much disparity in XYZ dataset versus another one"?
You're putting out 🔥fires🔥 and have to prioritize only the most urgent requests. Your manager has talked about hiring a data analyst to do some of these immediate reports, but you all haven't been able to get the headcount.
Your team has started looking into no-code app builders and other automated data analytics tools, but haven't been able to find something that fits the bill.
What Does the New World Look Like?
Your team just adopted Enterprise DataChat (note: this product doesn't exist, but it's not difficult to imagine. I have no insider knowledge here. This is pure speculation.)
Your AI engineering team just integrated several DataChat plugins to give access to business data sources and defined some auto-function-calls to perform semantic retrieval against corporate presentations and data in your business unit. They've also made available some DataChatFunctionExtractors
to extract data from various unstructured data sources.
Now, instead of building presentations, you log into DataChat and specify your report requirements. It's generated automatically. You can inquire about different variations but finalize on your same format, but include some notes for questions you're sure to get. It's taken a quarter of the time it used to take.
Moreover, because your organization has adopted DataChat, you're able to provide a closed-domain chat application as part of your presentation. Leadership can now chat with your presentation and ask and answer their own questions about the content and data feeding it.
DataChat enables you to monitor this "conversation" in real time, you've got complete observability on the entire conversation about your deliverable and even able to double check and work with your engineering team to add more data sources and functions to make this experience even more full-featured.
The Delta from Old to New
The old world is unidirectional - content is produced and consumed. It’s a factory.
The new world enables interactive consumption. You'll not just digest presentations, but actively question them, live. You’ll build your own derivatives with a few clicks and prompts.
AI Engineers Build tools, plugins, and integrate AI to your data and ecosystem.
Citizen AI Engineers bring AI and tools to the critical business problems that are relevant to your domain.
The History - We’ve Seen this Before
This is not a novel idea. It's just a repetition of history (in a new domain). It came up in the past decade with Data Scientists.
Data Scientists
On the ground at Databricks, I watched the rise of the Data Scientist. The problem is, hiring data scientists for so many enterprises was just impossible.
As the power of data science provides organizations with differentiating competitive advantages, the demand for talent is rising. Supply, meanwhile, remains too scarce to meet that demand. This has led to data science and machine learning (ML) being opened up to nontraditional roles, such as the citizen data scientist.
Gartner, 2021
Citizen Data Scientists
Citizen data science is a version of data science that lowers the barrier to entry and enables non-technical users to take on bigger and bigger company problems.
A citizen data scientist is a person who creates or generates models that leverage predictive or prescriptive analytics, but whose primary job function is outside of the field of statistics and analytics.
Gartner, 2021
They're given tools to become more independent and capable. They bridge parts of the enterprise.
Complementary roles such as business translators, developers, data engineers and machine learning architects together can support citizen data scientists to fill in the skill gaps that they lack.
Gartner, 2021
These Citizen Data Scientists perform "Augmented Analytics".
Augmented analytics provides a guided, smart approach to conducting several steps, such as augmented data preparation, augmented data discovery and augmented data science. D&A leaders can add these to the existing toolkit for CDS.
Move Over Data Science: Citizen AI Engineers and Augmented AI
This is going to be a crazy decade. I posit Data Science will not become irrelevant, but will actually reduce in scope. The Age of Citizen AI Engineering is upon us.
But don't take my word for it. Forward-looking technologists and enterprises are already seeing these changes too.
Suhail, I couldn’t agree more.
Or a recent HBR IdeaCast episode on Leading a Workforce Empowered by New AI Tools when talking about the Rise of AI Tools. In the podcast, HBR discusses the rise of AI tools being used by "Citizen Developers". Those that are there to solve business problems but need help from IT and developers to truly achieve the dream.
The Ecosystem is Ready as well
Tools like:
All of these companies are about giving business users access to data and enabling them to build applications without developers.
These tools are powerful. But if all of a sudden, they're augmented with AI in major and fundamental ways - you've got extremely powerful business users that are less and less reliant on technical users to deliver business value out of data. That’s the Citizen AI Engineer.
Zapier is probably one of the most forward looking companies here. They recently held a “Zapathon” to dive into all of AI’s potential - across technical and non-technical users.
Who's Ahead, Who's Behind?
The question inevitably comes - where is the competition. I'm not quite ready to call winners but at this time, OpenAI is quite far ahead. The reason being is simple: ChatGPT Advanced Data Analysis.
The Leading Edge: ChatGPT Advanced Data Analysis
When OpenAI rebranded this product, I knew it was laying a foundation for something greater. It's hard to understate how powerful this tool is.
Advanced Data Analysis is far more important than ChatGPT Enterprise. ChatGPT enterprise are just checkboxes. It’s a vehicle. There’s nothing novel.
But, if OpenAI can deliver on Advanced Data Analysis enabling access to and manipulation of enterprise data, at scale, we're going to see a lot of roles change overnight across the enterprise.
The Competitive Landscape
There's going to be more "data scientist's in a box" style products. That's a massive opportunity and the bedrock of the citizen AI engineer opportunity.
Where Does the Rest of the Competition Stand?
Overall, it's too early to tell. If there's one thing we don't know is how things are going to change as new developments come out. Here are some thoughts in no particular order or relevance - aside from Nvidia.
Nvidia - stands alone here. They're going to sell hardware, doesn't matter to who. Jensen will sell a lot of it.
Open Source - This isn't a company, but the power is there. Tools like OpenInterpreter and GPT4All represent, to me, the most likely force to reduce OpenAI's dominance. Check out the tools.
Cohere/Anthropic/et al - These companies aren't as in the public eye as OpenAI and candidly I don't know enough about their product plans. They could do extremely well, but the core problem in AI is one of distribution. Technology can do amazing things. If you can't get it into the right hands, it’s irrelevant.
Google / GCP - Google allegedly has Gemini - “a serious threat to GPT-4”. I’m torn. On the one hand, Google has always had great tech. But their lack of customer intimacy has repeatedly hurt them. I’m not saying they can’t win, but it’s not clear to me they will.
AWS - AWS is the standard in the cloud and they’ve got great distribution. Their Bedrock partnership strategy seems interesting and keeps them in the game while they play a bit of catchup.
Azure - Azure is a natural winner. I'm continually impressed with Satya's ability to predict the future and forge partnerships in critical areas. From Data to AI, he’s got strong coverage.
Databricks is in a great place. They’re generally close to the AI engineer and developer persona. Certainly closer than the immediate competition - assuming the role doesn't drift too much towards JavaScript land. If you bet on a proliferation of models outside of the OpenAI ones, then Databricks should be in great shape because data (Spark) and model training (Mosaic) will be critical.
Snowflake has potential, but I haven't seen a strong AI offering outside of Frank Slootman and Jensen holding a snowboard together that represents that they're ready to capitalize on this opportunity.
HuggingFace - HuggingFace has an interesting role as a repo for AI, but at some point the rubber has to hit the road and their bias towards the ML engineer / research might not prevent them from succeeding in this domain.
AI21 Labs - They’ve got some great metrics for RAG Based Q&A Bots and they’re hitting the ground running. Early to say, but seems like a great opportunity.
Anyscale - Anyscale has a great position around ML tooling to build these kinds of tools. Ray, the open source project Anyscale is built around and created by the founders of Anyscale, is used to train ChatGPT. Lots of potential and the space is so early.
Conclusion
In this post, I define a new role in a new world. The Citizen AI Engineer. Building on the foundational role definition of the AI engineer, I posit that we're going to see a Cambrian explosion of business users that are AI-enabled.
The developer role will continue to be critical and see immense gains in productivity and power, but the massive wave coming is for business users. It's just not evenly distributed.
What the Future Brings
The next 10 years are going to represent massive, structural change in the enterprise. How companies are built, grow, and scale. I don't know all the implications and no one else does, but entrepreneurs have unbelievable opportunity. I believe that general purpose tools will be great, but specialized tools for specific problems will be even better.
The Citizen AI Engineer (or whatever the role will be called) will be a foundational part of this new world.
If you're building in this space, I'd love to talk.
Special thanks to Richard Liaw and Max Pumperla for their contributions to this post.