A company of 50 to 500 people does not need an AI department or a Chief AI Officer. It needs one accountable business owner for AI, one AI champion per department, engineering capacity sized to the use cases that actually reach production, and light governance: a one-page usage policy, a register of AI tools, and AI literacy training under Article 4 of the AI Act.
Search for "how should I structure my company for AI" and you get two kinds of answer. Some explain how to start an AI company. The rest are written for American enterprises with a Chief AI Officer, a centre of excellence and a budget few European mid-sized firms have. Imagine a 180-person manufacturer in Pitești or Brno: it fits neither.
The difference matters, because the wrong structure costs more than no structure. Name an "AI lead" with no authority and AI stays a hobby. Hire a CAIO into a 150-person firm and you have an expensive person looking for work. What follows is the version I think fits mid-sized companies in Romania and across the EU. The implementation steps themselves (bottleneck, pilot, ROI) are in my guide to implementing AI in your company. This piece is only about who does what.
Who should own AI in a mid-sized company?
A business leader, not IT alone. The AI owner should be someone who feels the result in their own numbers, usually the COO or a general manager, reporting straight to the CEO.
The reason is plain. AI changes how the work is done, not only the systems it runs on. IT knows how to grant access, secure data and connect a tool to the ERP. It cannot decide that procurement will work differently from Monday. When AI is "the IT project", the tools get installed and nobody uses them. I went through the same failure from the process side in why AI is not delivering results.
The decision rule I suggest: the AI owner is the person who would lose something if the project fails to deliver. If you cannot name anyone like that, you do not have a project yet. You have a curiosity.
The split that works at this size:
- The business owner (COO or a general manager) picks the use cases and answers for adoption and for the metric.
- IT handles accounts, data access, security and integration with the ERP and other systems.
- The DPO or in-house lawyer decides what data may go into which tool, and handles vendor contracts and GDPR.
- The CEO gives it ten minutes a month and decides what stops and what scales.
Do I need a Chief AI Officer?
Below 500 people, usually not, at least not as a new full-time hire. The title should follow the workload, not come before it.
A CAIO makes sense when AI is your product, or when you have several AI systems in production across different departments and someone has to coordinate them full time. Until then, an existing executive can take the role, with a clear share of their time and real authority. It costs less, and that person already knows how the company works.
If you want the title anyway, give it to someone who already has a team and a P&L. In my view, a CAIO with no people and no budget soon becomes an evangelist with good slides.
What does an AI champion do in each department?
An AI champion is the person in a department who tests tools on real work, shows colleagues what works and tells the owner where the next project is. It is not a technical role.
In practice, the champion:
- Collects use cases from the department, with volumes and time lost, not just ideas.
- Tries the first tool or prompt on real tasks and notes where it goes wrong.
- Keeps a simple page of examples that work, so colleagues stop reinventing the same prompt.
- Flags risks: customer data, personal data, decisions that should not be automated.
- Meets the other champions and the owner every two weeks.
Who should it be? Not necessarily the youngest or the most "technical". Pick someone curious, respected by colleagues and who knows the process in detail. Take a hypothetical accountant with twelve years in the firm who knows every exception in the invoice flow: she is worth more in this role than an eager junior. And put the time in the calendar, a few hours a week. Without it, the role exists only on paper.
Where should the engineers sit: hire, embed or partner?
It depends on how many AI use cases you run in production and how much they depend on your own data and processes. For the first project, few mid-sized companies need an internal AI team.
| Option | When it fits | Watch out for |
|---|---|---|
| Ready-made tools, no engineers | The first months: a team assistant, transcription, drafting | You still need a policy and training. You build nothing of your own. |
| External partner, per project | One clear use case with a start and an end | Make sure the code, the documentation and the evaluation set stay with you. |
| Embedded engineer (forward deployed engineer) | A use case that has to reach production and daily use, on your systems | One of your own people should pair with the engineer from week one to take the system over. |
| Internal hire | Several AI systems in production that need continuous upkeep | A lone AI engineer with no prepared data risks having nothing to build. |
The forward deployed engineer model comes from software companies that sent their engineers to work directly at the client. Adapted to a mid-sized firm, it means one engineer who sits in your team, works on your systems and takes a single use case through to production and adoption. It starts with an audit. At the end, you keep the code, the documentation and the evaluation set, plus an internal engineer who paired with them from the start. What Sapio, the company I founded, does is on its services page.
Whichever option you pick, the rule holds: engineers answer to the business owner, not the other way round.
What AI governance does a small or mid-sized company actually need?
Three short documents and one habit. A 200-person company does not need an ethics committee.
- An AI usage policy (one page): which tools are approved, what data never goes into them (customers' personal data, trade secrets), and who checks an output before it reaches a client.
- An AI register (one table): every tool or system, who uses it, for what, what data it receives and who is responsible. It helps with GDPR, with audits and when a large customer asks.
- AI literacy training. Article 4 of the AI Act has required, since 2 February 2025, that companies using AI systems take measures on their staff's AI literacy. The Digital Omnibus (Regulation (EU) 2026/1744, published in the Official Journal on 24 July 2026) rewrote the article: companies must now "take measures to support the development of AI literacy", taking into account people's experience and the context of use, without having to guarantee a specific level for any individual (text on EUR-Lex, read 26 September 2026). The obligation stayed; the wording became more proportionate. What that means in practice is in AI Act Article 4: what SMEs need to do.
- The habit: a quarterly review of the register by the owner, IT and the DPO together.
Fear of the law holds back more firms than it should. In Eurostat's 2025 data, among EU enterprises that considered using AI but did not, 52.5% cited a lack of clarity about the legal consequences and 70.9% a lack of relevant expertise (Eurostat, read 26 September 2026). The light governance above addresses both for far less than a compliance programme.
This article is not legal advice. For specific cases, especially if you use AI in hiring or in decisions about people, talk to a lawyer.
What does the structure look like by company size?
Treat the table as a rule of thumb, not a standard. Adjust it to how many AI processes you run in production, not only to headcount.
| Company size | AI owner | AI champions | Engineers | Governance |
|---|---|---|---|---|
| 10–49 people | The founder or general manager | 1–2 people, informally | None in-house; ready-made tools or a project partner | One-page policy, register, short training for everyone |
| 50–149 | COO or general manager, with time set aside | One per department | Project partner or an embedded engineer for the first system | Add the DPO, quarterly review |
| 150–299 | An existing executive with an explicit AI owner role | One per department, with calendar hours | Embedded engineer plus an internal engineer who takes over | The register becomes mandatory for any new tool |
| 300–500 | A dedicated executive, possibly titled Head of AI | A champions network with a monthly meeting | A small internal team, once 2–3 systems are in production | Role-based policy, training by level |
Eurostat shows how uneven the starting line is. In 2025, 20% of EU enterprises with at least 10 employees used AI, but only 5.2% in Romania, the lowest share in the EU (Eurostat, 11 December 2025). By size, across the EU: 17% of small firms, 30.4% of medium-sized ones and 55% of large ones (Eurostat). A mid-sized company that sets up a simple structure now is not behind. It is ahead of most.
What should I do in the next 30 days?
- Name the AI owner and write one paragraph on what they answer for.
- Pick a champion in two or three departments and block time in their calendars.
- Write the one-page usage policy and start the register, even if it has three rows.
- Run one training session for the management team, then one for the champions. My AI workshop for leadership teams is built for exactly this step.
- Only then choose the first project and decide where its engineers sit. The preparation it needs is in what to change in your business for AI.
Notice the order. Structure comes before the tool, but not by much. A month, not a year.
I keep coming back to the same thought: an org chart cannot make AI work, but it can stop it. "Who should own AI?" sounds like a question about a job title. It is really a question about who is allowed to change how the work gets done. If the answer is nobody, no title will help.
If you want to settle the owner, the champions and the first project for your company together, the details are on my AI consulting page.