AI Team Structure: What Roles Do You Need?

The Starting Team: 1 to 3 People

Most companies start here. You have an AI initiative, leadership buy in, and a budget for one to three hires.

• One Senior AI Engineer: Full stack on AI. Ships features, talks to customers, picks the infrastructure. This hire sets the tone for everything that follows.
• Optional: one Data Engineer: Only if you have meaningful data infrastructure work ahead. If you are primarily calling external APIs, skip this for now.
• Optional: one Product Manager with AI fluency: Only if your AI product surface is customer facing. Otherwise the CEO or Head of Product can cover it.

The anti pattern at this stage: hiring a Research Scientist or ML Engineer who has never shipped production code. They will build beautiful notebooks that never become products.

The Growth Team: 5 to 10 People

You have shipped the first AI product, customers are using it, and the roadmap is no longer guesswork.

• One or two AI Engineers: Product focused, shipping features week over week.
• One ML Engineer: Owns model training, evaluation, and deployment for any custom models.
• One Data Engineer: Builds and maintains data pipelines, feature pipelines, and data quality tooling.
• One AI Product Manager: Owns roadmap, prioritization, customer discovery, and eval definition.
• One MLOps Engineer or ML Platform Engineer: Add when you hit three models in production or two engineers are doing infrastructure work.
• Optional: one AI Solutions Engineer: Add when enterprise deals require customer facing technical work.

At this stage the team starts feeling like a real function rather than a skunkworks. Engineering manager or tech lead emerges organically from within the group, usually the first senior AI engineer.

The most expensive hiring mistake at the growth stage is adding a second generalist AI engineer when what you actually need is an MLOps engineer and a product manager.

The Scale Team: 15 Plus People

Your AI function is a distinct org. You have multiple product surfaces, dedicated roadmaps, and meaningful revenue tied to AI features.

• Engineering Manager or Director of AI: Owns the org. Hires, one on ones, planning, partnership with product and go to market.
• Two to four AI Engineers: Product focused, organized by product surface or feature area.
• Two to three LLM Engineers: Specialized in fine tuning, inference optimization, and frontier model work.
• One to two RAG Engineers: Own retrieval systems, embedding pipelines, and evaluation.
• Two ML Engineers: Custom model training, evaluation, and traditional ML work.
• Two to three MLOps Engineers: Platform, serving, observability, cost management.
• One or two Data Scientists: Experimentation, analysis, model evaluation.
• Two Data Engineers: Data pipelines, warehousing, feature stores.
• Two AI Product Managers: One per major product surface.
• One AI Solutions Engineer per two or three AEs: If you have an enterprise motion.
• Optional: one Research Engineer or Applied Scientist: Only if your competitive moat depends on proprietary model work.

Structure usually splits into platform and product pods. Platform owns infrastructure, tooling, and shared services. Product pods embed AI engineers with product engineering on specific surfaces.

Common Mistakes

Patterns we see over and over at companies South works with.

• Hiring researchers before product market fit: Research talent is expensive, slow to translate to product, and demotivated by ambiguous goals. Hire applied engineers first.
• Skipping MLOps until production breaks: When you finally hire MLOps, the first six months are retroactive cleanup instead of forward progress.
• Skipping the PM: Engineering led AI roadmaps optimize for technically interesting problems. Without a PM, you will ship demos, not product.
• Over rotating on AI Solutions Engineering: Hiring a team of SEs before you have enterprise deal flow creates expensive idle capacity.
• One job description for "AI Engineer": Covering prompt engineering, fine tuning, MLOps, and RAG in a single role produces unicorn searches that take nine months to fill.
• Hiring ahead of the problem: Adding the third ML Engineer before you have data to train on just creates expensive notebook work.

How to Sequence the Hires

If you could only add one hire per quarter, this is the sequence that works for most product led AI teams:

1. Senior AI Engineer (product focused)
2. AI Product Manager
3. Second AI Engineer
4. Data Engineer or MLOps Engineer (whichever is the bigger bottleneck)
5. ML Engineer (if you are training custom models)
6. Second MLOps Engineer or RAG Engineer (depending on stack)
7. Engineering Manager
8. AI Solutions Engineer (if enterprise motion)

This sequence prioritizes shipping over credentials, product judgment over technical depth in any single specialization, and operational reliability at the moment production starts hurting.

The LatAm Hiring Angle

Here is the math that makes LatAm sourcing attractive for AI teams.

• A full growth stage team of 5 (AI Engineer, ML Engineer, Data Engineer, MLOps Engineer, AI PM) sourced from Latin America through South typically costs $500k to $700k fully loaded per year.
• The equivalent US team runs $1.3M to $1.8M fully loaded.
• Two to three US engineers cost roughly what a full five person LatAm team costs.

The tradeoff is real but smaller than companies assume at senior levels. Mid level talent gaps have closed substantially since 2023. Senior engineers from Nubank, Rappi, Mercado Libre, Globant, and dLocal are working remote for US AI startups at scale in 2026.

Key Takeaways

• Starting teams need one Senior AI Engineer first, a PM second, and infrastructure hires only when the pain is real.
• Growth stage teams (5-10 people) should include an MLOps Engineer, a PM, and an AI Solutions Engineer for enterprise motions.
• Scale teams (15+) split into platform and product pods with specialized LLM, RAG, and MLOps roles.
• The most common mistakes: hiring researchers too early, skipping MLOps, skipping PM, and writing unicorn job descriptions.
• LatAm teams of five typically cost what two to three US engineers cost fully loaded.

Frequently Asked Questions

When should I hire my first AI Engineer vs ML Engineer?

AI Engineer first, always. The distinction: AI Engineers build with foundation models and production systems; ML Engineers train custom models. Unless your value proposition depends on proprietary models, you want the AI Engineer.

Do I need a Head of AI at Series A?

Usually no. A strong Senior AI Engineer with founder or executive level autonomy is enough until you have five plus AI engineers. Consider a fractional Chief AI Officer if you want senior judgment without a full time hire.

How do I know when to add my first MLOps Engineer?

Three signals: three or more models in production, existing engineers spending 20 percent plus of their time on infrastructure, or production incidents traceable to ML specific gaps (missing lineage, drift, unversioned data).

What ratio of AI engineers to product engineers is healthy?

There is no universal ratio, but a common pattern at AI native startups is 30 to 50 percent AI engineers within the total engineering org. At AI feature companies, 10 to 20 percent is more common.

Should my AI team report to the CTO or a dedicated AI leader?

At small scale, CTO. At mid scale (20 plus AI engineers), a dedicated Director or VP of AI reporting to the CTO. Only at large scale does a Chief AI Officer reporting to the CEO make sense, and even then the decision is often political rather than structural.

Hire AI Team Talent with South

South builds out full AI teams from Latin America for US and European companies, from first hire to scale org. Tell us your stage, stack, and headcount goals and we will return matched candidates across every role you need within two weeks. Start hiring with South.