If you want to know where enterprise AI stands — not in theory, not in glossy decks, but in the raw reality of implementation – you book your ticket to Ai4 2025. The conference was held on August 11-13 at MGM Grand, Las Vegas, and gathered Fortune 500 executives, government decision-makers, VCs, startups, and research heavyweights under one roof.

Our Business Development Executive, Mario Stalder, visited this year’s Ai4 conference and came back with firsthand insights into how global enterprises handle the realities of AI deployment.

The Ai4 2025 agenda covered, but wasn’t limited to, the following topics:

• AI Systems and Infrastructure: Architecting and future-proofing scalable AI systems, managing computational resources, handling big data, and building supporting digital ecosystems.
• Data Strategy and Governance: Building data-driven organizations, aligning data strategy with business goals, data governance, compliance, and ensuring data quality for AI initiatives.
• Industry and Functional Applications: AI in media and entertainment, SaaS, marketing, sales, customer experience, and product management.
• Cybersecurity: AI for threat detection and response, automating security tools, and enhancing incident response.
• Ethics, Policy, and Social Impact: Ensuring fairness and transparency, eliminating bias, AI alignment with human values, regulatory frameworks, AI safety, Green AI, and diversity in AI.
• Technical Deep Dives: Sessions on Generative AI, Retrieval-Augmented Generation (RAG), MLOps, and specialized summits for AI research and policy.

The room has moved past speculative proofs-of-concept and is now obsessed with governance, scalability, and whether the ROI is worth the compute bill.

CPG: The Reality Check Industry

If you want a stress test for AI’s scalability, look at CPG. Here is an industry sitting on mountains of data, under relentless market pressure, yet still failing to move from potential to production. Why? Because the architecture is ancient and the data is fragmented.

Thirty- to forty-year-old systems choke on modern AI workloads. Data silos, both internal and external, make the idea of a 360-degree customer view laughable. And even when they “go cloud,” they often just forklift their silos into shinier hosting.

This isn’t a CPG-only issue. Manufacturing, insurance, and logistics all have the same DNA-level issues. But CPG magnifies the pain with hyper-fragmented consumer demand, volatile e-commerce cycles, and a chronic lack of direct customer relationships. The takeaway: until you fix your data architecture, AI is a vanity project.

The Cost of Technical Debt

This deep-seated architectural and data-related technical debt leads to negative business outcomes. The top 50 consumer brands saw growth crater from 7.7% to 0.7% over a decade. Only 5% of CPG leaders say AI has made a real business impact. The rest are still cleaning data, migrating to the cloud, or testing pilot versions. The irony? They have the data. They just can’t connect it.

Leaders fixate on cost control, like forecasting tweaks and back-office automation. Useful, yes, but strategically timid. Meanwhile, AI-native challengers sprint past, redesigning entire customer engagement models.

Architecture, Talent, Focus

Breaking the cycle requires a foundation reset: data lakes, data fabric, data mesh — architectures that dissolve silos and enable fluid, real-time data movement. Platforms like Azure or Snowflake can anchor this, but only if paired with cross-functional teams that marry AI expertise with business domain depth. The culture has to shift, too: decision-making moves from gut feel to analytics-first, with tools that make AI accessible.

Finally, focus matters. Pick high-value use cases with measurable ROI (supply chain forecasting, trade promotion optimization, personalization engines) and scale from there.

Agentic AI in the Trenches: Who’s Building the Future

Generative AI hype is already yesterday’s news. The next big thing is agentic enterprise. We’re talking autonomous agents that can run multi-step workflows, plug into enterprise systems, and get things done without someone babysitting every move. Sounds great — until you try to do it for real. Then the friction shows up fast.

To make the agentic enterprise work, you need to get two worlds talking:

• Application & Governance Layer — building, orchestrating, and controlling agents so they don’t break things. That’s the CDO’s playground.
• Infrastructure & Security Layer — what guarantees agents run. That’s where the CIO and CISO live, dealing with networks, security, identity, and uptime.

Agents Missing in Action

Catalina Herrera, Field CDO at Dataiku, in her session "Agents Missing in Action: Hard Truths About AI's Next Chapter," uncovered three killers of agentic AI:

1. Fragile connections. Agents need tools to act – Jira, Salesforce, APIs, and databases. In most enterprises, those connections are duct-taped together in ways that snap under load. Without central orchestration, the whole thing is a Jenga tower.
2. Black box failures. When an autonomous agent screws up, you need to know why. Without a transparent trail of what it did, what it saw, and why it acted, you can’t debug it, and you definitely can’t trust it.
3. Accountability. If an agent blows up a deal or leaks data, who takes the hit? Without governance, auditability, and clear ownership, everyone ducks responsibility.

Dataiku makes orchestration and governance non-negotiable. Centralized integration replaces brittle connections. Guard Services track quality, cost, and safety. Trace Explorer shows each step of an agent’s decision. GenAI Governance keeps a full list of agents in the enterprise. And building agents is open to both no-code business teams and pro-code developers, so business and tech experts can work together.

Infrastructure as Revolution

While Herrera focuses on fixing current systems, Jeetu Patel, President and Chief Product Officer at Cisco, is redesigning the whole thing from the ground up. In his fireside chat with Matt Egan, called "The Catalyst for Driving an Agentic AI Revolution", he framed agentic AI as a complete restart, not a minor upgrade. With billions of agents coming, old enterprise systems won’t cope.

Cisco’s role here is to provide the essential tools. Networks, security, and data systems are rebuilt for agent-scale work. Cisco AI Defense adds safety at every stage: checking models, protecting during runtime, blocking malicious prompts, and preventing data leaks. These guardrails let you move faster with confidence.

Patel’s big move is AgenticOps — a system to manage IT in a world full of AI agents. The AI Canvas gives NetOps, SecOps, and DevOps one shared dashboard. Cisco’s Deep Network model is a networking-focused LLM that helps optimize and fix problems in real time. All of it runs in one place, combining Cisco’s tools and Splunk for visibility.

This AI-powered base also improves network uptime, streamlines supply chains, and powers Webex AI Agent for smoother customer interactions.

Two Layers, One Shot at Winning

The agentic stack is binary. The Application Layer (Dataiku) builds and governs the agents. The Infrastructure Layer (Cisco) keeps them fast, secure, and visible. Neglect either and you lose. A perfect agent on a slow, leaky network is worthless. A flawless network with no agents worth running is equally pointless.

The CDO, CIO, and CISO have to architect both ends together. That’s where the real value gets built.

The Data-Model Interface: Architecting with RAG

An LLM fresh from the box is brilliant in general knowledge but clueless about your business. It knows Shakespeare and quantum mechanics, but not your proprietary sales pipeline or last quarter’s supply chain logs.

To make it work for you, it needs access to your “library” — the proprietary, constantly updated data that defines your competitive edge. And you don’t just need the library, you need an intelligent librarian to fetch the right book at the right time. Tengyu Ma, Chief AI Scientist at MongoDB and Stanford Assistant Professor, laid out exactly how to build both in his keynote, “RAG in 2025: State of the Art and the Road Forward”.

RAG vs. Fine-Tuning vs. Long-Context

Tengyu Ma frames the question of how to inject domain knowledge into LLMs as a strategy-level decision, not just an engineering choice. You’ve got three weapons on the table — Retrieval-Augmented Generation (RAG), Fine-Tuning, and Long-Context Windows. Each comes with trade-offs you ignore at your peril.

Retrieval-Augmented Generation

RAG connects your model to an external, real-time knowledge base, like MongoDB or similar. A user query first triggers a retrieval step: pull the most relevant documents from the database, then feed them to the LLM alongside the query. The LLM answers with the context in hand.

Benefits: You get grounded, current answers with citations. Hallucinations drop, traceability rises, trust improves. Security is better because your proprietary data stays in the database, not baked into the model weights. You can swap out the model without re-engineering your entire knowledge store.

Challenges: RAG is only as good as its retrieval. Bad ingestion, sloppy chunking, or weak embeddings will tank the output. The retrieval pipeline itself is an engineering project — and a fragile one if done halfway.

Fine-tuning

Fine-tuning retrains a pre-trained model on your curated, domain-specific dataset, modifying its internal weights. You’re embedding your knowledge directly into its neurons.

Benefits: Perfect for imprinting style, tone, or personality. Also excels at teaching industry-specific jargon, complex decision patterns, and workflows too nuanced for a simple doc dump. If you want behavioral change, FT is the scalpel.

Challenges: Expensive in data prep, labeling, and compute cycles. You risk “catastrophic forgetting” — losing general skills in the process. And the knowledge is frozen in time; change the data, and you pay to re-run the whole process.

Long-Context Windows

LC means you stuff millions of tokens directly into the model’s prompt for a single query. It processes everything at once.

Benefits: Simple to conceptualize. No separate retrieval layer; you just give the model everything it might need.

Challenges: Prohibitively expensive in compute and latency, both scaling with context size. Worse, many models suffer from the “lost in the middle” effect — they lose track of information buried in the prompt’s midpoint, which defeats the purpose.

MongoDB as the Intelligent Librarian

Tengyu Ma’s future state for RAG doesn’t stop at “fetch and pass along.” The next phase is about turning the retrieval layer into an active, intelligent operator — one that handles the ugly, labor-intensive parts of data prep and query refinement. The database stops being a dumb vault. It becomes an active player in the reasoning loop.

In Ma’s framing, MongoDB is the librarian who knows your collection, anticipates your requests, and brings the right book before you finish asking. That’s powered by tools like Atlas Vector Search, which can cut through mountains of unstructured data with semantic precision.

Automating the Pain Points

Right now, building RAG is full of human bottlenecks. Ma’s vision is to strip them out.

• Automated Chunking and Enrichment: Instead of manually slicing documents and tagging them with metadata, the system handles it. The payoff: faster builds and better retrieval accuracy.
• Advanced Retrieval Techniques: Move beyond plain vector similarity. Hybrid search combines keyword and semantic search. Re-ranker models refine initial hits. Query decomposition breaks a complex question into sub-questions that are easier to solve, and stitches the answers back together.
• Controllability and Domain Specificity: Retrieval shouldn’t be guessing in the dark. If a user says “Jaguar,” the system should know if they mean the animal or the car, and it should take natural-language cues to bias results. Domain-specific embedding models (finance, law, code) sharpen accuracy even further.

Ma’s view also doubts the false choice between RAG and fine-tuning. The sharpest operators will do both. Fine-tuning sets the how — the model’s style, tone, reasoning habits, and understanding of internal jargon. RAG feeds the what — the fresh, verified facts the fine-tuned model needs to answer correctly. Together, you get a system that speaks in your voice and thinks with your latest intelligence.

Three Moves That Decide Whether Your AI Strategy Is Future-Proof

The AI space is a high-stakes game where the wrong move wastes time and money. Getting from big ideas to tangible results comes down to three moves that need solid engineering expertise to make them happen:

1. Declare War on Data Debt

Stop treating data modernization like an IT side quest. If your infrastructure can’t feed AI at scale, it’s a business survival issue. The goal isn’t “improvement.” It’s the demolition of the silos that choke agility.

That means implementing cloud-native architectures like data mesh or fabric, designed to connect every stray data source and make it usable. And for advanced RAG, you need an intelligent data layer built for vector search, automated prep, and semantic understanding. This is where Janea Systems' software engineering expertise makes a difference.

2. Architect for Governed Autonomy

AI runs on a two-layer stack: the Application Layer and the Infrastructure Layer. Your CDO/AI teams control the governance and orchestration. Your CIO/CISO teams keep the network secure, observable, and fast. If those two aren’t in lockstep, you’re building a house where strangers design the roof and the foundation.

Janea Systems brings governance frameworks, standards, and MLOps discipline that make AI initiatives transparent, predictable, and scalable, so machine-scale autonomy doesn’t come with machine-scale risk.

3. Build a Portfolio

Forget the false RAG vs. fine-tuning binary. Innovative leaders build a customization portfolio. Janea Systems specializes in designing these portfolios for maximum ROI, aligning the proper technique with the right business outcome.

AI success isn’t magic. Engineers must execute architecture, governance, and customization with precision. Get those three right, and you win. Janes Systems delivers AI that works at scale and drives measurable ROI. Our projects include:

• Bing Maps QAS: We re-engineered the deep learning pipelines for Bing Maps' quality assurance, resulting in a 50x faster AI and a 7x reduction in training time.
• Microsoft PowerToys: Our team was instrumental in porting the PowerToys suite to ARM64, which involved complex, system-level challenges like managing processor-specific code, handling native dependencies, and fixing OS-level interaction issues.
• PyTorch: As the maintainer for PyTorch on Windows, Janea Systems addressed significant instability and feature gaps that hindered developers. The engineering effort cut the number of open Windows-specific issues by nearly 50%, bringing the platform to parity with Linux.

Ready to move from AI theory to impact? Let’s talk about how Janea Systems can help you build the foundation, frameworks, and custom AI solutions at scale.