Cephable is officially available on Microsoft Marketplace! For Microsoft customers, this means you can now purchase Cephable Professional directly through your M365 or Azure billing. It’s a simpler way to get started, scale across teams, and manage AI tooling where you already buy your enterprise software.

You can now add local, on-device AI to your suite of tools to help teams control, create, and automate across your organization.

👉 Get started on Microsoft Marketplace here

But this isn’t just about a new purchasing channel. It’s about where enterprise AI is headed.

On-Device AI: How Work Actually Gets Done

Most real work doesn’t happen inside a centralized AI tool. It happens in emails, documents, CRMs, and internal apps. On-device AI brings intelligence directly into those moments.

For employees, that means they can:

• Get work done privately and securely
• Keep sensitive information local
• Use AI naturally inside their daily tools

It’s faster, more practical, and aligned with how people actually work.

Why This Matters Right Now

Teams are already using AI every day to write, generate, and move faster. The question isn’t if AI belongs in the workplace anymore. It’s how to deploy it responsibly at scale, with tools your team will actually want to use.

That’s where hybrid AI comes in:

• Cloud AI for large models and centralized workloads
• On-device AI for everyday, in-the-moment work

This approach lets enterprises move forward without forcing everything into the cloud or locking innovation down. It gives teams flexibility while giving IT visibility and control, and creates opportunities to use tools that run locally and rely less on the cloud.

What This Unlocks with Cephable

Cephable is the local AI platform that helps people control, create, and automate across the software they already use. That includes voice commands, dictation, on-device intelligent tasks like text generation and translation, quick actions, and more.

Now, with Cephable on Microsoft Marketplace, teams can:

• Purchase through existing M365 billing
• Choose the right tier based on team size
• Scale with volume discounts
• Start with a free 30-day trial for up to 5 users
• Manage access and permissions in the Cephable portal
• Download the Windows app directly from the Microsoft Store

Ready to get started? This month, we’re offering a free 30-day trial for up to 5 teammates. Get started today through Microsoft Marketplace.

A New Era for Windows and Enterprise AI

Microsoft’s Ignite 2025 announcement reinforced what we’ve believed all along: AI belongs on your devices, governed by security and optimized for performance. As CEO of Cephable, I’m proud that our work was highlighted in Microsoft’s vision for Windows as the premier developer platform for AI. This isn’t just validation, it’s a signal that the future of enterprise productivity is here, and Cephable is leading the charge.

“Cephable empowers users with its suite of AI productivity tools… leveraging Foundry Local models to offer on-device AI with state-of-the-art models like Phi and Qwen. By running these advanced AI workloads locally, Cephable not only enhances productivity but also ensures user data remains private, significantly reducing the risk of data leakage and minimizing cloud computing costs.” — Microsoft Ignite 2025

We’re bringing tons of new on-device AI capabilities to Cephable Windows users through the rest of the year. Check them out.

Foundry Local + Cephable: Customizable, App-Aware AI

With Foundry Local, Windows now provides a high-performance local AI runtime and a catalog of optimized models. Cephable builds on this foundation to deliver customizable, app-aware AI that adapts to your environment and device for optimal performance.

Dynamic Model Selection: Cephable intelligently chooses the best model for the task—whether summarizing documents, drafting emails, or automating workflows.

Hardware Optimization: We leverage CPU, GPU, and NPU acceleration for real-time responsiveness, even offline. Built for the now and the future of AI PCs.

Bring Your Own Model: Enterprises can integrate proprietary or preferred models without sending data to the cloud.

This means your AI adapts to your apps and workflows, not the other way around.

MCP Integration: Turning AI Insights into Actions

Cephable is a launch partner for Model Context Protocol (MCP) on Windows, enabling AI assistants to take real actions in your apps. Whether you’re using Microsoft Copilot or another MCP-compliant client, Cephable acts as the “hands” of your AI—clicking buttons, updating records, and executing tasks securely across all your apps and workloads.

• Cross-App Automation: Extend AI capabilities to legacy and third-party apps.
• Secure by Design: Every action is logged and governed under enterprise policies.
• Seamless Experience: Issue a command through any AI assistant; Cephable makes it happen.

This bridges the gap between knowledge and execution, unlocking true productivity gains.

Enterprise ROI: Security, Savings, and Productivity

Cephable’s local-first approach isn’t just about technology—it’s about measurable business impact. Here’s what CIOs and decision makers need to know:

Cost Savings

By running AI on-device, Cephable reduces reliance on expensive cloud APIs and token-based billing. For enterprises spending $500K annually on cloud AI, Cephable can offset 20–40% of those costs by eliminating per-query fees and bandwidth charges.

Productivity Gains

On-device and in-app automation means employees spend less time on repetitive tasks. Based on early deployments:

• 140+ hours/year saved per employee through task automation and instant AI assistance.
• For a 1,000-person organization, that’s 140,000 hours reclaimed annually—equivalent to $7M in productivity value (assuming $50/hour fully loaded cost).

Security & Compliance

• Zero data leaves the device, reducing risk of breaches and simplifying compliance.
• Full auditability of AI actions for governance and trust.

Ready to Experience Cephable Today?

This isn’t a future promise—Cephable is available now. We’re featured on the Microsoft Store homepage, making it easier than ever to get started. Install Cephable on your Windows devices and see how secure, on-device AI transforms productivity. You can also get started with a free 30-day trial of Cephable Professional to realize the full potential of on-device AI for your entire team.

Why This Matters

AI is moving from the cloud to the edge, from passive assistants to active partners. Cephable is at the forefront of this shift—delivering privacy-first, cost-efficient, and action-oriented AI for enterprises.

Join us today. Explore Cephable, experience the difference, and lead your organization into the next era of productivity.

AI budgets are climbing fast—average monthly AI spend is projected to jump 36% this year—yet only about half of organizations say they can confidently evaluate the ROI behind those costs. (CloudZero) At the same time, daily AI usage among desk workers has exploded (up more than 200% in six months in some surveys), which means every small “ask” to an AI model is now a real line item in your P&L. (Salesforce)

For CIOs and CTOs in finance, tech, and healthcare, the issue is no longer whether to invest in AI. It’s what you’re actually paying for—and how much of that token and infrastructure spend is quietly wasted on workloads that could (and should) be handled locally on the hardware you already own.

This piece is about that gap: how AI cloud costs creep up, what they look like in the real world, and why shifting non-research, everyday tasks to on-device AI—using CPU, GPU, and increasingly the NPU—should be a deliberate part of your 2026 planning.

The new AI line item: token + infra burn

Most AI conversations still focus on model quality, safety, and “transformation.” Far fewer get specific about the cost mechanics:

• Tokens: Every prompt and response is billed in tokens—essentially the words and characters your models chew through.
• Infrastructure: You’re paying not just for model inference, but for the cloud platforms, vector databases, GPUs, observability, networking, and security layers that sit around it.

Recent research on AI costs highlights three uncomfortable realities for large enterprises:

• Average AI spend per org is already in the mid–five figures per month, and is expected to rise by about 36% year over year. (CloudZero)
• Cloud-based AI tools take the biggest share of that budget, especially public cloud platforms and generative AI services. (CloudZero)
• Only 51% of companies feel confident they can actually calculate AI ROI, largely because of hidden cloud expenses and poor cost attribution. (APMdigest)

Layer that onto BCG’s finding that only about 5% of companies are realizing significant, scalable value from AI while 60% struggle to achieve material gains and you get a pretty clear picture: spend is up, usage is up, and value is uneven. (BCG Global)

From what I’m hearing in year-end strategy reviews, CIOs aren’t questioning AI’s potential—they’re worried about drift: AI costs growing faster than discipline, especially when every low-stakes summary, rewrite, or translation defaults to an expensive cloud model.

Not every prompt deserves the cloud

It’s worth drawing a hard line between three categories of AI work:

1. Research and high-value reasoning Deep market or risk research Complex modeling and simulation Cross-domain analysis that genuinely benefits from cutting-edge cloud models and massive context windows
2. Domain-specific, sensitive workloads Anything touching PII, PHI, or regulated transactions Use cases where the risk of data exfiltration or mishandling is non-negotiable
3. Everyday “digital chores” (non-research tasks) Summarizing documents, calls, and emails Rewriting content for tone or clarity Breaking complex material into simpler explainer content Translating internal docs or messages Cleaning up notes, tickets, and handoffs

That third bucket—digital chores—is where token burn quietly gets out of hand.

The irony is that technically, the market has already given us a better option. Hardware and model efficiency have advanced to the point that smaller, GPT-3.5-level systems can now run at a fraction of past costs, and open-weight models are within a couple of percentage points of closed models on many benchmarks. (Stanford HAI)

So while training frontier models still demands massive, specialized infrastructure, running capable models for everyday tasks no longer has to—especially if you’re willing to use the silicon you’re already purchasing: CPUs, GPUs, and NPUs on AI-class endpoints.

For a lot of non-research, non-public-data work, sending every prompt to the cloud is like using a private jet for last-mile delivery: impressive, but economically absurd.

I think you will be hearing the term ‘heterogeneous’ more and more as it applies to the hardware and specifically silicon. Partners of ours like Intel are making great strides to configure their silicon to solve customers needs by reducing friction to deploy AI at scale. And this term has impact outside of the hardware space, it also lives in how systems are aligned to do the right thing, at the right time. You can learn more about Intel’s approach in their article.

What AI cloud costs look like in healthcare, finserve, and tech

None of this is abstract. Here’s how cloud-centric AI costs tend to show up in the three sectors you and I live in.

Healthcare: Paying premium rates to clean up notes

Where cloud AI is being used:

• Clinical documentation and note-taking

• Care coordination messaging and handoffs

• Operational reporting and quality documentation

What it costs in practice:

• Thousands of daily prompts to summarize, rewrite, and structure clinician dictation

• Repeated calls on long EHR notes, discharge summaries, and referral letters—for formatting changes or minor edits

• All of it routed through premium cloud models, largely for convenience, not necessity

Where this is overkill:

• Intelligent dictation + revision workflows that never leave the four walls of a hospital network

• Routine transformations of already internal data (e.g., “shorten this note for the patient portal,” “clean up this handoff for the night team”)

Most of these tasks don’t require new public knowledge. They’re formatting, clarity, and compression jobs. They’re ideal candidates for local models that can run on clinician laptops or edge devices—keeping PHI closer to the point of care and cutting token and infra spend at the same time.

Financial services: Using frontier models to rewrite paragraphs

Where cloud AI is being used:

• Turning complex research and risk output into digestible summaries

• Drafting client communications, briefs, and talking points based on internal analysis

• Summarizing call notes, service tickets, and case histories

Where cloud is absolutely appropriate:

• Deep research against public market, macro, and regulatory data

• Complex modeling where you genuinely want the strongest possible general-purpose model

Where it’s clearly overkill:

• Polishing and shortening internal content that’s already been vetted

• Turning analyst reports into bullet points for a client email

• Breaking legal or regulatory text into “explain it to me like a client” summaries

In finserve, sensitivity to PII and sovereign data is already forcing careful scoping for cloud AI. What often slips through the cracks is the economics: paying frontier-model rates to do glorified word processing on content that never needed to leave your perimeter in the first place.

Tech companies: Burning tokens to move text between tools

Where cloud AI is being used:

• Internal documentation summaries

• Product specs → short briefs → release notes → customer-facing copy

• Support tickets and incident reports → postmortem drafts

• Developer-written content repackaged for other audiences

Where you don’t want this piece to go:

• Deep into dev tools. The goal here isn’t to critique Copilot or similar. It’s the layer around those tools.

Where overkill shows up:

• Taking content generated in dev or product tools and repeatedly summarizing, rewriting, and translating it as it moves into docs, marketing, sales enablement, and support

• Each transformation is small—“clean up this paragraph,” “turn this into a call script”—but they happen thousands of times a week

Here, cloud spend adds up not because a single query is expensive, but because volume multiplies small, unnecessary costs.

A conservative napkin math: what 20% misallocation costs you

Let’s do the simplest possible math for a 50,000-employee organization. And don’t take my math as gospel, do your own, see how it plays out.

Assumptions (all intentionally conservative):

• 60% of employees are active desk workers using AI regularly → 30,000 people

• Each uses AI for 20 small tasks per workday (summaries, rewrites, translations)

• Each task consumes roughly 1,000 tokens (prompt + completion)

• You’re using a premium cloud model at around $15 per million tokens

• 220 working days per year

• And only 20% of those prompts are “overkill” that could have been served by a local, on-device model

The math works out like this:

• 30,000 workers × 20 prompts/day = 600,000 prompts/day

• 600,000 prompts × 1,000 tokens = 600 million tokens/day

• 600M tokens ≈ 600 “million-token units”

• 600 × $15 ≈ $9,000/day in cloud inference for these chores

• Over ~220 days, that’s about $2 million/year in cloud AI just for everyday tasks

• If only 20% of that volume is “overkill” that could have run locally, you’re quietly spending ~$400,000/year on the wrong tool for the wrong job

That’s one workload category, under conservative assumptions, in a single large enterprise. As daily AI usage grows (and the Slack Workforce Index suggests it’s growing very quickly), that overkill slice scales with it. (Salesforce)

And remember: this is only the direct per-token spend. It does not include:

• The cost of cloud GPUs and dedicated infra

• New observability and cost management tools

• Security reviews, policy work, and approvals

• Data engineering and integration efforts to wire AI into your existing stack

CloudZero’s analysis is blunt on this point: hidden cloud and maintenance costs are among the biggest reasons companies struggle to measure AI ROI at all. (APMdigest)

Architect for heterogeneity: start local on whatever silicon you have

None of this is an argument to abandon cloud AI. You need cloud models for:

• Research and high-value reasoning

• Large-scale training and fine-tuning

• Scenarios where you genuinely benefit from global context and cutting-edge capabilities

But your architecture shouldn’t treat every prompt like it’s one of those.

A more durable pattern looks like this:

1. Classify workloads by risk and complexity, not just by “does it use AI.” Tier 1: High-value, research-grade, cross-domain work → Cloud is fine, maybe necessary. Tier 2: Sensitive internal workloads → Could be private cloud or tightly-controlled environments. Tier 3: Everyday digital chores on internal data → Default to local, on-device models wherever possible.
2. Architect for heterogeneity from day one. Start local on whatever silicon you already have—CPU and GPU in your current fleet—and then scale into the NPU wave as you refresh hardware.

Modern AI PCs ship with NPUs specifically optimized for low-latency, low-power inference on exactly the kind of small models that excel at summarization, rewriting, and translation. The point isn’t just speed; it’s cost control and predictability.

1. Don’t strand your existing fleet. If your entire AI strategy only runs on the newest hardware, you’re signing up for a long, expensive refresh cycle before you ever see broad value. Any serious local-AI plan should: Run efficiently on CPU/GPU for older endpoints Automatically take advantage of NPUs when they’re present Keep the experience consistent enough that users don’t care which chip is doing the work
2. Shift the default for low-risk tasks to local. Rather than writing stricter and stricter cloud AI policies and hoping employees comply, change the default: Local by default for non-research, low-risk tasks Cloud by exception for truly high-value workloads that justify the extra cost and complexity

When you do this, cloud AI becomes a deliberate strategic resource, not the path of least resistance for every “clean up this email” prompt someone fires off between meetings.

Where to start

If you’re reading this as a CIO or CTO and thinking “yes, but our AI roadmap is already in motion,” here’s a simple place to begin without ripping anything out:

1. Instrument what you already have. Get a clearer view of AI usage by function: who’s calling what models, for which tasks, and at what cost. You don’t need perfect attribution; directional stats are enough to spot obvious overkill.
2. Carve out the digital chores. Identify the top 3–5 non-research use cases in each major function (clinical doc, care coordination, ops; research → comms; internal docs → customer-facing summaries). Treat these as a distinct workload class and model what happens if 20% of those calls move local.
3. Pilot a local-first pattern in one or two high-volume teams. Pick a unit where the chores are obvious: clinical documentation, operations, or internal comms. Run a small pilot that keeps those workloads on-device, using models sized to run on CPU/GPU today and NPU hardware as it arrives.
4. Feed the results back into governance—not just finance. Use what you learn to refine not only budgeting, but policies: where cloud is warranted, where local is preferred, how exceptions are handled. The goal is a sustainable mix of cloud and local that your teams understand and trust.

Closing thought

We’re past the point where AI is a science experiment. It’s a fixed cost. And like any fixed cost, where you run the work matters as much as the work you choose to do.

Cloud AI isn’t going away—and it shouldn’t. But if every non-research, low-stakes prompt is hitting your most expensive models in your most expensive environment, you’re leaving easy money on the table and making it harder to see where AI is actually moving the needle.

If you’re wrestling with these questions—how to balance cloud and local, where NPUs and your existing fleet fit, and what “good” looks like for AI spend in a 50,000-person org—I’m always happy to compare notes.

If everything important in your company routes through a few upstream clouds or SaaS providers, small hiccups become hard stops. That’s the theme I’m hearing in year-end reviews across financial services, tech, and healthcare: AI-assisted work is spreading fast, dependencies are multiplying, and ordinary Tuesday problems—an IdP wobble, a throttled model endpoint, a flaky branch VPN—now pause real work.

This isn’t an argument against the cloud. It’s a reminder that resilience comes from optionality. When the network sneezes or a provider changes a policy, your teams still need to move.

Why this risk is rising

1) AI is everywhere, and that concentrates risk.
Summarizing, re-writing, translating, and drafting are now embedded into daily flows. Those micro-tasks often depend on identity, APIs, and model endpoints you don’t control. Leaders also expect hybrid human+agent teams to be standard going forward—which is great for throughput, but brittle when an upstream service blinks. Accenture’s executive survey points to agents moving deeper into the “digital core,” increasing the number of touchpoints that can fail at once.

2) Adoption is outrunning guardrails.
Shadow AI—employees adopting tools outside official channels—has penetrated most organizations. That increases the blast radius of outages and policy violations because critical work quietly depends on unsanctioned services. One 2025 snapshot notes the prevalence of insecure AI apps in shadow use and the concentration risk around a handful of popular platforms.

3) Regulatory uncertainty is still the top brake.
Across industries, regulatory compliance—and how quickly you can meet it—has emerged as the biggest barrier to deploying GenAI at scale. That matters for resiliency because when an upstream vendor trips a rule or shifts processing to a non-approved region, you may be obligated to suspend that integration immediately. If all work depends on it, policy compliance becomes business interruption.

4) The economics make “local” viable.
The cost to achieve “good-enough” performance with smaller models has fallen dramatically, while efficiency has improved. That lowers the barrier to putting capable models closer to where work happens—without large infrastructure programs.

How cloud dependency shows up on an ordinary Tuesday

• Identity chokepoint: A transient SSO issue strands clinicians out of the EHR or advisors out of core tools. With no local fallback for basic documentation, minutes compound into missed SLAs.
• Rate limits & regional incidents: A model endpoint or embeddings store throttles. Support teams lose summarize-and-respond; analysts lose drafting assist mid-workflow.
• Edge fragility: Branches and field teams feel every VPN hiccup. If your “AI assist” lives 100% in the cloud, the line stops when the link blinks.

None of these require a breach or a headline outage. They’re the predictable side effects of centralizing “digital chores” on infrastructure you don’t operate.

The regulatory twist: upstream violations, downstream consequences

When an upstream processor violates a rule (data residency, purpose limitation, safeguards) or changes processing locations under capacity pressure, you inherit the obligations:

• Forced disconnects: DPAs and sector guidance can require immediate suspension of processing with non-compliant vendors. If your workflows can’t operate without that service, Legal’s “pull the plug” becomes an operational outage.
• Discovery and containment burden: Shadow AI complicates incident mapping; proving containment across unsanctioned apps is slower and costlier. Reducing reliance on external calls for routine tasks narrows the attack—and outage—surface.

A practical resiliency posture (no re-platform required)

The intent here is de-risking the cloud, not replacing it. The pattern below avoids deep infra changes or multi-quarter platform programs.

1) Make “digital chores” local-first.
For high-volume, low-risk tasks—summarize, re-write, translate, label, templated draft—run on device and sync artifacts later. This removes a round-trip and keeps work moving when the link misbehaves. Falling inference costs and increasingly capable small models make this a low-friction addition rather than a rearchitecture.

2) Prefer “drop-in” tools over “big-bang” rollouts.
Executives cite regulatory uncertainty and risk management as major brakes on deployment. Local tools that don’t require new data pipelines, privileged access to systems of record, or cross-border data movement tend to face shorter legal and security reviews—because they process on the user’s machine and leave source systems untouched. Use them to relieve pressure while bigger programs mature.

3) Cache the obvious.
Keep stable prompts, policy text, product specs, and reference snippets cached locally with periodic refresh. During a blip, “good enough” context beats “no context.”

4) Design for graceful degradation—lightly.
You don’t need a dual-region AI mesh. A simple rule is enough: try cloud; if slow or unavailable, use local; queue external calls for later. Users should get a usable draft rather than an error.

5) Treat identity as a dependency.
Allow time-boxed offline capture for low-risk actions (notes, drafts), store locally with tamper-evident logs, and require re-auth before sync. This turns SSO wobbles into minor annoyances.

6) Measure continuity, not just accuracy.
Track “work-blocked minutes per incident,” “draft-creation latency during degraded mode,” and “offline task completion rate.” Those are the KPIs that prove resilience to your board—even if you can’t publish them externally. Deloitte’s tracking shows many firms still need 12+ months to untangle governance and value-realization challenges; continuity KPIs keep teams focused on business impact while the big rocks move.

Where this lands by sector

Financial services.
Keep KYC notes, case summaries, reconciliation stubs, and templated client comms moving locally if an API key is rotated, a model is throttled, or processing shifts jurisdictions. With regulatory compliance now the top deployment barrier, it’s pragmatic to reduce reliance on upstream processing for routine text operations.

Healthcare.
Clinicians should be able to capture and structure notes, produce plain-language patient instructions, and translate at the point of care—even during SSO or EHR API hiccups—then sync to the record on reconnect. That’s continuity and safety, not just convenience.

Technology & SaaS.
Field teams need airplane-mode workflows (intake → summarize → prep file) that perform in a hotel ballroom with shaky Wi-Fi. It’s also a strong signal to customers that you’ve designed for real-world conditions. Executives expect agents to work across the digital core; start with the repetitive chores and avoid tying them to fragile upstreams.

Why “local” helps—even when the cloud is healthy

• Latency compounding: Saving seconds on thousands of micro-interactions adds up quietly.
• Review scope: On-device processing narrows security and privacy review scope because sensitive transforms stay local; legal can evaluate the tool rather than re-approve your data flows. Deloitte’s survey work shows risk/governance is the blocking issue—shrinking the review surface speeds time to value.
• Cost predictability: As cloud inference gets cheaper, usage often grows faster than savings. Local handling of routine tasks keeps unit economics stable during spikes.

A 60–90 day path that won’t disrupt your stack

1. Map the choke points. For your top 10 AI-assisted workflows, list every dependency (IdP, API, model, region). Mark the ones that create “no-work” failures.
2. Pick three chores. Per business unit, choose three high-volume tasks (summarize → re-write → translate is a common trio) and make them work locally with clean sync. No new data pipelines.
3. Run a “degrade day.” Intentionally throttle a model endpoint or simulate IdP downtime for an hour. Capture continuity metrics and a short fix list.
4. Codify the disconnect. Write a one-page playbook for what happens if a provider violates a policy or moves processing to a non-approved region—how to suspend safely without stopping essential work.
5. Socialize the wins. Use before/after continuity metrics and a simple demo to align stakeholders. Deloitte’s cross-industry read shows leaders are increasing spend but remain disciplined; visible continuity wins earn the right to scale.

In short, this feedback is exactly why many of our enterprise teams chose to layer in a local, on-device option—and why they’ve been able to show both quick wins and durable ROI without re-platforming. If your organization is wrestling with the same outage and compliance realities, I’m happy to compare notes, share what’s working (and what isn’t), and pressure-test a lightweight continuity plan with your leaders. No pitch—just a pragmatic conversation about resilience, optionality, and getting value sooner rather than later.

Resources & References

https://www.f5.com/resources/reports/state-of-ai-application-strategy-report

https://www.techradar.com/pro/a-quarter-of-applications-now-include-ai-but-enterprises-still-arent-ready-to-reap-the-benefits

https://hai.stanford.edu/assets/files/hai_ai_index_report_2025.pdf

https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/the-top-trends-in-tech

https://assets-c4akfrf5b4d3f4b7.z01.azurefd.net/assets/2024/05/2024_Work_Trend_Index_Annual_Report_Executive_Summary_663b2135860a9.pdf

https://www.mckinsey.com/~/media/mckinsey/business%20functions/mckinsey%20digital/our%20insights/the%20top%20trends%20in%20tech%202025/mckinsey-technology-trends-outlook-2025.pdf

Most of my year-end AI conversations with execs sound like this:

“We are narrowing in on or revising a formal AI strategy with expected results.
Also… our sustainability team is quietly freaking out about all the extra compute.”

That tension is real. The same AI story you’re telling to investors and employees—faster, smarter, more automated—is bumping up against the story you’re telling about energy, water, and long-term ESG commitments.

This piece is about the silver lining I’m seeing in those conversations: if you’re willing to rebalance where AI runs, you can take real pressure off data centers by shifting a meaningful slice of routine AI work onto hardware you already own.

No magic. No greenwashing. Just better placement of workloads.

The uncomfortable math behind “AI everywhere”

A few data points to ground this:

• AI isn’t niche anymore. One large global survey found that 96% of organizations are already deploying AI models in some form.
• The infrastructure to serve that demand is exploding. Data-center construction tied to AI has soared ~40% year over year, raising red flags not just about power but about water consumption to keep those facilities cool.
• At the model level, training compute and power requirements are doubling on aggressive cycles—roughly every five to twelve months depending on what you measure—and the power required for AI training is increasing annually.
• The emissions gap is widening fast. Training early models like AlexNet emitted hundredths of a ton of CO₂; newer frontier models are in the hundreds to thousands of tons per training run—far above the ~18 tons a typical American emits in a year.

Even with hardware getting more energy-efficient over time, the combination of bigger models, more usage, and more automation is pushing total demand sharply upward.

That’s why AI is starting to show up not just in your digital strategy decks, but in ESG briefs and sustainability risk registers.

AI is now a sustainability question, not just a tech question

We’re already seeing early signs of cultural pushback.

Fast Company recently highlighted public sector workers in the UK who are reluctant to use AI tools specifically because of net-zero and climate-commitment concerns, and city IT teams in the U.S. beginning to vet AI projects through a sustainability lens.

At the same time:

• Regulators are moving quickly on climate and AI governance.
• Investors are reading ESG disclosures with more skepticism.
• Younger employees—especially in tech and healthcare—are paying attention to where and how AI runs, not just what it can do.

McKinsey’s latest tech trends work calls out data-center power constraints, grid access, and physical infrastructure frictions as a key scaling challenge for AI and other compute-heavy workloads.

In other words: even if your AI business case clears the financial hurdle, you still have to answer, “What is this doing to our footprint and our story?”

The silver lining: you already own a lot of the compute

Here’s the part that doesn’t get enough airtime in boardrooms:

You’ve already paid for an enormous amount of compute that sits on desks, in carts, and in bags—laptops, desktops, workstations, thin clients—with CPUs, GPUs, and increasingly NPUs that are idle or underused most of the time.

A few trends make this strategically interesting:

• Endpoint hardware is getting more efficient every generation. ML hardware performance has been improving while energy efficiency increases around 40% per year, meaning you’re getting more useful work per kilowatt on newer chips.
• Smaller, cheaper models are catching up. The AI Index shows the cost of running GPT-3.5-level performance dropping more than 280x in about 18 months, as small, efficient models become viable.
• Most high-volume enterprise use cases today are lightweight but frequent: drafting, summarizing, rewriting, translating, routing, and nudging actions—what I usually call “digital chores.”

Those “digital chores” are exactly the type of workloads that can run well on devices you already have, using combinations of CPU, GPU, and NPU, instead of hitting a distant data center for every single prompt.

This doesn’t eliminate the need for cloud AI. You’ll still need large models and shared services for:

• Multi-party workflows and external experiences
• Heavy multimodal workloads
• Training and fine-tuning
• Cross-tenant analytics

But you absolutely do not need to ship every bit of day-to-day reasoning to the cloud.

What “local, on-device AI” actually means (in plain English)

When I say “local, on-device AI,” I’m talking about:

• Models and automation that run on the employee’s machine (laptop, desktop, workstation), not in someone else’s data center.
• Data that never leaves the device for routine tasks—drafting, summarizing, translating, or triggering automations across apps on that same machine.
• Hardware-agnostic acceleration:
  • Runs on CPU only for older devices
  • Takes advantage of GPU where it’s available
  • Lights up NPUs on newer “AI PCs” as they roll into the fleet

That last part matters. A credible strategy here cannot assume every device has a cutting-edge NPU. You need software that:

1. Is efficient enough to be useful on CPU.
2. Can offload selectively to GPU where present.
3. Automatically accelerates on NPUs as refresh cycles bring newer endpoints into the mix.

That’s how you honor prior CapEx on existing hardware, while still being ready for the next generation of devices. When we work closely with our partners like Intel, we are constantly pushing the value of their silicon as far as we can. On-device AI processing is an important component to managing an enterprise workforce. It pulls forward the value of investment beyond simply ‘future-proofing’ – spoiler alert: the future is here (cliche but true).

Why this matters for ESG narratives in finserve, tech, and healthcare

The executives I talk to aren’t trying to turn “we moved some prompts off the cloud” into their core ESG pillar. They’re trying to do three things:

1. Avoid an ugly surprise in their climate math.
   AI is now a non-trivial line item in your energy and water story. The AI Index shows that model training emissions are already at “hundreds or thousands of tons” per frontier model, and that total power usage continues to rise even as hardware gets more efficient.
2. Show they’re not blindly scaling compute.
   Reports from McKinsey, Deloitte, and others all converge on the same theme: AI adoption is accelerating, but scaling is constrained by infrastructure, governance, and risk—not just algorithms.
   Being able to say, “We deliberately kept routine AI workloads on existing hardware and reserved cloud capacity for what truly needs it” is a credible posture.
3. Align AI with existing “sovereign” and data-residency commitments.
   In financial services and healthcare, “sovereign AI” (keeping sensitive data and models within national borders or strict network boundaries) is becoming a design requirement, not a buzzword.
   Local, on-device AI is one of the simplest ways to keep a large percentage of sensitive work out of multi-tenant clouds altogether.

And there’s a business backdrop that’s easy to forget: Microsoft’s Work Trend Index shows 82% of leaders say productivity must increase, while 80% of the global workforce says they don’t have enough time or energy to do their jobs.

Digital labor is coming. The question is whether you’ll only buy it via massive data-center expansion, or whether you’ll let some of that intelligence live on the devices you already power.

A simple blueprint: rebalance, don’t rip and replace

If you’re in a CIO/CTO/CSO triangle, here’s a pragmatic way to approach this without turning it into a science project.

Tier your AI workloads

Do a fast classification of existing and planned AI use cases along two axes:

1. Data sensitivity
   • Public / marketing
   • Internal but low-risk
   • Regulated / highly sensitive (PHI, PII, trading data, clinical notes)
2. Compute intensity
   • Lightweight, single-user “digital chores”
   • Medium, team-scale tasks
   • Heavy, organization-wide or external workloads

Then apply a simple rule of thumb:

1. Local first for lightweight, single-user tasks with sensitive or internal data:
• Email drafting and rewrites
• Note-taking and summarization
• Internal translation and tone-shifting
• Simple “do X, then Y” automations on the same device
2. Cloud when you must for:
• Cross-team or external experiences
• Heavy multimodal reasoning (long video, complex agents)
• Training, fine-tuning, and shared analytics

You’re not trying to maximize “edge” for its own sake. You’re trying to minimize unnecessary data-center hits for routine work.

Make hardware-aware decisions, not NPU-only bets

When you evaluate software and platforms:

• Require a clear story for CPU-only performance on your existing fleet.
• Ask how the same stack uses GPU where it exists—especially in engineering, research, and imaging-heavy teams.
• Confirm that NPU acceleration is additive, not a hard requirement, so your roll-out can follow natural device refresh cycles instead of a forced forklift upgrade.

This is how you avoid a two-class workforce where only the people on brand-new hardware get the ESG-friendly, low-latency AI experience.

Bake it into ESG and AI governance together

Most organizations are still catching up on AI governance. F5’s AI Readiness research found that only 2% of surveyed organizations qualify as “highly ready” to scale and secure AI across environments.

Use that to your advantage:

• Add “workload placement” (cloud vs device) to your AI steering committee charter.
• Involve your Chief Sustainability Officer early, not as an after-the-fact reviewer.
• Track a couple of simple metrics:
• % of AI interactions served on local hardware vs cloud
• Estimated incremental cloud energy / water impact avoided by keeping routine tasks local (even if it’s coarse at first)

You don’t need perfect telemetry on day one. You need a defensible narrative and a plan to make it more precise over time.

How to talk about this in your next strategy or board review

If you want language that lands with non-technical stakeholders without overselling, something like this tends to resonate:

“As we scale AI, we’re being intentional about where the compute runs.
For routine, single-user tasks—drafting, summarization, translation—we’re shifting more work to the laptops and workstations we already own, using their CPUs, GPUs, and NPUs instead of always calling out to distant data centers.
We reserve cloud AI for the large, shared workloads that truly require it. This lets us capture productivity gains while reducing incremental data-center energy and water impact, and it keeps more of our sensitive data inside our existing network and hardware footprint.”

That’s honest. It’s measurable. And it frames local AI as part of responsible scaling, not a side project.

If you’re adapting this for your own organization

You don’t need to answer these back to me, but they’re the questions I’d have you run through internally:

• Which of our current AI use cases are truly cloud-dependent, and which could be served just as well (or better) on existing endpoints?
• How much of our workforce is already on hardware with GPUs or NPUs—and how will that change over the next 24–36 months?
• Where are ESG, security, and AI governance currently disconnected—and how do we get those stakeholders looking at workload placement together?
• What would it take for us to report, even at a high level, the percentage of AI work we keep local vs in the data center?

If you’re wrestling with those questions in finserve, tech, or healthcare and want a sparring partner—not a pitch deck—I’m always up for a conversation.

References

https://www.f5.com/resources/reports/state-of-ai-application-strategy-report

https://www.techradar.com/pro/a-quarter-of-applications-now-include-ai-but-enterprises-still-arent-ready-to-reap-the-benefits

https://hai.stanford.edu/assets/files/hai_ai_index_report_2025.pdf

https://arxiv.org/pdf/2504.07139

https://www.fastcompany.com/91411720/ai-energy-use-pr-problem

https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/the-top-trends-in-tech

https://www.mckinsey.com/~/media/mckinsey/business functions/mckinsey digital/our insights/the top trends in tech 2025/mckinsey-technology-trends-outlook-2025.pdf

https://assets-c4akfrf5b4d3f4b7.z01.azurefd.net/assets/2024/05/2024_Work_Trend_Index_Annual_Report_Executive_Summary_663b2135860a9.pdf

https://www.microsoft.com/en-us/worklab/work-trend-index/ai-at-work-is-here-now-comes-the-hard-part

https://itsg.us.com/our-latest-insights/the-2024-work-trend-index-ai-at-work-is-here-now-comes-the-hard-part1727564411

Your teams already have an AI strategy.

They just designed it themselves, one unapproved browser tab at a time.

Across my own year-end reviews with customers in tech, financial services, and healthcare, the pattern is the same: on the slide, AI is “in pilot.” On the ground, shadow AI is everywhere—and it’s quickly becoming a board-level risk.

This piece is about that gap, why it matters, and why keeping sensitive work on the device (instead of spraying it across random clouds) needs to be part of your answer.

What we mean by “shadow AI”

Let’s keep this simple.

Shadow AI is the use of AI tools, models, or services by employees (or vendors) without formal approval, oversight, or integration into your governance and security controls. (Baker Donelson)

It’s the AI version of shadow IT: tools that sneak into critical workflows because they’re useful, not because they’re safe. (ISACA)

And it’s not hypothetical anymore:

• One recent study found 59% of employees admit to using unapproved AI tools at work; 75% of those share sensitive data (employee records, internal docs, proprietary code). (TechRadar)
• Senior executives are actually more likely than frontline workers to use shadow AI—93% in one survey—often assuming they understand the risk well enough to “manage it.” (TechRadar)
• IBM’s 2025 Cost of a Data Breach research shows 20% of organizations in their sample already suffered a breach involving shadow AI, adding roughly $670,000 on top of the average breach cost. (IT Pro)

If you’re thinking “we’re still in the planning phase with AI,” you’re probably not. You’re in the clean-up phase—you just haven’t looked under the couch yet.

What shadow AI actually looks like in your org

Shadow AI is not some exotic corner case. It’s painfully normal behavior from people trying to do their jobs faster.

In tech

• Engineering & DevOps – Log dumps, stack traces, and architectural snippets pasted into public chatbots to debug issues faster. That often includes infrastructure details and secrets that would never pass a security review if sent to a vendor.
• Product & UX – Roadmap drafts, customer feedback exports, and design specs dropped into “free” AI slide/summary tools to prep stakeholder updates.
• Customer support & success – Full ticket histories and CRM notes fed into AI reply generators that live outside your official stack and logging.

None of this feels malicious to the employee. They’re just trying to move. But collectively, it’s a slow-motion data exfiltration program you never signed up for.

In financial services

• Analysts & research – Portions of risk memos, models, and strategy docs pasted into generic AI tools to “clean up language” or “get a quick summary”—often including material non-public information.
• Front-office & relationship managers – Client calls recorded and processed by unvetted meeting-note apps that store audio and transcripts in third-party clouds far outside your vendor due diligence.
• Ops & compliance – KYC/AML documents and contract packets run through unapproved OCR + AI extraction tools hosted who-knows-where.

Meanwhile, IBM’s 2024 data shows the average financial-industry breach costs around $6.08M, roughly 22% higher than the global average. (IBM)
Layer shadow AI on top of that, and you’re adding vendors and data flows your risk team doesn’t even know to assess.

In healthcare

• Clinicians – Dictation into consumer-grade transcription or “AI scribe” apps that have no BAA, no clear data residency, and unclear reuse of PHI.
• Admin & rev cycle – Eligibility checks, pre-auth letters, and discharge summaries pasted into general-purpose chatbots to simplify patient-facing language.
• Research teams – “De-identified” datasets fed into external AI tools that haven’t been vetted for re-identification risk.

Healthcare is already the costliest vertical for breaches. In 2024, the average healthcare breach cost hovered around $9.7–9.8M per incident, and recent summaries still put healthcare at or near the top of the list. (Elliott Davis)

Now add HIPAA civil penalties that can run up to $2.1M per year per violation category, plus potential state AG actions and class actions. (The HIPAA Journal)
Feeding PHI into unapproved AI isn’t a minor policy violation. It’s potentially regulatory kindling.

Why this is a board-level risk, not just “IT’s problem”

From a board’s point of view, shadow AI lands in three buckets: financial, regulatory, and workforce integrity.

1. Financial impact

Let’s start with the money.

• The global average cost of a breach sits around $4.4–4.9M per incident depending on the year and methodology. (IBM Newsroom)
• In the U.S. specifically, recent IBM data puts the average breach at about $10.22M, the highest in the world. (Baker Donelson)
• IBM’s 2025 analysis suggests that when shadow AI is involved, you can tag on hundreds of thousands of dollars in additional costs (roughly +$670K on average) from extra forensics, difficult containment, and more complex vendor risk chains. (IT Pro)

Those numbers are before you count:

• Civil penalties and regulatory settlements (HIPAA, SEC disclosure issues, state privacy laws). (The HIPAA Journal)
• Class action settlements and compensation for impacted individuals—multi-million dollar payouts are quickly becoming normal. (The Sun)
• Business disruption and churn, which IBM notes are now among the largest contributors to breach costs globally. (IBM Newsroom)

For tech, finserve, and healthcare, these are not “IT incidents.” They’re earnings events.

2. Regulatory & audit exposure

Regulators don’t care whether the thing that leaked data was called a “chatbot,” a plug-in, or a browser extension. They care about:

• Who accessed what.
• Whether you had reasonable controls.
• Whether your real practices match your written policies.

Shadow AI creates policy drift: the more your people rely on unsanctioned tools, the further your actual operations drift from your stated governance. That gap is where regulators, auditors, and litigators go hunting.

3. Workforce integrity

Shadow AI is also a people problem.

When your best people feel they need to break policy just to do their job efficiently, you end up with:

• A culture where “the real work happens off the books.”
• Managers quietly tolerating risky behavior because the sanctioned tools are too slow or too locked down.
• A widening trust gap between the workforce and the functions that are supposed to be enabling them (IT, risk, compliance).

That’s workforce integrity. And if you’re not addressing it, you’re training your people to optimize for speed over safeguards.

Why banning AI tools doesn’t work

A lot of organizations tried to solve this with simple rules: “No AI tools,” “No external chatbots,” “Blocked domains.”

The data says that doesn’t work:

• Average enterprises are now using dozens of generative AI tools—one report puts it around 67 per company, with roughly 90% unlicensed or unapproved. (Axios)
• Employees reach for shadow AI precisely because they don’t have sanctioned options that match their workflow. When the choice is “break policy and finish in 10 minutes” or “follow policy and finish in 2 hours,” most people will choose the former. (TechRadar)

You will not win this with firewalls alone. You have to give people safe ways to get the same benefits.

Which brings us to on-device AI.

On-device AI as a design pattern, not a product logo

When I talk about “on-device” or “local” AI here, I’m talking about:

Models running directly on the employee’s laptop or desktop, using the device’s CPU, GPU, and—where available—NPU, without shipping raw data off to a vendor’s cloud.

Done right, this gives you a way to shrink your AI attack surface without asking people to go back to manual work.

Cephable does a lot of work with our partners, especially Intel, to optimize for their GPU and NPU enabled machines from OEMs like HP, Lenovo and Dell. The chipset Intel provides is an investment, sometimes future-proofing, but in reality the future is here when you can offload workloads to the local chip and protect the business at the same time.

Why on-device matters for shadow AI

1. Data stays on the device
   Summaries, rewrites, translations, and quick-generation tasks happen on the endpoint. Sensitive inputs—PHI, MNPI, customer PII, internal financials—never need to leave the network or hit a third-party API.
2. You can finally align AI with your data classifications
   Instead of a blanket “no external AI” rule, you can say:
• “These data classes (PHI, cardholder data, MNPI) must use local AI only.”
• “These classes can use private cloud.”
• “These low-risk classes can use approved public services.”
3. You’re planning for, not against, hardware refresh cycles
   A practical on-device strategy doesn’t assume everyone already has an “AI PC.” It should:
• Run acceptably on current CPUs/GPUs across your existing fleet.
• Accelerate when NPUs are present, so your AI experience naturally gets faster as you refresh devices.
• Avoid hard dependencies that mean “no AI for you” if the machine is more than 18 months old.
4. You leverage the hardware you already paid for
   In most enterprises, endpoint hardware is a sunk cost line item. Using local compute for AI shifts part of the workload away from metered cloud calls and into the machines your CFO already funded.
5. You keep a single security and governance perimeter
   If work is happening on sanctioned endpoints under MDM, EDR, and logging, you at least know where to look during an incident. Shadow AI scatters that work across random SaaS vendors with unknown controls and opaque training practices.

On-device AI isn’t a silver bullet. You still need good governance. But it lets you design AI into your environment on your terms, instead of reverse-engineering whatever tools your employees happen to like this quarter.

Cloud AI and on-device AI: which work belongs where?

This isn’t a turf war between cloud and local. You need both. The trick is matching workload to risk.

Here’s a simple way to think about it:

Cloud AI (when egress risk is acceptable)

• Public-facing content, marketing copy, generic research.
• Non-sensitive code assistance and documentation.
• Use cases where the speed and scale of large hosted models create clear value.

Guardrails: strong vendor due diligence, explicit data processing terms, logging, and clear allowed data classes.

On-device AI (when data must not leave)

• Clinical documentation, care notes, and internal patient communications in healthcare.
• Trading strategies, internal risk reports, pricing models, and deal memos in finance.
• Proprietary source code, detailed architecture diagrams, internal product roadmaps in tech.
• Any workflow where an accidental copy-paste into a random chatbot would qualify as a potential breach.

Guardrails: run only on managed endpoints; tie into your identity, logging, and device compliance posture.

Hybrid

Some workflows—like meeting summarization, claims intake, or underwriting—will mix data classes. The long-term direction for many organizations is:

• Local AI as the first touch on sensitive data.
• Optional, governed promotion of outputs (not raw inputs) into cloud AI services for additional enrichment, once you’re confident the content is safe to share.

The cost of not addressing shadow AI

If you choose to look the other way—keep AI in the “innovation” bucket and ignore the shadow reality—you’re effectively betting on four things:

1. No one will ever paste the wrong thing into the wrong tool.
   We already know that’s not true; 75% of shadow-AI users admit they’ve shared sensitive data. (TechRadar)
2. If a breach happens, it’ll be cheap.
   The trend line says otherwise:
• Global averages around $4.4–4.9M per incident. (IBM Newsroom)
• U.S. breaches averaging $10.22M. (Baker Donelson)
• Healthcare breaches pushing $7–10M on average. (Elliott Davis)
3. Regulators will treat AI-related leaks as “new and confusing.”
   They won’t. HIPAA, GLBA, SOX, SEC disclosure rules, and state privacy laws already cover unauthorized disclosure and insufficient safeguards. AI doesn’t buy you leniency; it just adds complexity. (The HIPAA Journal)
4. Your people will keep trusting your policies.
   The longer you leave shadow AI as the path of least resistance, the more you train your workforce to treat policy as optional. That’s hard to unwind.

When you add it up, the cost of inaction looks like:

• Multi-million-dollar breach response and recovery.
• Ongoing audit scrutiny and corrective action plans.
• Talent burnout and quiet cynicism (“We say we care about security, but everyone knows the real work happens elsewhere.”)
• Opportunity cost: time spent cleaning up ungoverned AI instead of deliberately compounding value from governed AI.

Where to start in the next 90 days

If you’re a senior leader in tech, finserve, or healthcare, here’s a pragmatic starting play:

1. Inventory reality, not intentions
   Use network logs, CASB tools, and simple surveys to map:
• Which AI tools are actually in use.
• Which teams are using them.
• What kinds of data they’re touching.
2. Classify your AI workloads by sensitivity
   Don’t start with tools; start with work:
• What are the core “digital chores” in your org—summarizing, rewriting, translating, documenting, drafting?
• For each, what’s the highest data classification it touches?
3. Design a split: local-first vs cloud-eligible
   Decide:
• Which workloads must be local-only because of PHI, MNPI, or regulatory exposure.
• Which can use private cloud (behind your firewall or in tightly controlled environments).
• Which can use public cloud services with the right guardrails.
4. Pilot on-device AI on existing laptops/desktops
   Pick a few teams (for example: a clinical unit, a research pod, an analyst group, or a product team) and:
• Run a local-only AI pilot focused on repetitive “digital chores.”
• Measure reductions in copy-paste into unapproved tools, time saved per task, and user satisfaction.
• Validate that the experience works on your current hardware mix (CPU/GPU), while taking advantage of NPUs where you have them.
5. Update governance so people can do the right thing by default
• Make AI policies readable and concrete: “For X-type data, use Y-type AI in Z place.”
• Create a clear path for employees to request new AI capabilities without going rogue.
• Ensure risk, compliance, and security are part of the design—not the department that says “no” after the fact.

Closing: This isn’t about being anti-cloud; it’s about being pro-control

Cloud AI isn’t going away, and it shouldn’t. It will continue to be the right answer for plenty of workloads.

But if you don’t pair it with a deliberate on-device AI strategy on the hardware you already own, you’re leaving the door open for your workforce to solve their own problems with shadow AI—and to drag your sensitive data right along with them.

If your end-of-year reviews are surfacing the same concerns—shadow AI, data leakage, and nervous questions about regulatory exposure—it’s a good moment to revisit how you’re balancing cloud and local AI inside your walls.

I’m always happy to compare notes on what I’m seeing across tech, financial services, and healthcare. Whether that’s a conversation that starts on our website’s contact form or a LinkedIn message, the important thing is that it starts—before your “AI strategy” is defined by the next unapproved download.